Shi's blog: Using Docker to Create Matching Development Environments: A Trick for Sharing Host User Accounts With Containers

At our workplace, Docker is primarily used to create matching development environments. Since we work with various machine learning repositories that have different requirements for CUDA, PyTorch, and Python, it becomes challenging to accommodate all the requirements on the native environment. Therefore, we use Docker containers as the easiest way to meet these diverse needs. However, this also comes with an annoying issue where sharing the host user account with the container environment is not straightforward. As a result, any shared host file, once modified by the container using the root user, cannot be modified without sudo on the host.

We need to update files across the container boundry because the host environment typically has better IDEs like VSCode. Therefore, we prefer modifying files on the host, but build and run our projects in a container. However, when a script or command is executed within the container and modifies a mounted file, the file becomes unsavable on the host unless we reopen it with sudo privilege.

A common solution on the internet for this problem is mounting the relevant files and the host user ID. For example:

docker run --gpus all -it \
    --workdir="/home/$USER" \
    -v="/etc/group:/etc/group:ro" \
    -v="/etc/passwd:/etc/passwd:ro" \
    -v="/etc/shadow:/etc/shadow:ro" \
    -v="/etc/sudoers:/etc/sudoers:ro" \
    -e="DISPLAY=$DISPLAY" \
    -e "TERM=xterm-256color" \
    -v="/tmp/.X11-unix:/tmp/.X11-unix" \
    -v="/tmp/host_launcher_fifo:/tmp/host_launcher_fifo" \
    -v /home/${USER}:/home/${USER} \
    -u $(id -u ${USER}):$(id -g ${USER}) \
    <docker_image>

The solution that maps the host user to a container is useful, but it comes with a significant drawback. When running any commands requiring sudo privilege, such as sudo apt update, it returns an incorrect password error. This is perplexing because the command line has already mounted both the passwd and shadow files, which should contain all the password information.

If I recall correctly, this solution worked for me a few years ago, but it has failed to work in recent years. I never felt motivated to resolve the issue until today when it became very annoying.

At first, I tried googling for a solution, but nobody seemed to report the same incorrect password issue. The above solution was deemed as the correct way to map a host user to a container.

While looking for solutions, I came across a forum thread that gave me an idea about the root cause of the problem. The thread's author encountered a similar issue when running a CentOS container on a Fedora host. The explanation given in the thread is that the two distros use different methods to hash passwords. Even if the passwd and shadow files are shared, the container can't create the same hash as the algorithms used are different. In my case, my host is Ubuntu 22.04, and my container is 18.04. I theorized that the root cause of my problem might be the same.

To verify this quickly, I created a container using Ubuntu 22.04 to match my host OS. As expected, the above solution worked without any issues. Based on this, I modified my solution. First, when creating the Dockerfile, we need to install the sudo tool, add a new user using the same host USER_ID and USERNAME, add the user to the sudo group, and set this user's password to the same password used on the host:

FROM <docker_base_image>

ARG USER_ID=1000
ARG HOST_USER_PASSWD=111111
ARG HOST_USERNAME=shiy

...

RUN useradd -l -u ${USER_ID} ${HOST_USERNAME}
RUN usermod -a -G sudo ${HOST_USERNAME} && usermod --password $(echo ${HOST_USER_PASSWD} | openssl passwd -1 -stdin) ${HOST_USERNAME}

USER ${HOST_USERNAME}
WORKDIR /home/${HOST_USERNAME}

RUN id ${HOST_USERNAME}

To use the same host user in the container, we should set the USER_ID to be the same as the host user's ID, which can be obtained by running the id -u command. With this configuration, we don't need to map the passwd and shadow files to the container anymore, as the container can't understand them.

docker run --gpus all -it \
    --workdir="/home/$USER" \
    -e="DISPLAY=$DISPLAY" \
    -e "TERM=xterm-256color" \
    -v="/tmp/.X11-unix:/tmp/.X11-unix" \
    -v="/tmp/host_launcher_fifo:/tmp/host_launcher_fifo" \
    -v /home/$USER:/home/$USER \
    --user=$(id -u):$(id -g)  \
    --group-add="sudo" \
 <docker_image>

Instead, we can specify the current user with the --user argument and add the sudo group using the --group-add argument. This way, the container will use the same host user, and sudo will work as expected. Moreover, any files created within the container can be modified on the host as well.