Docker build files: the context of the RUN command

Docker is a game changing tool that is simplifying server dependency management in a wide variety of applications. For many of these applications simply spinning up a new container and installing a few things may be sufficient. Once the container is completed and tested you commit it to an image, push the image to your Docker repo, and you’re ready to pull and run it anywhere.

If your image has a lot of working parts and a more complicated install, however, this workflow is probably not good enough. That’s where a DockerFile comes in. A DockerFile is basically a makefile for Docker containers. You use a declarative syntax to specify the base image to build from, and the steps to take to transform it into the image that you want. Those steps usually include executing arbitrary shell statements using the RUN command.

The format of the RUN command is simply “RUN the-command-text”. Initially you might be tempted to look at RUN as essentially a shell prompt, from which you can do anything you would at an interactive shell, but that isn’t quite the way things work. For example, have a look at this minimal DockerFile:

# A minimal DockerFile example
FROM my-image
RUN mkdir /home/root/test
RUN touch /home/root/test/test.txt
RUN cd /home/root/test
RUN rm test.txt

This seems pretty straightforward: start with the base my-image, then create a directory, create a file in that directory, cd into that directory, and finally remove the file. If we try to execute this file using “docker build”, however, we get the following output:

Uploading context 2.048 kB
Uploading context 
Step 1 : FROM mn:saucy-base
 ---> 69e9b7adc04c
Step 2 : RUN mkdir /home/root
 ---> Running in d4802792515c
 ---> 4be0a443060a
Step 3 : RUN touch /home/root/test.txt
 ---> Running in 27aee53a2a17
 ---> b67284690b98
Step 4 : RUN cd /home/root
 ---> Running in 58d5fedeee98
 ---> 3a5826ad206c
Step 5 : RUN rm test.txt
 ---> Running in 02f11782a5e7
rm: cannot remove 'test.txt': No such file or directory
2014/01/31 15:11:34 The command [/bin/sh -c rm test.txt] returned a non-zero code: 255

The reason why the rm command was unable to find test.txt is hinted at by the output above the error. In particular, note the following:

Step 4 : RUN cd /home/root
 ---> Running in 58d5fedeee98
 ---> 3a5826ad206c
Step 5 : RUN rm test.txt
 ---> Running in 02f11782a5e7

Every instance of the RUN command that Docker processes gets applied to a new container that resulted from the changes created by the previous command. That’s what “Running in 58d5fedeee98” tells us. The command is being executed in the container with that ID, which is clearly different from the ID of the container in which the next command runs.

What this means is that the context of each RUN command is essentially a new instance of the shell, and any previous non-persistent changes like setting the current working directory are lost. The following revised DockerFile shows one way around this issue:

FROM my-image
RUN mkdir /home/root/test
RUN touch /home/root/test/test.txt
RUN cd /home/root/test;rm test.txt

Now the command that sets the working directory and the command that removes the file execute in the same context. If we re-run the build command we get the following output:

Uploading context 2.048 kB
Uploading context 
Step 1 : FROM mn:saucy-base
 ---> 69e9b7adc04c
Step 2 : RUN mkdir /home/root
 ---> Running in 633dd0266b8e
 ---> 7b2a80409513
Step 3 : RUN touch /home/root/test.txt
 ---> Running in d8122e2fb2ec
 ---> 70d091a60051
Step 4 : RUN cd /home/root;rm test.txt
 ---> Running in 68589850d97c
 ---> b88df827ad5f
Successfully built b88df827ad5f

One other quick note: when you build a container from a DockerFile containing many steps, a lot of intermediate containers are generated. The way to avoid having to manually delete them is to use the -rm flag to build:

sudo docker build -rm=true - < DockerFile

This will remove all the intermediate containers, as long as the script completed successfully. If any of the commands in the script failed, then it will leave all those containers behind. In that case, the easy way to get rid of them is:

sudo docker rm $(sudo docker ps -a -q)

Thanks to Dan Sosedoff for the tip.

Leave a Reply

Your email address will not be published. Required fields are marked *