tl;dr; To build once and run Docker containers with different files use a volume mount. If that's not an option, like in CircleCI, avoid volume mount and rely on container build every time.
Laugh all you like but after almost year of using Docker I'm still learning the basics. Apparently. This, now, feels laughable but there's a small chance someone else stumbles like I did and they might appreciate this.
If you have a volume mounted for a service in your
docker-compose.yml it will basically take whatever you mount and lay that on top of what was in the Docker container. Doing a volume mount into the same working directory as your container is totally common. When you do that the files on the host (the files/directories mounted) get used between each run. If you don't do that, you're stuck with the files, from your host, from the last time you built.
# Dockerfile FROM python:3.6-slim LABEL maintainer="firstname.lastname@example.org" COPY . /app WORKDIR /app CMD ["python", "run.py"]
#!/usr/bin/env python if __name__ == '__main__': print("hello!")
Let's build it:
$ docker image build -t test:latest . Sending build context to Docker daemon 5.12kB Step 1/5 : FROM python:3.6-slim ---> 0f1dc0ba8e7b Step 2/5 : LABEL maintainer "email@example.com" ---> Using cache ---> 70cf25f7396c Step 3/5 : COPY . /app ---> 2e95935cbd52 Step 4/5 : WORKDIR /app ---> bc5be932c905 Removing intermediate container a66e27ecaab3 Step 5/5 : CMD python run.py ---> Running in d0cf9c546fee ---> ad930ce66a45 Removing intermediate container d0cf9c546fee Successfully built ad930ce66a45 Successfully tagged test:latest
And run it:
$ docker container run test:latest hello!
So basically my little
run.py got copied into the container by the
Dockerfile. Let's change the file:
$ sed -i.bak s/hello/allo/g run.py $ python run.py allo!
But it won't run like that if we run the container again:
$ docker container run test:latest hello!
So, the container is now built based on a Python file from back of the time the container was built. Two options:
1) Rebuild, or
2) Volume mount in the host directory
This is it! That this is your choice.
Rebuild might take time. So, let's mount the current directory from the host:
$ docker container run -v `pwd`:/app test:latest allo!
So yay! Now it runs the container with the latest file from my host directory.
So, if it's more convenient to "refresh the files in the container" with a volume mount instead of container rebuild, why not always do it for everything?
For one thing, there might be files built inside the container that cease to be visible if you override that workspace with your own volume mount.
The other crucial thing I learned the hard way (seems to obvious now!) is that there isn't always a host directory to mount. In particular, in tecken we use a base ubuntu image and in the run parts of the CircleCI configuration we were using
docker-compose run ... with directives (in the
docker-compose.yml file) that uses volume mounts. So, the rather cryptic effect was that the files mounted into the container was not the files checked out from the git branch.
The resolution in this case, was to be explicit when running Docker commands in CircleCI to only do build followed by run without a volume mount. In particular, to us it meant changing from
docker-compose run frontend lint to
docker-compose run frontend-ci lint. Basically, it's a separate directive in the
docker-compose.yml file that is exclusive to CI.
I feel dumb for not seeing this clearly before.
The mistake that triggered me was that when I ran
docker-compose run test test (first
test is the docker compose directive, the second
test is the of the script sent to
CMD) it didn't change the outputs when I edit the files in my editor. Adding a volume mount to that directive solved it for me locally on my laptop but didn't work in CircleCI for reasons (I can't remember how it errored).
So now we have this:
# In docker-compose.yml frontend: build: context: . dockerfile: Dockerfile.frontend environment: - NODE_ENV=development ports: - "3000:3000" - "35729:35729" volumes: - $PWD/frontend:/app command: start # Same as 'frontend' but no volumes or command frontend-ci: build: context: . dockerfile: Dockerfile.frontend