From Docker-Compose to Docker Swarm: Configs
Docker Configs let's Docker Swarm manage our (static) configuration, no shared volume storage required.
You have an application that consists of a number of Docker-Compose projects, and are using Docker-Compose to build, start and deploy.
Now you want to go one step further and move some of the Docker-Compose projects to another server. The most obvious choice to do this, well to try first, is Docker Swarm. You learn a few more Docker commands and you're done. Is it really that easy?
Spoiler alert. No, it's not easy if you have containers that use volumes, for example, for configuration data, high performance data, persistent data. Many examples on the internet avoid this issue completely, or casually mention using shared storage solutions like NFS (not encrypted) or GlusterFS, or use storage solutions from cloud providers. All these storage solutions all have the same feature: They are network-based and very slow compared to native storage. Yes, you can go faster but you will pay more.
Before you even consider Docker Swarm, I recommend that you read about this issue to understand if your use case is suitable for Docker Swarm. For example here is a discussion about this, 'Data(base) persistence in docker swarm mode', see links below. And from this Stackoverflow answer 'How does Docker Swarm implement volume sharing?', see links below:
Swarm Mode itself does not do anything different with volumes, it runs any volume mount command you provide on the node where the container is running. If your volume mount is local to that node, then your data will be saved locally on that node. There is no built in functionality to move data between nodes automatically.
This means it is possible to prepare a directory structure on a worker node, and make sure you run a container only on this node using the deploy placement parameters. Options ...
In this post we use Docker Swarm Configs to pass (static) configuration data to containers on worker nodes. This is a bit of an anti-pattern, but the nice thing is that we eliminate a storage requirement.
As always I am running on Ubuntu 22.04. At the moment I am not using Docker Desktop. I used VirtualBox to create a Ubuntu server VM and use it as a worker.
Docker Swarm does not manage volume data!
Docker Swarm is a services manager and services are containers (based on images), not volume data. This means you must manage the volume data yourself.
As an example, consider a DSN server container. By default, it connects to DNS service A on the internet, no external data here.With Docker Swarm, we can deploy this container, specify replicas, easy does it.
Now, we want the same DSN server container to connect to DNS service B on the internet. We follow the instructions of the container maintainer, create two (configuration) files, create a volume mapping, and we're done.
Part of the Docker-Compose file:
...
services:
dns_service:
image: ...
volumes:
- ./config/definitions1.conf:/opt/dns_server/config/definitions1.conf:ro
- ./config/definitions2.conf:/opt/dns_server/config/definitions2.conf:ro
...
But now we have a problem with Docker Swarm because it does not manage our volume data! If we deploy the service on a worker node, the data, the configuration files, is not there!
'Baking' configuration data into the service
When using Docker Swarm, we have no data in our volumes ... if we do nothing. There is a way around this, called 'Configs', not available in Docker-Compose. We can use this to specify files that will be mounted into the container like we can do with volumes. We look first at the Docker-Compose solution and then change this to Docker Swarm.
Using Docker-Compose
When using Docker-Compose, the 'docker-compose.yml' file for our DNS server project, see above,
uses volumes and looks something like:
# docker-compose.yml, using volumes
version: '3.7'
services:
app:
image: busybox_volumes
build: .
volumes:
- ./config/definitions1.conf:/opt/dns_server/config/definitions1.conf:ro
- ./config/definitions2.conf:/opt/dns_server/config/definitions2.conf:ro
command: sh -c "ls -l /opt/dns_server/config && cat /opt/dns_server/config/definitions*.conf && tail -f /dev/null"
We add a 'Dockerfile':
# Dockerfile
FROM busybox
And build the image:
docker-compose build
And then we run it:
docker-compose up
The text printed to the screen is:
app_1 | total 8
app_1 | -rw-rw-r-- 1 1000 1000 14 Aug 29 08:06 definitions1.conf
app_1 | -rw-rw-r-- 1 1000 1000 14 Aug 29 08:06 definitions2.conf
app_1 | definitions=1
app_1 | definitions=2
This is as expected. But if Docker Swarm would deploy this service to another host, a worker node, our files would be missing.
Using Docker Swarm
Here we remove the 'volumes' section and add the new 'configs' section:
# docker-compose.yml, using configs
version: '3.7'
services:
app:
image: busybox_configs
build: .
configs:
- source: definitions1
target: /opt/dns_server/config/definitions1.conf
- source: definitions2
target: /opt/dns_server/config/definitions2.conf
command: sh -c "ls -l /opt/dns_server/config && cat /opt/dns_server/config/definitions*.conf && tail -f /dev/null"
configs:
definitions1:
file: ./config/definitions1.conf
definitions2:
file: ./config/definitions2.conf
Build the image:
docker-compose build
And then we deploy:
docker stack deploy -c docker-compose.yml configs
The result of the deploy command shows the creation of the 'configs':
Creating network configs_default
Creating config configs_definitions2
Creating config configs_definitions1
Creating service configs_app
Nothing is printed to the screen, to see what is going on we look at the log of this service:
docker service logs configs_app
The result:
configs_app.1.vqumgq21leq8@myra | total 8
configs_app.1.vqumgq21leq8@myra | -r--r--r-- 1 root root 14 Aug 29 09:58 definitions1.conf
configs_app.1.vqumgq21leq8@myra | -r--r--r-- 1 root root 14 Aug 29 09:58 definitions2.conf
configs_app.1.vqumgq21leq8@myra | definitions=1
configs_app.1.vqumgq21leq8@myra | definitions=2
This means that the Docker Swarm manager correctly added our files.
In my case the service was deployed on the Docker Swarm manager node, not on the worker node. To run it on the worker node, we add the following to 'docker-compose.yml':
deploy:
placement:
constraints:
- node.role == worker
replicas: 1
Tag your image and push it to your Docker registry.
Only if you are using a private registry
If you are using a private, not secure, registry, you must tell all Docker nodes it is not secure by adding this information:
{
"insecure-registries":[
"<your registry ip address>:<your registry port>"
],
}
to the file:
/etc/docker/daemon.json
Restart Docker once you have done this:
sudo systemctl restart docker
Important: You must do this on all nodes.
Now, edit your 'docker-compose.yml' file to tell Docker Swarm where it can get your image.
From:
...
services:
app:
image: busybox_configs:latest
...
To:
...
services:
app:
image: <registry host>:<registry port>/busybox_configs:latest
...
Deploy the service to the worker node
Remove the service from this node, if it is still there:
docker stack rm configs_app
And deploy again:
docker stack deploy -c docker-compose.yml configs
Now you can check that the service is running on the worker node:
docker service ps configs_app
The result contains the node, see 'NODE':
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
xotczxp65lub configs_app.1 ... vmubs2204a Running Running 9 minutes ago
You can get more information for example with command:
docker service inspect --pretty configs_app
Oh, and the most important, what did our service exactly do? Nothing is printed to the screen, we must look at the logs:
docker service logs configs_app
The result is exactly the same as before. Meaning that Docker Swarm not only deployed our our service (container), but also made our 'configs' files available to our container running on the worker node. Done.
Summary
In this post, we used Docker Configs to pass (static) configuration data to containers. In 'docker-compose.yml', we removed the 'volumes' section and replaced it with the 'configs' section. Now Docker Swarm manages this data and we do not need a shared storage volume for this purpose. The annoying thing is that 'configs' does not work in Docker-Compose, which means you are moving away from a single 'docker-compose.yml' that can be used by Docker-Compose and Docker Swarm.
Links / credits
Data(base) persistence in docker swarm mode
https://forums.docker.com/t/data-base-persistence-in-docker-swarm-mode/20665
Deploy a stack to a swarm
https://docs.docker.com/engine/swarm/stack-deploy
Docker - configs
https://docs.docker.com/compose/compose-file/compose-file-v3/#configs
Docker Swarm Tutorial | Code Along | Zero to Hero under 1 Hour
https://takacsmark.com/docker-swarm-tutorial-for-beginners
How does Docker Swarm implement volume sharing?
https://stackoverflow.com/questions/47756029/how-does-docker-swarm-implement-volume-sharing
Read more
Docker Docker-compose
Most viewed
- Using PyInstaller and Cython to create a Python executable
- Using Python's pyOpenSSL to verify SSL certificates downloaded from a host
- Reducing page response times of a Flask SQLAlchemy website
- Connect to a service on a Docker host from a Docker container
- Using UUIDs instead of Integer Autoincrement Primary Keys with SQLAlchemy and MariaDb
- SQLAlchemy: Using Cascade Deletes to delete related objects