Running multiple (Flask) websites with a single Docker setup
By sharing the code and (optionally) templates we avoid making copies and reduce maintenance time.
I developed one Flask website on Docker but after some time when my code became more stable I wanted to use the same setup for other websites. For one website I made a copy because it had to work yesterday. But what I really wanted was to share all the code, and some of templates. Of course every website has its own static directory, logging directory, templates, etc. The main reason for sharing is that the code still is under heavy development and I do not want to maintain multiple copies of the same code.
Extending my Docker build and start script
I wrote an interactive script that I use to build the Docker images and to start the Docker container with the proper version. With these I can run the development, testing, and staginglocal, and staging and production, containers even parallel. I just open another terminal window and run my Docker start script. I had to this script to support multiple websites. In the main directory there are two subdirectories:
Docker-templates holds the Docker-compose templates when creating a new version for a website. A version always means a new set of docker-compose files and file with environment variables. In addition, for staging and production, it will also contain a (tarred) Docker image and optionally a static directory and database dump.
The Docker build script takes the files from the proper docker-templates directory, generates a new version number, patches the new version number in the files, creates the new version directory in docker-versions, and puts the files there. This setup allows me to make changes to Docker-compose and environment files in docker-templates. New versions include the latest changes and older versions remain unchanged. For development and testing all application files are outside the container meaning that generating new image is only necessary when adding or removing packages (requirements.txt). The new directory structure:
. |-- docker-templates | `-- sites | |-- peterspython | | |-- docker-compose_development.yml | | |-- docker-compose_production.yml | | |-- docker-compose_shared.yml | | |-- docker-compose_staginglocal.yml | | |-- docker-compose_staging.yml | | |-- docker-compose_testing.yml | | |-- env_development | | |-- env_production | | |-- env_staging | | |-- env_staginglocal | | `-- env_testing | `-- anothersite | |-- docker-compose_development.yml | |-- docker-compose_production.yml | |-- docker-compose_shared.yml | |-- docker-compose_staginglocal.yml | |-- docker-compose_staging.yml | |-- docker-compose_testing.yml | |-- env_development | |-- env_production | |-- env_staging | |-- env_staginglocal | `-- env_testing |-- docker-versions | `-- sites | |-- peterspython | | |-- 1.821_development | | | |-- deployment.env | | | |-- docker-compose_deployment.yml | | | `-- docker-compose_shared.yml | | | ... | | |-- 1.829_production | | | |-- deployment.env | | | |-- docker-compose_deployment.yml | | | |-- docker-compose_shared.yml | | | `-- peterspython_web_production_image_1.829.tar | `-- anothersite | |-- 1.778_development | | |-- deployment.env | | |-- docker-compose_deployment.yml | | `-- docker-compose_shared.yml | `-- 1.779_development | |-- deployment.env | |-- docker-compose_deployment.yml | `-- docker-compose_shared.yml |-- project
The Docker start script allows me to select a website (peterspython, anothersite) and then presents me a menu of all versions. When I select a version, a menu is presented with a list of actions like start a container, exec into a container, etc. Before starting an action, the Docker start script copies the files from the docker-version directory to the main directory so that we always have the files in the same place as if we were running a single site.
Note that this setup does not conflict with how Docker works. After we start a Docker container with Docker-compose, we can remove the docker-compose and environment variables files. When you restart Docker at a certain moment, it simply restarts the containers that were running before without needing these files.
Extending the project directory structure
I wanted to share all code between the sites but want them to have their own templates, static directory, log files, etc. The new project directory structure:
. |-- app | |-- blueprints | |-- factory.py | |-- services.py | `-- templates |-- sites | |-- peterspython | | |-- docker-volumes | | |-- static | | |-- stylesheets | | `-- templates | `-- anothersite | |-- docker-volumes | |-- static | |-- stylesheets | `-- templates | |-- app_run.py |-- Dockerfile |
The code in app is shared. Docker-volumes is a volume mount for log files etc. A difference with the single site setup is the location of the Flask configuration file config.py. I am loading this file in create_app() using the from_object() method. To make sure the file can be found I add the directory of the file, specified by the Docker environment variables, to sys.path:
def create_app(deploy_config): ... # to find config.py sys.path.insert(0, project_docker_volume_config_dir) # load config app.config.from_object(all_configs[deploy_config])
Another difference with the single site setup are the templates. I want it to work as follows. When the site directory does not contain a template then it falls back to the app template directory. It is very easy to tell Jinja where to look for templates. Jinja supports many loaders, I went for the ChoiceLoader:
from jinja2 import ( ChoiceLoader, ) def create_app(deploy_config): ... # set template paths template_loader = ChoiceLoader([ # site templates app.jinja_loader, # default templates FileSystemLoader('/home/flask/project/app/templates') ]) app.jinja_loader = template_loader ...
Here I presented a way how to share the code and (optionally) templates between multiple sites without making a copy of the whole directory tree for a new site. Although there are a lot of things that can be improved, I am using this and am happy I took the time to implement this.
Links / credits
- Using PyInstaller and Cython to create a Python executable
- IMAPClient and flattening the BODYSTRUCTURE
- Blocking unsafe resources in HTML email using BeautifulSoup
- Python Multiprocessing graceful shutdown in the proper order
- FastAPI + SQLAlchemy: Asynchronous IO and Back Pressure
- Connect two Docker containers having their own Docker Compose files
- Using UUIDs instead of Integer Autoincrement Primary Keys with SQLAlchemy and MariaDb
- SLQAlchemy dynamic query building and filtering including soft deletes
- SQLAlchemy server-side datetime calculations
- Threaded comments using Common Table Expressions (CTE) for a MySQL Flask blog or CMS
- Python Multiprocessing graceful shutdown in the proper order
- Adding url_for() links to Jinja templates of a Flask multilanguage website