angle-uparrow-clockwisearrow-counterclockwisearrow-down-uparrow-leftatcalendarcard-listchatcheckenvelopefolderhouseinfo-circlepencilpeoplepersonperson-fillperson-plusphoneplusquestion-circlesearchtagtrashx

Welcome

This website is a multi-language CMS / Blog I build with Python, the framework Flask and SQLAlchemy. It is deployed on my ISPConfig3 (Nginx + Mariadb) server using Docker which is also used for development. Read more about this website.

Coronavirus

I investigated the deaths in The Netherlands in 2020 and looking at the data, I can only wonder why we have never had, and still do not have, a serious discussion about reopening the economy for people under 65 or, for example, people under 50. Read the findings and conclusions.

Recent

Most viewed

Tags

AiohttpAJAXAlembicAlpineAPIApplication settingsAsyncBabelBeautifulSoupBootstrapCachingCaptchaCeleryColorsCompileCryptographyCSRF protectionCythonDecoratorDeep LearningDeepLDispatcherMiddleWareDistributionDNSDockerDocker-composeEmailExceptionsFastAPIFlaskGunicornIconsIMAPInternetISPConfigJavascriptJinja2Log fileMachine LearningMariaDBMessage brokerMultilanguageMultiprocessingPentestingPikaPostfixProfilingPyInstallerpyOpenSSLPytestRabbitMQRedisSecurity testingSQLAlchemyTestingThreadsTimezoneToastsUbuntu TouchUUIDWerkzeugWheelsWTForms

Python Multiprocessing graceful shutdown in the proper order

16 June 2021 0

Peter

post main image For a new project I needed a deamon process that must execute many, more or less identical, operations on different resources. In this case the operation is IO bound and I solved it by using ThreadPoolExecutor. So far, so good. Next I wanted to store the results in files. Of course we use a queue to communicate between processes. The worker() process uses q.put() to add items to the queue and the ...

FastAPI + SQLAlchemy: Asynchronous IO and Back Pressure

4 June 2021 0

Peter

post main image APIs are becoming more and more important. Companies want to share their data with customers. Or want to enable third parties to create applications based on their APIs. Few months ago I created an API with Flask, SQLAlchemy, Marshmallow and APISpec, it was not really difficult, and works fine. Then I read more about FastAPI, an API framework that also supports Python async out of the box. It is based ...

Connect two Docker containers having their own Docker Compose files

26 May 2021 0

Peter

post main image I wanted to create a network between a database Docker container and an application Docker container, both having their own Docker Compose files. And I also wanted to make sure doing it right before implementing this in the actual docker-compose files. Test first baby ... I also did this some time ago but had to look into this again because Docker Compose now has a name option in networks section. ...

Using Locust to load test a FastAPI app with concurrent users

24 May 2021 1

Peter

post main image I just completed my first FastAPI app. This app allows users to have their own items, meaning that the user data models all have a user_id field. Nothing special but as FastAPI introduces some things new to me, like Dependency Injection, I was unsure if my app would work the way I wanted. My question was: if I load test the API with many concurrent users, does it still work? Warning. Only run a load ...

Documenting a Flask RESTful API with OpenAPI (Swagger) using APISpec

22 April 2021 1

Peter

post main image When you create an API, you want to document it and today it is obvious to use OpenAPI for this. I am already using the Marshmallow package. The same people also developed the package APIspec, with a Flask plugin in an additional package apispec-webframeworks. To present the OpenAPI documentation I use the package flask-swagger-ui and the package flask-jwt-extended is used to protect the endpoints. Below ...

Flask RESTful API request parameter validation with Marshmallow schemas

30 March 2021 1

Peter

post main image When you build a RESTful API the first thing you do is define the status codes and error responses. RFC 7807 'Problem Details for HTTP APIs' specifies the minimum parameters you should return. If you did not looked into this, I suggest you do. Of course you often want to include more details about what went wrong. APIs are for developers and we do want to make it easy for them to understand why a call ...

Flask SQLAlchemy CRUD application with WTForms QuerySelectField and QuerySelectMultipleField

8 March 2021 0

Peter

post main image For a new Flask application using WTForms and SQLAlchemy, I had many relationships between tables and was looking for the easiest way to manage these tables. The most obvious choice is to use the QuerySelectField and QuerySelectMultipleField present in the wtforms-sqlalchemy package. Since I haven't used them before, I created a small application to play with. Below I show you the code (development ...

Migrating from Bootstrap 4 to Bootstrap 5

17 January 2021 0

Peter

post main image Nowadays, websites use a lot of CSS and Javascript. When you build websites, you can't test them all the time with the major browsers. Instead, you need a framework that supports all the major browsers. This framework needs to be well supported and that means only a few remain. For this site, I chose Bootstrap 4. This is not a coincidence. A few years ago I was looking for the best framework for a ...

Using Python's pyOpenSSL to verify SSL certificates downloaded from a host

17 December 2020 2

Peter

post main image While writing a script to check if websites correctly redirected to 'https:/www.' I thought to add some SSL certificate checks as well. This means I had to verify SSL certificates downloaded from a host. Is the certificate really for this website? Show me the expiration date. Is the certificate chain correct? And can we trust the certificate(s)? Initially I got stuck where many people got stuck because ...

Why your website canonical name must be 'www' (or 'app' or something else)

27 November 2020 2

Peter

post main image I know, there are many articles about this subject. But I thought it was useful to write a post about this because I did not know all the details. I assume your website can be accessed from the internet using a 'without-www' URL and a 'with-www' URL. This article is not about selecting a website URL for marketing purposes. Even if you are using a 'with-www' URL for your website, you can still communicate ...

Flask's SERVER_NAME, subdomains and 404 errors

25 November 2020 2

Peter

post main image This is a short post about Flask and the config variable SERVER_NAME. Like many developers I bumped into this at a certain moment, and I thought I share my story. Hopefully this will prevent headaches for some. My websites must be available by typing the following addresses in the browser: https://example.com = 'without-www', and, https://www.example.com = 'with-www' Whats more, for this ...

Flask, Celery, Redis and Docker

29 October 2020 0

Peter

post main image This is a post about how I use Docker and Docker-composer to develop and run my Flask website with Celery and Redis. There are many articles on the internet about this and if you are searching for them do not forget to search on Github.com. I just grabbed the bits and pieces and created my own setup. Before going into this I want to mention two other things that came up when adding Celery to Flask. The ...

Celery, Redis and the (in)famous email task example

10 October 2020 0

Peter

post main image Being able to run asynchronous tasks from your web application is in many cases a must have. One way to achieve this is to use Celery. There are many articles on the internet and some examples are given. I really liked Miguel Grinberg's posts about Celery. Very clear, thank you Miguel. But Miguel's post and most other articles are just an introduction. When you start with Celery you suddenly realize ...

Running multiple (Flask) websites with a single Docker setup

30 September 2020 0

Peter

post main image I developed one Flask website on Docker but after some time when my code became more stable I wanted to use the same setup for other websites. For one website I made a copy because it had to work yesterday. But what I really wanted was to share all the code, and some of templates. Of course every website has its own static directory, logging directory, templates, etc. The main reason for sharing is ...

Testing the Docker .dockerignore file patterns

22 September 2020 0

Peter

post main image This is a short post about something I wanted to do a long time ago and for which I finally found some time. It was not really a case of TL;DR (Too Long; didn't read), but more something like TLT;DR (Too Little Time; didn't read). As you probably know, the Docker .dockerignore file is used to prevent certain files and directories ending up in your Docker image. The Docker .dockerignore file is used ...

Using Python to get Postfix sent status for messages with a message-id

20 August 2020 0

Peter

post main image The problem: I have a website that sends emails. The emails (meta) are stored in a database before sending to the Postfix MTA. Every email the website sends has a unique message-id, something like  159766775024.77.9154479190021576437@peterspython.com. I want to check if the emails are actually sent and add this information to the email (meta) database records.  Postfix has statuses like: status=bounced ...

Functional testing a multilanguage Flask website with Pytest

25 July 2020 1

Peter

post main image Testing is hard work. It is totally different from creating or modifying functionality. I know, I also developed computer hardware, integrated circuits, hardware test systems. I wrote tests for CPUs, computer products, developed test systems. With hardware you cannot make mistakes. Mistakes may be the end of your company. Welcome to the wonderful world of software testing With software many things ...

IMAPClient and fetching body parts

26 June 2020 0

Peter

post main image I decided to temporarily shift focus from developing the software for my CMS / Blog to a smaller project. Main reason is that I hoped to learn new things about Python that are useful. I always wanted to have my own IMAP client software. Maybe my choice was also heavily influenced by some annoyance about the IMAP client Dekko2 for Ubuntu Touch, the OS of my mobile phone. I know that I should be happy ...

Must-read: Internet advertising is a company town by Trevor Jim

17 June 2020 0

Peter

post main image When searching for something on the internet you sometimes bump into an article that perfectly describes what you are already telling friends, colleagues for many years. Whether you are a publisher, a software developer, an advertising company, this Trevor Jim's article is a must-read. Here are the first lines: Internet advertising is a company town February 28, 2017 If you’ve watched a lot ...

Create your own Python custom exception classes tailored to your application

17 June 2020 0

Peter

post main image Using exceptions in Python looks easy but it is not. Probably you should study exceptions and exception handling before writing any Python code but TL;DR. There are examples on the internet, unfortunately most are very trivial. Anyway, I researched this and came up with some code I thought I share with you. Leave a comment if you have suggestions. What is an error and what is an exception, what is ...