- SQLAlchemy PostgreSQL: Add a second BigInteger Primary Key
- Redirect on an exception in Flask using a decorator
- SQLAlchemy Many-To-Many: Four ways to select data
- Testing the RabbitMQ Pika publishing examples
- An attempt to solve Tic-Tac-Toe using Keras and LSTM
- LSTM multi-step hyperparameter optimization with Keras Tuner
- Using Python's pyOpenSSL to verify SSL certificates downloaded from a host
- Using UUIDs instead of Integer Autoincrement Primary Keys with SQLAlchemy and MariaDb
- Flask RESTful API request parameter validation with Marshmallow schemas
- SLQAlchemy dynamic query building and filtering including soft deletes
- Migrating from Bootstrap 4 to Bootstrap 5
- Documenting a Flask RESTful API with OpenAPI (Swagger) using APISpec
28 May 2022 0
Suppose we use UUID Primary Keys in our project. This works fine, but there are also a few tables with millions of records. We know that SELECT and INSERT operations with UUID-based Primary Key tables can be much slower compared to Integer based Primary Key tables. What we want is for these huge tables to have only BigInteger Primary Keys and link them to other UUID primary key based tables, ...
7 May 2022 0
In a Flask application, you typically implement global exception handlers. In many cases, this is sufficient. But what if you want more control? In one project, I was connecting to an API and I wanted a number of routes that used the API to redirect to a 'start' page in case of an API error, with an appropriate message of course. I implemented this using a 'redirect_decorator' exception handler that ...
12 April 2022 0
This is a short post about Many-To-Many selection with SQLAlchemy. In the past I used the association (link) table in ORM queries, because I thought it must be fastest. This time I did a small test comparing various ways to select data. Spoiler: The association (link) table way is (of course) fastest. The model We have a Many-To-Many relationship between Orders and Products. # link table: order ...
25 March 2022 0
A lot has been written about synchronous vs asynchronous publishing with RabbitMQ, see links below, I am not going to repeat this here. As this is my first time using RabbitMQ, I wanted to try both the synchronous and asynchronous publishing versions, using examples included with Pika, the RabbitMQ (AMQP 0-9-1) client library for Python. To my surprise, the Pika asynchronous example did not detect ...
2 March 2022 0
After implementing my first Deep Learning LSTM model for a project I was thinking if Deep Learning could also solve a game. The first game that comes to mind is Tic-Tac-Toe. Then you search the internet and there appear to be many many people who had the same idea. Of course. Below I present my solution to solve Tic-Tac-Toe using Keras and LSTM (Long Short Term Memory). This is Deep Learning and not ...
13 February 2022 0
A previous post was about Hyperparameter optimization with Talos. I could not get this to work with my LSTM model for univariate multi-step time series forecasting, because of the 3D input, so I switched to Keras Tuner. In this post I try to predict the next period of a sine wave using the Hyperband tuning algorithm. To reduce tuner time, I reduced the number of hyperparameters that can be optimized ...
6 February 2022 0
For a project I was looking for a way to match an incoming sentence with a list of fixed sentences. This is a complex subject but when searching the internet I bumped on the amazing project Sentence-Transformers. I modified one of the examples on this site to use a text file with sentences and typed some input sentences as a demo. Then I downloaded a text file from the internet 'Choosing a Cat, by ...
2 February 2022 0
In the two previous posts I showed you my first steps with Keras. I used examples found on the internet and changed the dataset into something trivial, meaning I generate the data myself and know the expected values. But I also told you that I had no idea why parameters like neurons, epochs, batch_size had these values.So what we have is not really a black box. On the outside there are also some switches ...
30 January 2022 0
Many problems are time related. We have collected some samples and now want to use them to predict the next value. That is what this post is about. It is not about predicting many future values, that is a different topic.As a data scientist noob, I just work through some examples I found on the internet. I change the input sequence and see what happens. You might find this useful. I found the 'Machine ...
28 January 2022 0
I have a dataset, many rows with N inputs and 1 output, and want to predict the output value for any new combination of input values. I am also a data science noob, but stories on the internet about Deep Learning suggest we can easily create some kind of black box with some neurons, nodes, in it, and then use the dataset to train the black box. After this we can feed any inputs and our black box gives ...
8 December 2021 0
I have a Python program running fine on Linux. Few months ago I wanted to run this on Windows. This was the first time I used Python on Windows. Install Python app, create virtual environment, copy and run. No problems ... oh but there was a problem. My session sometimes disappeared ... WTF! I noticed the problem by repeatly hitting F5 in a very short time. Time for a thorough investigation. The ...
27 November 2021 0
I was implementing some charts with Chart.js and needed to pass colors from my Flask application to the template. The graph is a line graph and the number of lines can vary. With the graph I also show a table with the (numeric) values used in the graph. The lines have different colors, I choose them from a list of colors. The table of line values has a column for each line. I want the header cell ...
6 October 2021 0
You have created a Python application and want to distribute it. You probably have it running in a Python Virtual Environment. But customers don't have this setup, some may not even have Python installed. There are several programs that can convert your Python application into a single executable file. Here I am using PyInstaller. You may also want to protect some of your code and/or speed up some ...
27 September 2021 0
Application developers want to use proven solutions to create an application. Many times this works but with the IMAPClient package there are a number of things missing. The whole idea of IMAP is to get only what you request. Suppose you have an email with many attachments but you want to view or download only one of them. To be able to do this you need the 'body_number' of this attachment and then ...
30 August 2021 0
I have created an IMAP E-Mail Reader using IMAPClient and Flask. The IMAP E-Mail Reader decodes the email into valid HTML. Then it needs to display this HTML through the browser. Works fine, so far so good. In this post I describe how I implemented an option in my IMAP E-Mail Reader to block unsafe resources in the HTML. To do this, I use BeautifulSoup and Python's Regular expression operations. Why ...
16 June 2021 0
For a new project I needed a deamon process that must execute many, more or less identical, operations on different resources. In this case the operation is IO bound and I solved it by using ThreadPoolExecutor. So far, so good. Next I wanted to store the results in files. Of course we use a queue to communicate between processes. The worker() process uses q.put() to add items to the queue and the ...
4 June 2021 0
APIs are becoming more and more important. Companies want to share their data with customers. Or want to enable third parties to create applications based on their APIs. Few months ago I created an API with Flask, SQLAlchemy, Marshmallow and APISpec, it was not really difficult, and works fine. Then I read more about FastAPI, an API framework that also supports Python async out of the box. It is based ...
26 May 2021 0
I wanted to create a network between a database Docker container and an application Docker container, both having their own Docker Compose files. And I also wanted to make sure doing it right before implementing this in the actual docker-compose files. Test first baby ... I also did this some time ago but had to look into this again because Docker Compose now has a name option in networks section. ...
24 May 2021 1
I just completed my first FastAPI app. This app allows users to have their own items, meaning that the user data models all have a user_id field. Nothing special but as FastAPI introduces some things new to me, like Dependency Injection, I was unsure if my app would work the way I wanted. My question was: if I load test the API with many concurrent users, does it still work? Warning. Only run a load ...
22 April 2021 1
When you create an API, you want to document it and today it is obvious to use OpenAPI for this. I am already using the Marshmallow package. The same people also developed the package APIspec, with a Flask plugin in an additional package apispec-webframeworks. To present the OpenAPI documentation I use the package flask-swagger-ui and the package flask-jwt-extended is used to protect the endpoints. Below ...