Do not hesitate to reinvent the wheel if you want your software with open source components to live longer

10 March 2020 Updated 11 March 2020 by Peter
In Flask

The lifetime of many open source components can be short. Write your own if you want to make sure your software lives on.

post main image
https://unsplash.com/@neilsoniphotography

The problem: a customer wants an application with a certain functionality and want this yesterday. What you do is look for a plug-and-play solution, libraries and/or extensions. You tell your customer that you can do it, customer is happy, you do it. For example, you take Wordpress, select and configure some plugins. Problem solved, or is it?

Yes, the problem is solved but for how long? After some months, Wordpress is updated because of a vulnerability. Then suddenly some plugins or themes do not work anymore. Then PHP moves to PHP 7.4 and again some plugins or themes do not work anymore. In the end this forces you or your customer to run older, often vulnerable, versions of software. But this can also be your business model, updates. 

Big tech companies force change to eliminate competition

Developing software that lasts a couple of years is very difficult. Standards change often initiated by big tech companies. Of course they want to change standards. They call it improvement, they call it progress but it is simply essential for their business to add new features, and discontinue old ones. It is a perfect and legitimate way to eliminate competition.

For example look what Google is doing with Android. Every year they release a new version. And all the phone manufactures must follow. Is the new version better? Most of the time, no! They do big rewrites of certain parts of code so if you had tried to understand and got used to the internals of version X you knowledge is worth nothing when the next version is released. Yes, they also introduce new vulnerabilities, but these are just mistakes. A multi-billion software company of course does not have the resources to thoroughly analyze and test their software. Google also forced websites to use HTTPS. I know, for many non-tech people this is a good idea. But now many websites use certificates from Letsencrypt, another dependency. And Letsencrypt competitors are not allowed, thank you all!

Open source developers create change

I want to state that I am very happy with the availability of open source software. I respect the open source developers and communities very much. Without them, the software we are using today would be totally different. Open source software forces big tech to think about changes, to keep developing new versions, to keep ahead. In many ways open source developers create change often without them knowing it.

The lifetime of open source software is often shorter than commercial software

If you use open source software to build your applications you have many reasons to be very careful. As many people have stated, it is very difficult to make money with open source software and projects. Open source software often is developed by one or a few persons because they needed it for something and then share it with the world, in fact giving it away. Many developers hope that other people will get involved with their project but many times this is not the case. After some time the developer starts working on other projects, or gets a time-consuming job and does not have the time to maintain his software. You can call this the dark side of open source software.

Let's look at the software I use for this website

This website is build with Python 3, the framework Flask, few Flask extensions, SQLAlchemy, Bootstrap 4, jQuery, MariaDb. Will Python 3 survive the next five years? I hope so. Python 2 was a mistake because it missed essential functionality. The Python maintainers decided to make Python 3 not backwards compatible with Python 2. For this reason Python 2 should live on as a independent language from Python 3, but the community decided not to support it anymore. Not good but I can understand this. The main reason is that the community is not making money on supporting Python so they go for Python 3. Python 2 is a very good example what can go wrong with open source software.

Will Flask survive the next five years? I hope so but I am scared. The creator and main developer of Flask may want to spend his time on other projects the next years. Can other persons take over? I am not convinced. Flask extensions are another thing. They are typically developed by individuals. Flask was introduced in 2010 and already a large number of extensions have not been updated for years, what will be the fate of the extensions you are including with your product today?

The same is true for SQLAlchemy, a beautiful piece of software, ORM the way it should be. But what will happen when the creator decides to quit and do something else? Bootstrap is open source and they seem have a policy to move forward breaking previous versions. I am now on Bootstrap 4 but assume you were on Bootstrap 3 and had to migrate to Bootstrap 4. Not an easy task. Bootstrap already announced version 5, which will work without jQuery. I am looking forward to this migration ... NOT.

jQuery is very much used but will try to keep up with changes forced by big tech. I can only hope they have the capacity to do so. Projects like jQuery-UI and jQuery-Mobile are more or less dead. And then there is MariaDb. I believe they are here to stay for at least five years because they deliver a very good product. I also think that Oracle wants them to exist as they stimulate Oracle MySQL developments. MariaDb also sells support, makes money, very important.

Should you use open source software to develop your next application?

Is software from commercial organizations better? In many cases the answer is no. But you get support and it is essential for them to stay up-to-date. Still, I think it is a good choice to use open source components to create new software. But only if you first investigate thoroughly the lifetime problem of the components. And of course you must check the licensing before anything else.

Minimizing risky dependencies by writing your own components

Many years ago I took over a project that was written in Perl. When PHP arrived I build my own small framework and converted it to PHP. Because I used no libraries everything was under my control. Fortunately PHP appeared to be reasonably backwards compatible, meaning that it was not much effort to migrate to new versions. Many PHP frameworks came and went away but my PHP software is still running. But of course it only could survive because I was there. I often see on websites like Stackoverflow comments that you should not reinvent the wheel. But developing your own components is often NOT reinventing the wheel!

  • you can use knowledge from already existing similar projects
  • you can leave out bloat
  • you can add missing functionality instead of fighting someone elses code
  • you can skip hard-to-understand code from others
  • you can update your components yourself instead of waiting for others
  • you learn a lot by looking at code of others

I read on the internet that some people think that is some kind of arrogance when someone decides to write his own component. I totally disagree, there is no black and white and it is up to you or your team to make the decision when to build your own component. There are many reasons not to, but also many reasons to do it. Weight them carefully.

Two components I recently build: datepicker and Jinja Bootstrap macro

In a previous project I used the jQuery datepicker. Nice but ... after some time I needed functionality like moving to another month without selecting a date. That was the start of many headaches to work around this and other limitations all the time. For this website I also need a datepicker, it must be integrated with Flask and WTForms and it must adhere to the Content Security Policy. The latter can be solved by exporting the datepicker parameters from Python as application/json.
The Jinja Bootstrap macro should make it very easy to generate all kind of forms, it must be flexible and support a wide range of fields including my own datepicker. I hate building forms 'by hand' i.e. putting all form field separately in a template when with a little extra effort in the macro this can be done just by writing:

{{ bootstrap_form(demo_form, id='demo_form') }}

Because I consider both components vital to my future efforts, I decided to write both myself.

Summary

When you build your software with open source components you often take more risks than using commercial components. Do not listen to others, make your own decisions and do not be afraid to write your own components. When you write your own components the main rule is to keep it readable so that others can understand it. One problem with many open source components is that they are optimized. That is nice but optimization often does not translate to readability. Todays computers and phones are far more powerful and optimizations should only be considered when absolutely necessary.

Links / credits

How to make money from open source software
https://www.cio.com/article/3178621/how-to-make-money-from-open-source-software.html

Open Source, SaaS and Monetization
https://lucumr.pocoo.org/2019/11/4/open-source-and-saas/

REINVENT THE WHEEL | meaning in the Cambridge English Dictionary
https://dictionary.cambridge.org/dictionary/english/reinvent-the-wheel

Leave a comment

Comment anonymously or log in to comment.

Comments

Leave a reply

Reply anonymously or log in to reply.