Flask, Babel and Javascript language files
This post describes a method to generate Javascript language files de.js, en.js, etc. and how to add them to your multilanguage Flask app.
This Flask website is multilanguage. The implementation is described in previous posts. So far all my translations were in the Python code and the HTML templates. On a few places I needed some translations in Javascript and did this by pulling this Javascript code inline in the HTML template. For example, for forms I needed:
e.target.setCustomValidity('Please fill out this field.');
I pulled this Javascript into the HTML template and changed it to:
e.target.setCustomValidity("{{ _('Please fill out this field.') }}");
That was easy and works fine.
Language in Javascript files
I knew that one day I had to change this and implement multilanguage for Javascript files. This day came sooner than expected because I wanted to implement the Content Security Policy header. The minimum we should do is remove inline scripts and the eval() function:
Content-Security-Policy: script-src 'self'
de.js
en.js
es.js
fr.js
nl.js
ru.js
<script src="{{ url_for('static', filename='js/mlmanager.js') }}?v={{ et }}"></script>
<script src="{{ url_for('static', filename='js/locales/' + lang_code + '.js') }}?v={{ et }}"></script>
<script src="{{ url_for('static', filename='js/base.js') }}?v={{ et }}"></script>
// mlmanager.js
var ML = function(params){
this.key2translations = {};
this.keys = []
if(params.hasOwnProperty('key2translations')){
this.key2translations = params.key2translations;
this.keys = Object.keys(this.key2translations);
}
this.t = function(k){
if(this.keys.indexOf(k) === -1){
alert('key = ' + k + ' not found');
return;
}
s = this.key2translations[k];
return s.replace(/"/g,'\"');
};
};
When creating a new ml object we pass also the translations. Method t is use to get a translation. A translated language file, e.g. de.js, looks like:
// de.js
var ml = new ML({
'key2translations': {
'Content item': "Inhaltselement",
'Please fill out this field.': "Bitte füllen Sie dieses Feld aus.",
},
});
Finally, in the file with the actual Javascript code, base.js, we change the text that must be translated from:
e.target.setCustomValidity('Please fill out this field.');
to:
e.target.setCustomValidity( ml.t('Please fill out this field.') );
Problem: how do we generate the Javascript language files de.js, en.js, etc.
The standard Babel documentation only mentions commands like init, extract, update, compile. What we need is a way to:
- extract the texts to be translated from the javascript files
- automatically generate the language files de.js, en.js, etc.
Extract the texts to be translated from the Javascript files
I decided not to scan the Javascript files but instead create a new HTML (template) file, jsbase.html, holding all the texts for the Javascript files, example:
var ml = new ML({
'key2translations': {
...
'Content item': "{{ _('Content item') }}",
'Please fill out this field.': "{{ _('Please fill out this field.') }}",
...
},
});
We put this file in the templates directory so it will be scanned by Babel when we issue the standard translation commands:
pybabel extract -F babel.cfg -k _l -o messages.pot .
pybabel update -i messages.pot -d app/translations
# do yourself: translate all texts in the po files either manual or automated
pybabel compile -d app/translations
Now we have the translated texts for the Javascript files somewhere in the messages.po files. You can check this e.g. by dumping a messages.po file:
from babel.messages.pofile import read_po
import os
def show_catalog(lc):
lc_po_file = os.path.join('app_frontend', 'translations', lc, 'LC_MESSAGES', 'messages.po')
# catalog = read_po(open(lc_po_file, mode='r', encoding='utf-8'))
# without encoding parameter works if the default encoding of the platform is utf-8
catalog = read_po(open(lc_po_file, 'r'))
for message in catalog:
print('message.id = {}, message.string = {}'.format(message.id, message.string))
show_catalog('de_DE')
This prints a list of message ids and string:
...
message.id = Sub image, message.string = Unterbild
message.id = Sub image text, message.string = Unterbildtext
message.id = Select image, message.string = Bild auswählen
...
Automatically generate the Javascript language files de.js, en.js, etc.
What we need is a way to translate this jsbase.html outside of Flask to our languages and generate the files de.js, en.js, etc. We could use the above code to get the texts from Javascript files and generate the language files de.js, en.js, etc. But this is cumbersome and prone to error.
Then I bounced into a way to render a template outside of Flask, see links below. The idea is to render the jsbase.html template letting Babel put the proper translations into it. Then all we need to do is write the rendered result to language files de.js, en.js, etc. Is it really that easy? Here is the code that does this:
from jinja2 import Environment, FileSystemLoader, select_autoescape
from babel.support import Translations
import os
import sys
def generate_translated_js_file(
app_translations_dir,
language_region_code,
app_templates_dir,
template_file,
js_translation_file):
template_loader = FileSystemLoader(app_templates_dir)
# setup environment
env = Environment(
loader=template_loader,
extensions=['jinja2.ext.i18n', 'jinja2.ext.autoescape'],
autoescape=select_autoescape(['html', 'xml'])
)
translations = Translations.load(app_translations_dir, language_region_code)
env.install_gettext_translations(translations)
template = env.get_template(template_file)
rendered_template = template.render()
with open(js_translation_file, 'w') as f:
f.write(rendered_template)
This function loads the selected language, uses render() to translate and writes the result as de.js, en.js, etc. Note that I use multiple apps in my setup, app_frontend, app_admin, using DispatcherMiddleware. To generate all Javascript language files for all apps and languages I call the above in another function:
def generate_translated_js_files():
# app translations directory has subdirectories de_DE, en_US, es_ES, ...
# lang_code is language code used in the Flask app
language_region_code2lang_codes = {
'de_DE': 'de',
'en_US': 'en',
'es_ES': 'es',
'fr_FR': 'fr',
'nl_NL': 'nl',
'ru_RU': 'ru',
}
template_file = 'jsbase.html'
for app_name in ['app_frontend', 'app_admin']:
# app/translations
app_translations_dir = os.path.join(app_name, 'translations')
# app/templates
app_templates_dir = os.path.join(app_name, 'templates')
for language_region_code, lang_code in language_region_code2lang_codes.items():
if not os.path.isdir( os.path.join(app_translations_dir, language_region_code)):
print('error: not a directory = {}'.format( os.path.isdir( os.path.join(app_translations_dir, language_region_code) )))
sys.exit()
# shared/static/js/locales is the directory where we write de.js, en.js, etc.
js_translation_file = os.path.join('shared', 'static', 'js', 'locales', lang_code + '.js')
# translate
generate_translated_js_file(
app_translations_dir,
language_region_code,
app_templates_dir,
template_file,
js_translation_file)
# do it
generate_translated_js_files()
Note that this a bit double at the moment because frontend and admin share the same static directory.
Problems
Of course there are problems. When the Javascript code was in the HTML template I added Jinja code:
{% if ... %}
...
{% else %}
...
{% endif %}
to use a certain part of the Javascript code. We cannot do this anymore ... :-(. To be more specific, in my case Javascript is calling another page with a url that can exist or not and the url also depends on the language. For example, the link in the Javascript inline code of the HTML template looks like this:
{% if 'Privacy policy' in app_template_slug %}
moreLink: '{{ url_for('pages.page_view', slug=app_template_slug['Privacy policy']['slug']) }}',
{% else %}
moreLink: '',
{% endif %}
What I did is a total rewrite of the cookie consent. For now the HTML is no longer generated in the Javascript but in the HTML template. It still needs more work.
Summary
This is a first implementation but it works fine. The magic is using the Babel and Jinja APIs. Possible improvements:
Instead of of having strings as index for the translation:
ml.t('Please fill out this field')
we may want to use objects:
ml.t( t.Please_fill_out_this_field )
And instead of having a translations Javascript file with Javascript objects we may want to use a JSON file holding the translations only. Anyway, next steps will be to selectively add more custom Javascript files and more translations.
Links / credits
Analyse your HTTP response headers
https://securityheaders.com
Babel documentation
https://readthedocs.org/projects/python-babel/downloads/pdf/latest/
Best practice for localization and globalization of strings and labels [closed]
https://stackoverflow.com/questions/14358817/best-practice-for-localization-and-globalization-of-strings-and-labels/14359147
Content Security Policy - An Introduction
https://scotthelme.co.uk/content-security-policy-an-introduction/
Explore All i18n Advantages of Babel for Your Python App
https://phrase.com/blog/posts/i18n-advantages-babel-python/
Give your JavaScript the ability to speak many languages
https://github.com/airbnb/polyglot.js
Read more
Babel Flask Javascript Jinja2
Recent
- Hiding database UUID primary keys of your web application
- Don't Repeat Yourself (DRY) with Jinja2
- SQLAlchemy, PostgreSQL, maximum number of rows per user
- Show the values in SQLAlchemy dynamic filters
- Secure data transfer with Public Key encryption and pyNaCl
- rqlite: a high-availability and distributed SQLite alternative
Most viewed
- Using Python's pyOpenSSL to verify SSL certificates downloaded from a host
- Using UUIDs instead of Integer Autoincrement Primary Keys with SQLAlchemy and MariaDb
- Using PyInstaller and Cython to create a Python executable
- Connect to a service on a Docker host from a Docker container
- SQLAlchemy: Using Cascade Deletes to delete related objects
- Flask RESTful API request parameter validation with Marshmallow schemas