angle-uparrow-clockwisearrow-counterclockwisearrow-down-uparrow-leftatcalendarcard-listchatcheckenvelopefolderhouseinfo-circlepencilpeoplepersonperson-fillperson-plusphoneplusquestion-circlesearchtagtrashx

Documenting a Flask RESTful API with OpenAPI (Swagger) using APISpec

Create your own view class and utility functions to reduce code and avoid errors.

22 April 2021 Updated 22 April 2021
In API, Flask
post main image
https://unsplash.com/@sincerelymedia

When you create an API, you want to document it and today it is obvious to use OpenAPI for this. I am already using the Marshmallow package. The same people also developed the package APIspec, with a Flask plugin in an additional package apispec-webframeworks. To present the OpenAPI documentation I use the package flask-swagger-ui and the package flask-jwt-extended is used to protect the endpoints.

Below I show you a way to create your own API view class together with API view class utility functions to reduce code and avoid errors.

Quick start by example

There is very much information on the internet about building an API with Flask and Swagger. I was happy to find a nice small example to get started, 'REST API Development with Flask', see links below. What happens here is:

  • Setup your basic OpenAPI spec
  • Document your endpoints using docstrings
  • Scan all view functions and add them to your spec

Generating OpenAPI documentation by scanning docstrings

Is it a good idea to use docstrings to generate your OpenAPI document? Docstrings are tied to a function or method. I have mixed feelings about this. When you have many functions or methods in different files, making changes to your docstrings can be very time consuming.

The alternative would be to create a standalone OpenAPI document. Here all information is in one place, most of the time, in a single document. You first create the specifications and use this document to create your functions and methods. With the proper software you probably can generate big parts of the code automatically. This also makes it very easy to make changes.

Although the latter is the preferred way, I am using the docstring scanning method here.

Problem: docstring duplication

Most of the time we have a number of model classes which are treated more or less the same. I created a demo application consisting of the following classes:

  • Friend, can have zero or one city, and zero or more hobbies
  • City
  • Hobby
  • User

The OpenAPI docstrings for these classes are more or less the same. For example, the update methods for the City and Hobby are:

class CitiesAPI(...):

    ...

    @jwt_required()
    def update_by_id(self, city_id):
        '''
        ---
        put:
          tags:
          - City
          summary: Update city by id
          description: Update city by id
          parameters:
          - in: path
            schema: CitiesRequestPathParamsSchema
          requestBody:
            required: true
            content:
                application/json:
                    schema: CitiesUpdateRequestBodyParamsSchema
          responses:
            400:
              description: One or more request parameters did not validate
            404:
              description: City not found
            200:
              description: City updated
              content:
                application/json:
                  schema:
                    type: object
                    properties:
                      data: CitiesResponseSchema
        '''
        <code of the method>

and

class HobbiesAPI(...):

    ...

    @jwt_required()
    def update_by_id(self, hobby_id):
        """
        ---
        put:
          tags:
          - Hobby
          summary: Update hobby by id
          description: Update hobby by id
          parameters:
          - in: path
            schema: HobbiesRequestPathParamsSchema
          requestBody:
            required: true
            content:
                application/json:
                    schema: HobbiesUpdateRequestBodyParamsSchema
          responses:
            400:
              description: One or more request parameters did not validate
            404:
              description: Hobby not found
            200:
              description: Hobby updated
              content:
                application/json:
                  schema:
                    type: object
                    properties:
                      data: HobbiesResponseSchema
        """
        <code of the method>

Note that these methods are decorated with the @jwt_required() decorator. I assume you also want to do this (at some time).

To avoid duplication I created a base APIModelViewClass and some utility functions in APIModelViewClassUtils.

Why you cannot use Flask MethodView

I looked into Flask MethodView as the documentation states is is useful when building APIs. This class has methods like get, post that are called depending on the request method. This means that the GET request method calls the get method of the MethodView class.

For a City, my API must have the following functions:

list           GET   /cities
create         POST  /cities
get_by_id      GET   /cities/<int:city_id>
update_by_id   GET   /cities/<int:city_id>
delete_by_id   GET   /cities/<int:city_id>

We are using the GET method both for a list of cities and a single city. The Flask documentation suggests we decide inside the get method which function we must perform:

    def get(self, city_id):
        if city_id is None:
            # return a list of citys
            pass
        else:
            # expose a single city
            pass

But now we cannot add a docstring anymore ... :-(

A way out of this is to use two MethodView classes. One for the list function and create functions and one for the 'by_id' functions. But even with this solution there are problems. It appears that when you use Blueprints, not all functions are added by apispec, see also 'Only single path added for Flask API MethodViews #14'.

I am not sure what the status is of this at the moment, so I decided to create my own 'MethodView', called APIModelViewClass, and my own utilities class APIModelViewClassUtils.

Using a decorator to add the docstrings

The obvious choice is to use a decorator to add the OpenAPI docstring to the API methods. Once they are there, they can be scanned and loaded by apispec.

Many of my API methods are already decorated with the @jwt_required() decorator, meaning that we must combine these decorators.

I created a class APIModelViewClass, that is at the moment just a stub, and a class APIModelViewClassUtils
containing utility functions, at the moment containing three methods:

  • decorate_with_docstring()
  • endpoint_decorator()
  • endpoints_register()

Later I may merge these classes. Below is part of the code.

class APIModelViewClassUtils:

    def decorate_with_docstring(fn, op, params):
        tags = params.get('tags')
        item_name_cap, item_name, item_name_plural_cap, item_name_plural = params.get('item_names')
        in_path_schema = params.get('in_path_schema')
        in_query_schema = params.get('in_query_schema')
        in_body_schema_create = params.get('in_body_schema_create')
        in_body_schema_update = params.get('in_body_schema_update')
        out_data_schema = params.get('out_data_schema')

        '''
        if op == 'list':
            fn.__doc__ = """
            ...

        elif op == 'update_by_id':
            fn.__doc__ = """
            ---
            put:
              tags:
              - {tags}
              summary: Update {item_name} by id
              description: Update {item_name} by id
              parameters:
              - in: path
                schema: {in_path_schema}
              requestBody:
                required: true
                content:
                    application/json:
                        schema: {in_body_schema_update}
              responses:
                400:
                  description: One or more request parameters did not validate
                404:
                  description: {item_name_cap} not found
                200:
                  description: {item_name_cap} updated
                  content:
                    application/json:
                      schema:
                        type: object
                        properties:
                          data: {out_data_schema}
                    """.format(
                        tags=tags,
                        item_name=item_name,
                        item_name_cap=item_name_cap,
                        in_path_schema=in_path_schema,
                        in_body_schema_update=in_body_schema_update,
                        out_data_schema=out_data_schema)
        '''

    @classmethod
    def endpoint_decorator(cls, params):
        def wrapper(fn):

            op = fn.__name__

            # decorate endpoint
            cls.decorate_with_docstring(fn, op, params)

            @wraps(fn)
            def decorator(*args, **kwargs):

                # verify_jwt_in_request works like @jwt_required
                verify_jwt_in_request()

                return fn(*args, **kwargs)

        return wrapper

    @staticmethod
    def endpoints_register(bp, api_model_view_class, url, id_attr_name, id_type='int'):
        model_obj = api_model_view_class()
        
        '''
        # update_by_id
        if hasattr(model_obj, 'update_by_id'):
            bp.add_url_rule(url_by_id, view_func=model_obj.update_by_id, methods=['PUT'])
        '''


class APIModelViewClass():
    pass

In decorate_with_docstring() we decorate the method with a formatted docstring. In endpoint_decorator() we first decorate the method and then check if access is granted using verify_jwt_in_request(). In endpoints_register we register the endpoints used by this class.

Now (part of) the CityAPI looks like:

class CitiesAPI(APIModelViewClass):
    params = {
        'tags': 'City',
        'item_names': ['City', 'city', 'Cities', 'cities'],
        'in_query_schema': 'CitiesRequestQueryParamsSchema',
        'in_path_schema': 'CitiesRequestPathParamsSchema',
        'in_body_schema_create': 'CitiesCreateRequestBodyParamsSchema',
        'in_body_schema_update': 'CitiesUpdateRequestBodyParamsSchema',
        'out_data_schema': 'CitiesResponseSchema',
    }

    '''

    @APIModelViewClassUtils.endpoint_decorator(params)
    def update_by_id(self, city_id):
        result = APIUtils.request_schemas_load([
            CitiesRequestPathParamsSchema(),
            CitiesUpdateRequestBodyParamsSchema()])

        city = APIUtils.get_by_id_or_404(City, city_id, 'City', 'city_id')

        for k, v in result['body'].items():
            setattr(city, k, v)
        app_db.session.commit()

        return jsonify({
            'data': CitiesResponseSchema().dump(city)
        }), 200

We define the parameters for the decorator outside the __init__() function. They are constants and do not take up much space. The above can be optimized and done better but at least it is easy to read and maintain.

Open API security

Above I showed how I used flask_jwt_extended to protect endpoints. Now we must tell OpenAPI what we did. I choose to protect all endpoints by default in OpenAPI and later remove this protection in the docstrings of unprotected methods.

# create apispec
spec = APISpec(
    title=title,
    version=spec_version,
    openapi_version=openapi_version,
    plugins=[FlaskPlugin(), MarshmallowPlugin()],
    **settings
)

security_scheme_bearer = {
    "type": "http",
    "description": "Enter JWT Bearer token",
    "scheme": "bearer",
    "bearerFormat": "JWT"
}

security_scheme_basic_auth = {
    "type": "http",
    "scheme": "basic"
}

spec.components.security_scheme("BearerAuth", security_scheme_bearer)
spec.components.security_scheme("BasicAuth", security_scheme_basic_auth)

In the docstrings of methods that are protected we do nothing. In the docstrings of methods that are not protected I add:

    security: []

Testing

Testing an API documented with OpenAPI is ... difficult. We have the actual interface and the documented (OpenAPI) interface. We should write a test suite at least for the documented interface. I did not do this yet but instead used Pytest and the requests package to do functional testing of the API. I started doing some tests with postman.

Summary

Coding my API using the Marshmallow package was a nice experience. Sending out consistent (error) messages was a challenge because every package has its own default format. And documenting my API with OpenAPI (Swagger) with Python was like a nightmare. Flask MethodView and Blueprints not working as expected with apispec, unclear documentation in apispec-webframeworks.

I had to learn new things, like how to write OpenAPI lines by hand. I am happy with the final result, and know exactly how it works, meaning it is easy to improve and make changes.

There must be better ways to do this. I have been reading about FastAPI, this is a Python framework dedicated to building fast APIs. It has much better performance compared to Flask and appears to support many things out-of-the-box. Maybe I will try this the next time, or convert this API to FastAPI. But it also has 540 Open issues on Github ... can be good, can be bad.

Links / credits

apispec
https://apispec.readthedocs.io/en/latest/

Authentication and Authorization (OAS3)
https://swagger.io/docs/specification/authentication/

FastAPI - The Good, the bad and the ugly
https://dev.to/fuadrafid/fastapi-the-good-the-bad-and-the-ugly-20ob

Flask-JWT-Extended’s Documentation
https://flask-jwt-extended.readthedocs.io/en/stable/

Only single path added for Flask API MethodViews #14
https://github.com/marshmallow-code/apispec-webframeworks/issues/14

Pluggable Views
https://flask.palletsprojects.com/en/1.1.x/views/

Postman | The Collaboration Platform for API Development
https://www.postman.com

REST API Development with Flask
https://www.datascienceblog.net/post/programming/flask-api-development/

Read more

API Flask

Leave a comment

Comment anonymously or log in to comment.

Comments

Leave a reply

Reply anonymously or log in to reply.