RESTful API with Python and Docker

Exercise: RESTful API with Python and Docker

Alert

These are my notes for an exercise class of my course Laboratory of cloud computing, big data and security, Università Cattolica del Sacro Cuore, ed. 2020. It can be difficult to follow this document without attending the lecture.

Preliminaries

We are going to install Docker on our VirtualBox Linux Mint VM. That's not mandatory, but it helps because with it we are all in the same environment.

Make sure to have python installed, or install it with:

$ sudo apt install python3

Goals

  • We will develop a simple HTTP RESTful API with Python and we will dockerize it.
  • Our application will BE a REST API and at the same time will USE other REST APIs.
  • Our API will compute the daylight time for a given city:
    • input: the name of the city
    • output: the sunrise time, the sunset time, and the daylight time

Weather API

First, we have a look at OpenWeatherMap, a professional API for weather-related stuff.

https://openweathermap.org/

Take a look at the pricing model: https://openweathermap.org/price

We will use the free

The API key...

Set the environment variable for the key:

$ export APIKEY=7c5...

Test the API from the console:

$ curl "http://api.openweathermap.org/data/2.5/weather?q=brescia&appid=$APIKEY"

{"coord":{"lon":10.3,"lat":45.63},"weather":[{"id":804,"main":"Clouds","description":"overcast clouds","icon":"04d"}],"base":"stations","main":{"temp":288.71,"feels_like":287.3,"temp_min":287.59,"temp_max":289.26,"pressure":1018,"humidity":64},"visibility":10000,"wind":{"speed":1.62,"deg":249},"clouds":{"all":100},"dt":1605619763,"sys":{"type":3,"id":197942,"country":"IT","sunrise":1605594088,"sunset":1605627988},"timezone":3600,"id":3181553,"name":"Provincia di Brescia","cod":200}

Note: for security reason, it's better never store keys in "public" code or text files (Dockerfile included).

Use the API with python

File prova_api.py.

import requests
import os

APIKEY = os.environ['APIKEY']  # reads the environment variable

response = requests.get("http://api.openweathermap.org/data/2.5/weather?q=brescia&appid="+APIKEY)

print(response.status_code)
print(response.json())

$ python3 prova_api.py

200
{'coord': {'lon': 10.3, 'lat': 45.63}, 'weather': [{'id': 804, 'main': 'Clouds', 'description': 'overcast clouds', 'icon': '04d'}], 'base': 'stations', 'main': {'temp': 288.71, 'feels_like': 287.3, 'temp_min': 287.59, 'temp_max': 289.26, 'pressure': 1018, 'humidity': 64}, 'visibility': 10000, 'wind': {'speed': 1.62, 'deg': 249}, 'clouds': {'all': 100}, 'dt': 1605619763, 'sys': {'type': 3, 'id': 197942, 'country': 'IT', 'sunrise': 1605594088, 'sunset': 1605627988}, 'timezone': 3600, 'id': 3181553, 'name': 'Provincia di Brescia', 'cod': 200}

My API with Flask

First, install flask. Flask is a simple web server written in Python.

$ sudo apt install python3-flask

Let's write our preliminary app in the file myapi.py.

import flask

app = flask.Flask(__name__)
app.config["DEBUG"] = True


@app.route('/', methods=['GET'])
def home():
    html = """
           <h1>Sunrise-sunset API</h1>
           <p>Example RESTful API for the course Lab. of Cloud Computing, Big Data and security @ UniCatt</p>
           """
    return html

app.run()

And run it with the command:

$ python3 myapi.py

Try it by connecting to http://localhost:5000/ from the web browser.

Let's add a new entry-point named /api/prova and return a JSON for testing.

import flask

app = flask.Flask(__name__)
app.config["DEBUG"] = True


@app.route('/', methods=['GET'])
def home():
    html = """
           <h1>Sunrise-sunset API</h1>
           <p>Example RESTful API for the course Lab. of Cloud Computing, Big Data and security @ UniCatt</p>
           """
    return html


@app.route('/api/prova', methods=['GET'])
def api_prova():
    output = {
        'sunrise': 1,
        'sunset': 2,
        'city': 'Brescia'
    }
    return flask.jsonify(output)


app.run()

Reading parameters

import flask
from flask import request, jsonify

app = flask.Flask(__name__)
app.config["DEBUG"] = True


@app.route('/', methods=['GET'])
def home():
    html = """
           <h1>Sunrise-sunset API</h1>
           <p>Example RESTful API for the course Lab. of Cloud Computing, Big Data and security @ UniCatt</p>
           """
    return html


@app.route('/api/prova', methods=['GET'])
def api_prova():
    city = request.args.get('city')
    output = {}
    if city:
        output = {
            'sunrise': 1,
            'sunset': 2,
            'city': city
        }
    return jsonify(output)

app.run()

Alternative, arguably more elegant, way:

@app.route('/api/prova/<city>', methods=['GET'])
def api_prova(city):
    output = {}
    if city:
        output = {
            'sunrise': 1,
            'sunset': 2,
            'city': city
        }
    return jsonify(output)

Retrieve data from REST API, elaborate it and return it

Now, we retrieve the weather data from the OpenWeatherMap API, we elaborate and return the result.

The goal is to add a new entrypoint in our API, namely /api/daylight/<city>, and return the daylight, i.e., the timespan between the sunrise and the sunset.

@app.route('/api/daylight/<city>', methods=['GET'])
def api_daylight(city):
    output = {}
    uri = f"http://api.openweathermap.org/data/2.5/weather?q={city}&appid={APIKEY}"
    print(uri)
    res = requests.get(uri)
    if res.status_code == 200:
        data = res.json()
        sunrise = data['sys']['sunrise']
        sunset = data['sys']['sunset']
        output = {
            'sunrise': sunrise,
            'sunset': sunset,
            'daylight': sunset-sunrise,
            'city': city            
        }
    return output

Exercise: make the result more human-readable, such as

{
  "city": "Brescia",
  "daylight": "9:25:00",
  "sunrise": "Tue, 17 Nov 2020 07:21:28 GMT",
  "sunset": "Tue, 17 Nov 2020 16:46:28 GMT"
}

TIP: use the datetime module.

Check the solution
# [at the import section]
from datetime import datetime, timedelta

# [all the other code]

@app.route('/api/daylight/<city>', methods=['GET'])
def api_daylight(city):
    output = {}
    uri = f"http://api.openweathermap.org/data/2.5/weather?q={city}&appid={APIKEY}"
    print(uri)
    res = requests.get(uri)
    if res.status_code == 200:
        data = res.json()
        sunrise = datetime.fromtimestamp(data['sys']['sunrise'])
        sunset = datetime.fromtimestamp(data['sys']['sunset'])
        daylight = sunset-sunrise
        output = {
            'sunrise': sunrise,
            'sunset': sunset,
            'daylight': str(daylight),
            'city': city            
        }
    return output

BUT THINK: is it a good idea to make things more human-readable at the price of less machine-readability in a REST API?

Exercise: add another entrypoint for the API with the city as input and the current temperature and the temperature range as output

Example:

$ curl "http://localhost:5000/api/temperature/brescia"
{
  "city": "brescia",
  "temp": 14,
  "temp_min": 13,
  "temp_max": 16,
  "temp_range": 3
}

Dockerization

Since Flask accepts connection only from localhost by default and we are going to connect through a Docker virtual network, we need to explicitly allow connections from everywhere. So, we need to modify the last line of the myapi.py file to:

app.run(host='0.0.0.0')  # accept connection from every host

File requirements.txt:

flask==1.1.1
requests==2.22.0

File Dockerfile:

FROM python:3.8-buster

WORKDIR /app

COPY requirements.txt .

RUN pip install -r requirements.txt

COPY myapi.py .

EXPOSE 5000

CMD ["python", "myapi.py"]

Build the image with:

$ docker build --tag myapi .

Run the container with:

$ docker run -p 5000:5000 --env APIKEY=$APIKEY --rm myapi

* Serving Flask app "myapi" (lazy loading)
* Environment: production
  WARNING: This is a development server. Do not use it in a production deployment.
  Use a production WSGI server instead.
* Debug mode: on
* Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger PIN: 610-128-137
172.17.0.1 - - [16/Nov/2020 16:42:28] "GET /api/daylight/Brescia HTTP/1.1" 200 -

Note that this is configuration is not recommended in production. In production we need to set up a WSGI server (e.g. gunicorn) and a reverse proxy (e.g. nginx). See for example https://towardsdatascience.com/how-to-deploy-ml-models-using-flask-gunicorn-nginx-docker-9b32055b3d0.

Exercise

We now want to develop a REST API that given a train id number (e.g. S11119/8810) will output:

  • where the train will meet the sunrise and/or the sunset
  • how long the journey is during the daylight time and how long during the night

Example:

$ curl "http://localhost:5000/api/trainlight/S11119/8810"
{
  "way": {
    "from": "BARI CENTRALE",
    "start_time": "Tue, 17 Nov 2020 05:30:00",
    "to": "MILANO CENTRALE",
    "destination_time": Tue, 17 Nov 2020 12:55:00"
  },
  "sunrise": {
    "time": "Tue, 17 Nov 2020 06:54:00",
    "location": ["FOGGIA", "TERMOLI"]
  },
  "daytime": "06:01:00",
  "nighttime": "01:24:00"
}

NOTE: the entire exercise is challenging. Try to divide it into sub-problems and address one sub-problem at a time.

API viaggiatreno

To get information about trains in Italy, take a look at:

Compare viaggiatreno API with OpenWeatherMap API. Which one is better? Why?