When it comes to observability Grafana is the go-to tool for visualisation.
A Grafana dashboard consists of various forms of visualisations which are usually backed by a database.
This is not always the case. Sometimes instead of pushing the data from the database as is, you might want to refine the data. This cannot alway be achieved through the functionalities the db provides. For example you might want to fetch results from a proprietary api. This is where the grafana-infinity-datasource plugin kicks in. With the grafana-infinity-datasource you can create visualisations based on json, xml , csv etc. You can issue an http request towards a REST api and plot the received data.
Let’s assume we have an eShop application. We will create a simple python api using FastApi managing the items of the eShop and the purchase volume.
Through this api we will add items and purchase-volume entries.
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from typing import List
from datetime import datetime
app = FastAPI()
class Item(BaseModel):
id: int
name: str
description: str = None
price: float
class Purchase(BaseModel):
price: float
time: datetime
items = []
purchases = []
@app.post("/items/", response_model=Item)
def create_item(item: Item):
items.append(item)
return item
@app.get("/items/", response_model=List[Item])
def read_items():
return items
@app.get("/items/{item_id}", response_model=Item)
def read_item(item_id: int):
for item in items:
if item.id == item_id:
return item
raise HTTPException(status_code=404, detail="Item not found")
@app.delete("/items/{item_id}", response_model=Item)
def delete_item(item_id: int):
for idx, item in enumerate(items):
if item.id == item_id:
return items.pop(idx)
raise HTTPException(status_code=404, detail="Item not found")
@app.post("/purchases/", response_model=Purchase)
def create_purchase(purchase: Purchase):
purchases.append(purchase)
return purchase
@app.get("/purchases/", response_model=List[Purchase])
def read_purchases():
return purchases
also we need FastApi to be added to the requirements.txt
fastapi
We shall host the application through Docker thus we shall create a Dockerfile
FROM python:3.11-slim WORKDIR /app COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt COPY main.py main.py EXPOSE 8000 CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
We should proceed to the Grafana visualisations.
Essentially we have two different sources of data.
The model Item will be visualized in a table and the model purchase will be visualized through a time series graph.
I shall use Docker Compose to provision Grafana as well as the python application
version: '3.8'
services:
app:
build: .
ports:
- 8000:8000
grafana:
image: grafana/grafana:latest
ports:
- "3000:3000"
volumes:
- ./grafana:/var/lib/grafana
environment:
- GF_SECURITY_ADMIN_USER=test
- GF_SECURITY_ADMIN_PASSWORD=infinity
- GF_INSTALL_PLUGINS=yesoreyeram-infinity-datasource
Essentially through the environment variable on Docker I enable the the infinity-datasource plugin.
We can get our instances up and running by issuing
docker compose up
Docker Compose V2 is out there with many good features, you can find more about it on the book I authored:
A Developer’s Essential Guide to Docker Compose.
We can now populate the application with some data:
$ curl -X POST "http://127.0.0.1:8000/purchases/" -H "Content-Type: application/json" -d '{"time": "2024-07-15T12:40:56","price":2.5}'
$ curl -X POST "http://127.0.0.1:8000/purchases/" -H "Content-Type: application/json" -d '{"time": "2024-07-15T12:41:56","price":4.0}'
$ curl -X POST "http://127.0.0.1:8000/purchases/" -H "Content-Type: application/json" -d '{"time": "2024-07-15T12:42:56","price":1.5}'
$ curl -X POST "http://127.0.0.1:8000/purchases/" -H "Content-Type: application/json" -d '{"time": "2024-07-15T12:43:56","price":3.5}'
$ curl -X POST "http://127.0.0.1:8000/items/" -H "Content-Type: application/json" -d '{"id": 1, "name": "Item 1", "description": "This is item 1", "price": 10.5, "tax": 0.5}'
Onwards create a dashboard on Grafana.
One visualisation for items.
One visualisation for purchase volume.
As you can see in both cases I used the http://app:8000 endpoint, which is our application and the DNS that Compose application can resolve.
That’s it, we plotted our data from a REST api using Grafana.
Emmanouil Gkatziouras 