Queen MQ is a partitioned message queue backed by PostgreSQL, built with uWebSockets, libuv, and libpq async API. It features unlimited FIFO partitions that process independently, consumer groups with replay, transactional delivery, tracing, and ACID-guaranteed durability. With Queen you get Kafka semantics on PostgreSQL, using stateless clients that speak HTTP to a stateless server, easy to deploy and manage, but still powerful and flexible enough to sustain tens of thousands of requests per second across thousands of partitions.

For a experimental PostgreSQL extension version of Queen MQ, see pg_qpubsub.

Born at Smartness to power Smartchat, Queen solves a unique problem: unlimited FIFO partitions where slow processing in one partition doesn't block others.

Perfect for:

• Processing messages in order, without losing them somewhere in the middle
• Avoid slow message processing blocking other partitions, solving Head of Line Blocking problem
• When you need tens of thousands of partitions to process messages in parallel, respecting the order of the messages
• Process the same messages in multiple pipelines
• Have a clear view of the message processing flow and traceability
• Build event-driven microservices with exactly-once delivery guarantees
• Critical systems that need to be highly available and reliable with zero message loss

Its push peaks for single request is around 45k req/s, with sustatined load (PUSH+POP) around 10k req/s, and with consumer groups around 60k req/s.

Create a Docker network and start PostgreSQL and Queen Server:

# Create a Docker network for Queen components
docker network create queen

# Start PostgreSQL
docker run --name qpg --network queen -e POSTGRES_PASSWORD=postgres -p 5433:5432 -d postgres

# Wait for PostgreSQL to start
sleep 2

# Start Queen Server
docker run -p 6632:6632 --network queen -e PG_HOST=qpg -e PG_PORT=5432 -e PG_PASSWORD=postgres -e NUM_WORKERS=2 -e DB_POOL_SIZE=5 -e SIDECAR_POOL_SIZE=30 smartnessai/queen-mq:0.12.2

Then in another terminal, use cURL (or the client libraries) to push and consume messages

Push message:

curl -X POST http://localhost:6632/api/v1/push \
  -H "Content-Type: application/json" \
  -d '{"items": [{"queue": "demo", "payload": {"hello": "world"}}]}'

that returns something like:

[{"message_id": "...", "status": "queued", ...}]

Consume message:

curl "http://localhost:6632/api/v1/pop/queue/demo?autoAck=true"

that returns something like:

{"messages": [{"data": {"hello": "world"}, ...}], "success": true}

Then go to the dashboard (http://localhost:6632) to see the messages and the status of the queue.

📚 Complete Documentation

• Quick Start Guide
• Basic Concepts

• JavaScript Client
• Python Client
• C++ Client
• HTTP API Reference

The repository is structured as follows:

• lib: C++ core queen library (libqueen), implementing libuv loops, sql schema and procedures
• server: Queen MQ server, implementing the HTTP API that talks to the libqueen library
• pg_qpubsub: PostgreSQL extension for using queen-mq semantics as a PostgreSQL extension
• client-js: JavaScript client library (browser and node.js)
• client-py: Python client library (python 3.8+)
• client-cpp: C++ client library (cpp 17)
• proxy: Proxy server (authentication)
• app: Vue.js dashboard (vue 3)
• website: Documentation website (vitepress)
• examples: JS client examples

JS clients from version 0.12.0 can be run inside a browser

Full Release Notes →

• Clients 0.12.1: Fixed bug in transaction with consumer groups

Queen MQ is released under the Apache 2.0 License.