This seems to be a common pattern with background workers in a web application, but I haven't seen it explicitly documented so I want to open this up as a question. If necessary, I'm happy to provide a PR with a documentation update.

Let's say I have a web app and I have 30 different background jobs that get run. For example:

1. send an email
2. after a user authenticates with facebook, queue a job to fetch new data from the facebook api and update the user's profile
3. after receiving a webhook from stripe, queue a job to update a customer's subscription data
4. after some action happens, queue a job to send an api call to zapier
5. after receiving a webhook from zapier, and process it

The way bee-queue (and bull) are set up, each of these 30 background jobs would have their own "queue". Below is an example showing the first two queues.

As one might imagine, with 30 different background jobs I will have 30 different instances of Queue on each webserver and 30 different instances of Queue on each worker server.

From the docs:

Queues are very lightweight — the only significant overhead is connecting to Redis — so if you need to handle different types of jobs, just instantiate a queue for each:

My Questions

1. Is there a way to re-use the Redis connection so that each webserver and each worker server only maintains 1 connection? Or is that something I need to worry about?
2. Is there a better recommended pattern that I am missing?
3. Are there other gotchas or things to watch out for if I have 30+ different queues?

Webserver

const Queue = require('bee-queue')
const emailQueue = new Queue('EMAIL_DELIVERY', {
  redis: process.env.REDIS_URL,
  isWorker: false,
  getEvents: false
})

const facebookUpdateQueue = new Queue('FACEBOOK_UPDATE', {
  redis: process.env.REDIS_URL,
  isWorker: false,
  getEvents: false
})

function sendEmail (messageData) {
  const job = queue.createJob(messageData)
  return job.save()
}

function updateFacebook (data) {
  const job = facebookUpdateQueue.createJob(data)
  return job.save()
}

Worker server

const Queue = require('bee-queue')
const emailQueue = new Queue('EMAIL_DELIVERY', {redis: process.env.REDIS_URL})
const facebookUpdateQueue = new Queue('FACEBOOK_UPDATE', {redis: process.env.REDIS_URL})

emailQueue.process((job) => {
  return Email.deliver(job.data)
})

facebookUpdate.process((job) => {
  return FacebookUpdater.process(job.data)
})