BullMQ is a job queue for Node that stores everything in Redis. It has three main building blocks:
- A Queue adds jobs:
queue.add(name, data, opts). - A Worker takes jobs from one queue and runs a processor function on each job.
- QueueEvents listens to events for the whole queue, such as completed, failed and progress. It works from any process, because it reads them from Redis streams.
Jobs, their data and the queue state all live in Redis. Because of this, the producers (the code that adds jobs) and the workers can run in different processes or on different servers.
A job moves through these states:
- It starts in
waiting. If it has a delay, it starts indelayed. If it has a priority, it starts inprioritized. - A worker moves it to
activewhile it runs. - It ends in
completed, with a return value, or infailed, once it has used all its attempts. - A retry with backoff (a wait before the next try) moves a failed attempt back to
delayed. - In a flow (a parent job with child jobs), the parent waits in
waiting-childrenuntil its children finish.
- A Queue adds jobs, a Worker processes them, and QueueEvents listens to their events.
- All state lives in Redis, so producers and workers are independent of each other.
- The main states are waiting, delayed or prioritized, then active, then completed or failed.
- Retries go back through delayed, and parent jobs in flows wait in waiting-children.
Read more
- BullMQ docs (https://docs.bullmq.io/)docs.bullmq.io