CodePark/ questions / BullMQ & Node job queues
START PRACTICINGPRACTICE
Backend question · Message queues & streaming › BullMQ & Node job queues

A NestJS app runs 4 replicas, and each one uses setInterval to send a daily report email. What goes wrong, and how would you schedule it with BullMQ instead?

Every replica runs its own timer, so the report is sent 4 times a day. Timers also have other problems:

  • They are lost when the app restarts.
  • Their timing shifts with every deploy.
  • Nothing retries a run that failed.

With BullMQ, the schedule lives in Redis. In BullMQ v5 you register a job scheduler, for example:
queue.upsertJobScheduler("daily-report", { pattern: "0 8 * * *" }, { name: "report" })

The scheduler is stored under its id ("daily-report"). So all 4 replicas can call this at startup, and there is still only one schedule. For each run, BullMQ creates one job. Only one worker processes that job, on whichever replica takes it first. The job gets the normal retries, backoff and failure tracking.

Older code uses the repeat job option for the same purpose. Job schedulers are the newer API.

Keep the processor idempotent, which means running it twice has the same effect as once. This matters because a retry may send the report again.

  • setInterval runs once on every replica, and it is lost when the app restarts.
  • BullMQ keeps the schedule in Redis, stored under the scheduler id.
  • upsertJobScheduler is safe to call from every replica.
  • BullMQ creates one job per run, and one worker processes it, with retries.