CodePark/ questions / Persistence (RDB & AOF)
START PRACTICINGPRACTICE
Backend question · Redis & caching › Persistence (RDB & AOF)

A primary has persistence disabled for speed and is set to restart automatically. Its replicas do have persistence. What can go wrong?

The replicas can wipe their own data. Here is how:

  • The primary crashes and restarts quickly.
  • It has no persistence, so it comes back with an empty dataset.
  • The replicas reconnect and sync with it.
  • They copy the empty dataset and delete their own copies.

Persistence on the replicas does not help. The replicas copy whatever the primary has.

The Redis replication docs warn about exactly this. The fixes:

  • Turn on persistence on the primary, or
  • Do not let the primary restart automatically. Then Sentinel or an operator can promote a replica to primary first.

Sentinel alone is not enough. If the primary restarts before the failover timeout, Sentinel can still accept it as primary. So automatic restarts of a primary without persistence are dangerous even with Sentinel.

  • A restarted primary without persistence has an empty dataset.
  • The replicas resync and copy the empty dataset.
  • Persistence on the replicas does not protect against this.
  • Turn on persistence on the primary or turn off its auto-restart.
  • Sentinel does not fully protect you if the restart is fast.