CodePark| 1 bit | 41 bits | 10 bits | 12 bits |
| sign | ms since custom epoch | machine id | sequence |
Use the Snowflake layout. Each machine builds 64-bit ids from a timestamp, its machine id and a counter, with no coordination between machines.
Why not the simple options?
- A database auto-increment is a single point of failure and a bottleneck.
- UUIDv4 needs no coordination. But it is 128 bits and random, so it can't be sorted. Random ids also fragment B-tree indexes, because new rows land in random places in the index.
- UUIDv7 fixes the ordering, but it is still 128 bits.
The Snowflake layout fits in 64 bits:
- 1 sign bit, not used
- a 41-bit timestamp in milliseconds, counted from a custom start date (epoch). This covers about 69 years.
- a 10-bit machine id: up to 1,024 generators
- a 12-bit sequence counter: 4,096 ids per millisecond per machine
Each machine creates ids locally, with no coordination. If the sequence runs out within one millisecond, the machine waits for the next millisecond. Ids sort roughly by creation time, which is good for indexes.
Problems to handle:
- Machine ids must be unique. Assign them from configuration or from a coordination service.
- Clocks can move backwards, for example after an NTP (Network Time Protocol) adjustment. The generator must then refuse or wait. It must never issue duplicate ids.
- Ids are only roughly ordered across different machines.
- Auto-increment is a bottleneck, and UUIDv4 is 128 bits and can't be sorted.
- Snowflake uses a 41-bit timestamp, a 10-bit machine id and a 12-bit sequence.
- No coordination is needed per id, and each machine makes about 4,096 ids per millisecond.
- Each machine must get a unique machine id.
- The generator must handle the clock moving backwards.
Read more