|
| 1 | +Sequential UUID generators |
| 2 | +========================== |
| 3 | + |
| 4 | +This PostgreSQL extension implements two UUID generators with sequential |
| 5 | +patterns, which helps to reduce random I/O patterns associated with |
| 6 | +regular entirely-random UUID. |
| 7 | + |
| 8 | +Regular random UUIDs are distributed uniformly over the whole range of |
| 9 | +possible values. This results in poor locality when inserting data into |
| 10 | +indexes - all index leaf pages are equally likely to be hit, forcing |
| 11 | +the whole index into memory. With small indexes that's fine, but once |
| 12 | +the index size exceeds shared buffers (or RAM), the cache hit ratio |
| 13 | +quickly deteriorates. |
| 14 | + |
| 15 | +Compare this to sequences and timestamps, which have a more sequential |
| 16 | +pattern and the new data almost always end up in the right-most part of |
| 17 | +the index (new sequence value is larger than all preceding values, same |
| 18 | +for timestamp). This results in a nicer and cache-friendlier behavior, |
| 19 | +but the values are predictable and may easily collide cross machines. |
| 20 | + |
| 21 | +The main goal of the two generators implemented by this extension, is |
| 22 | +generating UUIDS in a more sequential pattern, but without reducing the |
| 23 | +randomness too much (which could increase the probability of collision |
| 24 | +and predictability of the generated UUIDs). This idea is not new, and |
| 25 | +is described as |
| 26 | + |
| 27 | +This idea is pretty much what the UUID wikipedia article [1] calls COMB |
| 28 | +(combined-time GUID) and is more more thoroughly explained in [2]. |
| 29 | + |
| 30 | + |
| 31 | +Generators |
| 32 | +---------- |
| 33 | + |
| 34 | +The extension provides two functions generating sequential UUIDs using |
| 35 | +either a sequence or timestamp. |
| 36 | + |
| 37 | +* `uuid_sequence_nextval(sequence regclass, block_size int default 65536, block_count int default 65536)` |
| 38 | + |
| 39 | +* `uuid_time_nextval(interval_length int default 60, interval_count int default 65536) RETURNS uuid` |
| 40 | + |
| 41 | +The default values for parameters are selected to work well for a range |
| 42 | +of workloads. See the next section explaining the design for additional |
| 43 | +information about the meaning of those parameters. |
| 44 | + |
| 45 | + |
| 46 | +Design |
| 47 | +------ |
| 48 | + |
| 49 | +The easiest way to make UUIDs more sequential is to use some sequential |
| 50 | +value as a prefix. For example, we might take a sequence or a timestamp |
| 51 | +and add random data until we have 16B in total. The resulting values |
| 52 | +would be almost perfectly sequential, but there are two issues with it: |
| 53 | + |
| 54 | +* reduction of randomness - E.g. with a sequence producing bigint values |
| 55 | + this would reduce the randomness from 16B to 8B. Timestamps do reduce |
| 56 | + the randomness in a similar way, depending on the accuracy. This |
| 57 | + increases both the collision probability and predictability (e.g. it |
| 58 | + allows determining which UUIDs were generated close to each other, and |
| 59 | + perhaps the exact timestamp). |
| 60 | + |
| 61 | +* bloat - If the values only grow, this may result in bloat in indexes |
| 62 | + after deleting historical data. This is a well-known issue e.g. with |
| 63 | + indexes on timestamps in log tables. |
| 64 | + |
| 65 | +To address both of these issues, the implemented generators are designed |
| 66 | +to wrap-around regularly, either after generating certain number of UUIDs |
| 67 | +or some amount of time. In both cases, the UUIDs are generates in blocks |
| 68 | +and have the form of |
| 69 | + |
| 70 | + (block ID; random data) |
| 71 | + |
| 72 | +The size of the block ID depends on the number of blocks and is fixed |
| 73 | +(depends on generator parameters). For example with the default 64k |
| 74 | +blocks we need 2 bytes to store it. The block ID increments regularly, |
| 75 | +and eventually wraps around. |
| 76 | + |
| 77 | +For sequence-based generators the block size is determined by number of |
| 78 | +UUIDs generated. For example we may use blocks of 256 values, in which |
| 79 | +case the two-byte block ID may be computed like this: |
| 80 | + |
| 81 | + (nextval('s') / 256) % 65536 |
| 82 | + |
| 83 | +So the generator wraps-around every ~16M UUIDs (because 256 * 65536). |
| 84 | + |
| 85 | +For timestamp-based generators, the block size is defined as interval |
| 86 | +length, with the default value 60 seconds. As the default number of |
| 87 | +blocks is 64k (same as for sequence-based generators), the bloc may be |
| 88 | +computed like this |
| 89 | + |
| 90 | + (timestamp / 60) % 65536 |
| 91 | + |
| 92 | +Which means the generator wraps around every ~45 days. |
| 93 | + |
| 94 | + |
| 95 | +Supported Releases |
| 96 | +------------------ |
| 97 | + |
| 98 | +Currently, this extension works only on releases since PostgreSQL 10. It |
| 99 | +can be made working on older releases with some minor code tweaks if |
| 100 | +someone wants to spend a bit of time on that. |
| 101 | + |
| 102 | + |
| 103 | +[1] https://en.wikipedia.org/wiki/Universally_unique_identifier |
| 104 | + |
| 105 | +[2] http://www.informit.com/articles/article.aspx?p=25862 |
0 commit comments