Ordering and global position
This page explains the global position, and the design choice that makes it safe to read.
Every event has a global position
Section titled “Every event has a global position”Positions order every event of the store. Deedbox assigns them without gaps, in commit order: an event at position 11 commits after the event at position 10. So when a reader sees position 11, position 10 is already committed. A reader that asks for “everything after my checkpoint” never misses an event that commits later.
Compare positions; never do arithmetic on them. Deedbox assigns every position without a gap. Deleting a stream removes that stream’s earlier events, so the stored positions can have gaps afterwards.
One counter serializes appends
Section titled “One counter serializes appends”Deedbox takes positions from a single counter row. Each append updates the counter as its last statement and commits right after. The row stays locked in between, so the next append waits. A rolled-back append also rolls back the counter, so no position is ever lost.
In one append, in this order:
- Lock and update the stream row.
- Run inline projections and appending hooks.
- Update the position counter, and check this instance’s heartbeat row in the same statement.
- Insert the events at the new positions.
- Commit, which releases the counter.
The cost is that appends take turns for that short window. See the benchmarks for the ceiling. A transaction you own keeps the counter locked until you commit, so commit soon after an append. When Deedbox owns the transaction, its commit does not take your cancellation token, so a cancelled request never leaves the write in doubt.
Reads stop at the committed head
Section titled “Reads stop at the committed head”Every read by position, such as the runner’s, first reads the counter, then reads only events at or below it. Every position at or below the committed counter belongs to a committed append, so the read never meets an append in flight.
That matters on SQL Server without READ_COMMITTED_SNAPSHOT, where reads take locks. A read that waits on an append in flight can resume past positions that the next append reuses after a rollback, and return a later position first. Bounding the read by the counter prevents that. On Postgres and with READ_COMMITTED_SNAPSHOT, reads see committed rows only, and the bound changes nothing.
If your own SQL reads the events table by position on such a SQL Server, bound it the same way. Read the counter into a variable first, then read up to it, in one batch:
DECLARE @head bigint = (SELECT value FROM [deedbox].[position]);SELECT global_position, event_id, event_type FROM [deedbox].[events]WHERE global_position > @after AND global_position <= @head ORDER BY global_position;Why not a sequence or a row version
Section titled “Why not a sequence or a row version”A sequence or row version hands out numbers before commit. Two transactions can then commit out of order, and a reader can pass a number that commits later. Other stores guess when such a gap is safe to skip, and several have shipped bugs that skipped events. Deedbox never skips; the counter makes that guess unnecessary. A torture suite checks it on both databases with rollbacks, long transactions and competing readers.