the replay that disagreed with reality by sorting

The requirement was absolute: a replayed stream had to be byte-identical to what the live system emitted, or nothing built on the replay could be believed. It nearly was. A full-scale comparison across millions of records showed a small number of decisions missing and a slightly larger number present that should never have existed.

The cause was an assumption rather than a bug in the ordinary sense. The replay source sorted events by timestamp before delivering them, because the live feed is timestamp-ascending by construction and sorting looked like a harmless normalisation. It is not harmless. The historical record contains backfill blocks appended out of order, where a stretch of earlier time is written after later time has already gone past. Sorting collapsed those into the same moment and manufactured decisions the live system never made.

The correction was to deliver records in the order they physically appear, and to make the engine flush only at the end of the stream, where the ordering rule the reference uses becomes exactly reproducible. A test now locks the property in: a replay must survive an out-of-order backfill without inventing anything.

Worth noting what it cost to find. The synthetic domain passed cleanly. Only the real, messy archive exposed it, and only at full scale.

It proves the team treats identical as a literal requirement and hunts the last few disagreements instead of accepting a small tolerance.