an empty reply is not a quiet day

The archive is the ground truth for every later measurement, so the way it fails matters more than the way it works. The failure found was quiet. The upstream service would occasionally answer a data request with a successful, well-formed, completely empty payload. The puller accepted it. Written down, the archive now held a day on which apparently nothing had happened — and nothing downstream could distinguish that from a day of genuine quiet.

The fix inverted the default. An empty reply for a day the system has reason to believe exists is a fault, never data: cool off, retry, and if the answer stays empty, exit and let supervision restart the puller rather than write a hole into the record. Only the few periods that are genuinely empty are allowed through, and they are allowed by name rather than by inference.

Around that came the operational cost. A cap on concurrent pullers so the service is not hammered. State rebuilt from what is already on disk rather than from memory. A watcher that notices when the archive has stopped growing and restarts what has stalled. A post-pass verifier that walks the finished archive looking for the holes any of the above might have missed.

None of this is interesting until a year of results rests on it, at which point it is the only thing that matters.

It proves the team treats missing data as a claim to be tested, not as an absence to be tolerated.