the backup that was empty for weeks

A backup job is a producer with no consumer. Nothing in the system notices when the archive is empty. The job either exits zero, and the emptiness is invisible, or it fails into a log nobody reads, and the schedule keeps firing regardless.

Both happened here. A weekly archive job failed on every run for weeks because of a flag the environment silently ignored — the job had no home directory, so the setting that was supposed to prevent the error did nothing. Separately, a nightly archive shipped an empty directory because the staging step resolved a path one level higher than intended: the file was created, it was simply not the file anybody wanted.

What caught both was a weekly job whose only purpose is to restore. It unpacks the archive into a scratch space, restores the database, compares counts against the manifest the backup wrote, and fails loudly when they differ. On its first run it found the empty directory, which is exactly what it was for.

The other half is gating. A migration only starts once a restore has been proven on a scratch instance and the counts match, because a restore you discover is broken while cutting over is not a restore.

What this proves is that the artifact is not the backup; the restore is. A team that has scheduled the restore has a different relationship with its data than one that has scheduled the copy and hoped.