field notes KEEL

every layer that was supposed to be transparent

The first service migration looked like a copy and a connection string: take a database out of the application's container, run it as a supervised process on the host, point the application at it. The move took far longer than that, and none of the delay came from the database.

Three layers that were supposed to be invisible each got in the way. An interception layer in front of the application's network calls captured the outbound connection to the new database address before it left the instance, so the application never reached it, and the layer was removed for this workload. The orchestration template engine interpreted the configuration file's own templating syntax and expanded its shell variables, so the file had to be installed as a plain host file rather than rendered. And the database process would not drop privileges the way the scheduler wanted, so the invocation was rebuilt around an explicit host configuration, an explicit privilege drop and an explicit working directory, which also closed a rendering race and an environment-loss problem that had been producing intermittent startup failures.

Each of those reads as "the database is broken". None of them were. The pilot's real output was a written list of these traps for the migrations behind it, which is much cheaper than rediscovering them one service at a time.

The move was gated as well: a backup restored into scratch space, row counts compared against the source, before anything live was touched. What it proves is that in a migration the time goes into the layers advertised as transparent, and the most valuable artifact a pilot produces is an honest list of the ways they were not.

drawn from: pilot migration cutover commits

Every note carries a codename and one idea. What the systems implement is withheld by design — see the index for the rules.