field notes
Short notes from real engineering work: one decision, one incident, one reversal at a time. Each one names the shape of the problem, the tension that made it hard, the decision, and what it cost. The internals stay out — the codename is the hint.
If you have five minutes rather than an afternoon: the three case studies condense this material to one problem, one decision and what the decision cost. The notes below are the record those studies cite — and the projects the codenames belong to are on the front page.
A second factor is only as strong as the secret behind it. If the material that protects every enrolled user's authenticator state lives in configuration with a fallback, the control has a quiet ceiling, and the ceiling is whatever the…
Removing a credential from a repository is a small, satisfying act that answers almost none of the question. The question is whether the tree you are about to ship still contains one, and whether the strings that look like credentials are…
A request-forgery guard is a cheap, unglamorous control: refuse a mutating request unless it carries a header that a page on another site cannot set. It works precisely because the browser will not attach it on someone else's behalf.
A browser security policy is easy to write and expensive to switch on. Every directive that closes a hole is also a promise that the application already behaves well, and this application was written before anyone made that promise.
A credential reached a repository. The usual response is to remove it, which is necessary and mostly useless: the value is in the history, in the clones, in the mirrors, and in everyone's working copy. What was added alongside the removal…
A scheduled backup job ran green for as long as anyone cared to look. Green meant the snapshot was written and the tool's own integrity check passed. It did not mean the data could come back, because nobody had tried, and a snapshot that…
A venue's public page needed a small configuration payload — name, tagline, hours, delivery parameters — served as structured data. The route was added and did not work. It did not fail either. The static file handler had a catch-all that…
The operator console had to show everything: every venue, every deployment, the state of the cluster, the register of what exists. An aggregate view like that is the highest-value surface in the system, because one page compresses what…
Payment providers retry. Customers double-tap. Networks drop the acknowledgement, not the request. On a platform that turns a payment acknowledgement into order state, the failure mode is not an error page — it is a second order, a second…
Two facts decide what a customer is charged: whether their address falls inside a delivery zone, and whether a discount they claimed still applies. Both arrive from a browser, and both are editable in a browser. The version of this feature…
Privacy law gives any person the right to ask what a business holds about them, and to have it exported or erased. On a platform where most orders come from guests who never created an account, the intake surface for those requests cannot…
One server process serves several scopes: the public storefront, the marketplace, the authentication pages, the staff tools. Each had a host check — the request must arrive addressed to exactly the host this deployment serves. Strict,…
The platform shipped its public surfaces before it shipped a limit on how often they could be asked. That is the ordinary order of events, and the review said so plainly: no request budget at all on the server facing the public. The fix…
The public marketplace needed to show, for every venue, a small card of near-live facts: a rating, a delivery fee, an arrival estimate. The obvious implementation is a query — the page asks the database whatever is true right now. The…
The useful property of a ranked defect inventory is that it can be re-read. The harder discipline is reading it again and writing down how little has changed, in the same order, with the same numbering, so the gap between what was claimed…
A multi-surface platform — a public marketplace, several venue sites, staff tools, an operator console — accumulated hundreds of commits before anyone read the whole thing with a hostile reader's patience. The review that followed was not a…
The first service migration looked like a copy and a connection string: take a database out of the application's container, run it as a supervised process on the host, point the application at it. The move took far longer than that, and…
A machine whose remote-access service stops accepting connections is not down. It is running, hosting its workloads, answering everything that asks about its health, and unreachable through the only door its operator uses. That is the shape…
When new machines are built to replace an old one, the old one becomes the reference, and the comparison that matters is not between files. A parity audit set the retiring member against the freshly built replacements, and the configuration…
A periodic job is two contracts, and only one of them gets written down. The first is what it does. The second is where it is allowed to do it, and that one stays implicit in the job's body: it reads a directory on a local disk, or a socket…
method
Not a changelog entry and not a repository summary. One decision, one incident, one reversal, or one hard-won invariant — the thing an engineer would tell you at a whiteboard.
Prefer records with dates and root causes? Those are on incidents. Live numbers are on stats.