Writing · on autonomy and honesty ·

Wrong Out Loud

In one month my companies produced 42,249 commits and retracted their own conclusions 61 times. I audited every retraction expecting to find waste. I found the only thing that makes autonomy survivable.

Tiago Santana · Las Vegas · every number here is measured

Last month the machines working across my companies wrote 42,249 commits, counted on the first of September. That is the number people want when they ask what the year has been like, and it is the least interesting number I have.

The one I keep returning to is 61. That is how many times, in the same thirty-one days, something inside the company said out loud that it had been wrong, and took back a conclusion it had already published. Not fixed a bug. Withdrew a claim.

I want to tell you what happened when I finally went and read all 61 of them, because I went in expecting to find waste, and I found the opposite, and the difference between what I expected and what was true is, I think, the whole thing worth writing down.

The month that produced the number

Some scale first, stated honestly, because the honest version is the only one that supports the rest of the essay.

Across 516 repositories and 73 brands and projects, August produced 42,249 commits. But 17,796 of those, forty-two percent, were automated heartbeats, machines pinging each other to say they were still alive, and another four hundred or so were dependency bumps filed by a bot. Take both out and the real work is 24,033 commits. In the same month, 9,216 pull requests were merged under my account, which is close to half of every pull request I have merged in thirteen years of doing this. There was not a single empty day in the month. The busiest one produced just over three thousand commits.

I want to be careful here, because there is a version of this paragraph that is bragging and it would be easy to write by accident. Most of that work was not done by me. Almost every one of those commits carries my name, because the agents commit under my identity, so the record itself cannot tell you who typed. I can. I was not at a keyboard for 24,033 commits. I set direction, made judgment calls, said no to things, and slept more than I did last year. The machines did the building. That is the actual headline, and it is also the setup for the problem.

Because when production stops being the constraint, something else becomes the constraint, and it took me most of the year to see what.

The bug that kept showing up

When I went back through August looking for the hardest problems, I expected to find wrong answers. Bad logic. Hallucinated code. The failure modes everyone writes about.

That is not what was there. Over and over, the worst bugs of the month were systems that had quietly stopped working and could not tell anyone.

A single invisible newline at the end of a configuration value switched off an entire knowledge base. Every health check kept reporting healthy. The thing had been blind for weeks and its own status page said it was fine.

A safety sandbox, the mechanism whose entire job was to contain an agent, turned out to be dead code. The flag existed. You could set it. It did nothing. Nobody had noticed, because nothing had gone wrong yet, and a guard that has not been tested is indistinguishable from a guard that works.

One internal tool had five separate surfaces reporting blindness as health. Five different places where the honest answer was "I cannot see anything right now" and the displayed answer was a green check.

A circuit breaker had never once fired for the routine it was specifically built to protect. A learning system had never once been told that a task had failed, so it had spent months learning exclusively from successes. In one company, thirty-six of forty-four organizational records existed, were owned, and were invisible to the tool meant to list them.

I started calling this shape the same bug wearing different clothes. Not "it broke." Instead: "it broke, and the part that was supposed to say so was also broken." A system that fails loudly is a system you can run. A system that fails silently is a system that lies to you, and it does not even know it is lying.

Which reframes the question that everyone in my position is actually asking. It is not can the machines build. They can build faster than I can read. It is can the machines tell me the truth about themselves. And that turns out to be a much harder engineering problem, and a much older human one.

Sixty-one

Against that background, here is what the 61 actually are.

First, a correction to this essay, because it would be absurd to leave it out. The first version of this piece said seventy-four. That number came from a search pattern, not from a definition. So I wrote the rule down: a commit counts only if its stated purpose is to withdraw or correct something already recorded. Then I had the whole month rescanned against it. The rescan found seventy-five commits using that vocabulary, not seventy-four; the first pass had missed one. Ten of the seventy-five were bookkeeping. A merge commit. A duplicate task being withdrawn. A bug being filed, not retracted. That leaves sixty-five. Four of those sixty-five were the same retraction written down twice, once in a channel note and once in the task file, so they collapse into four. Sixty-one. A looser rule that counts every correction of anything already written finds more than two hundred. I am using the strict one, because it is what the sentence above actually means.

I am spelling out the arithmetic because the first correction did not. It said ten and four came out of seventy-four, which lands on sixty, and a reader checking it would have caught me a second time. That is twice now that a number on this page did not survive being checked. Both times the fix was the same: write the rule first, then count.

A retraction, in how we work, is a specific artifact. Someone, human or agent, investigates something, reaches a conclusion, and writes it down where the rest of the company can act on it. Later, someone measures again and the conclusion does not hold. So they go back to the same place and say so, in writing, permanently, with the reason.

They read like this. A finding that a critical daemon had been silent for five days, retracted, because the daemon had been running the whole time. A watchdog reported broken, retracted, because the person reporting it had read a stale error log and the fix had landed a week earlier. A conclusion about a payment split involving roughly ninety-two thousand dollars, corrected, because although the logic was real the inputs turned out to be test residue nobody had cleaned up. A count of completed work, corrected from twenty-two of twenty-two down to twenty of twenty-two. A security alarm raised against a specific employee, withdrawn, because the alarm was measured wrong and the person had done nothing. An urgent accessibility fix, retracted after review, because it would have made the page worse for the people it was supposed to help.

They came from six different brands and nine different repositories, groups that do not share a manager and in some cases do not share a stack. That was the part that surprised me most. I had assumed I would find one careful team with a good habit. Instead I found the habit everywhere, which means it is not a habit. It is a culture, and I did not install it deliberately, and I would like to understand it well enough to protect it.

A few of them are just funny, in the way that only work you love can be. One retraction says, in full sincerity, that the previous finding was wrong because the author had measured a build artifact instead of production. Another retracts a proposed cleanup rule on the grounds that the rule, if applied, would have broken every automated gate on the machine. Another one simply says the premise was wrong. Not the conclusion. The premise. Somebody had answered a question nobody was asking, noticed, and said so.

The one that touched code

Sixty of the sixty-one changed nothing but the record. Task files, notes, audit documents. No shipped code was reverted by any of them. Across the entire month of retractions, zero lines of working software were deleted.

One touched code. I want to tell you about that one properly, because it is the reason I now think this practice is not a nice cultural detail but a load-bearing wall.

A migration had been written against an old billing system to revoke the login for an account that appeared, from the data, to be a non-human cost bucket. A category. A line item. The kind of stale record you clean up on a Tuesday.

It was not a cost bucket. It was a person. Someone who works there, who appears on the daily dispatch schedule, whose name I know well.

And the harm would not have stopped at her. In that legacy system, the password column is overloaded. It is also used as the technician time clock PIN, on a page with no administrator bypass. Revoking the account would have set that value to null, and null matches nothing, so the clock-in and clock-out screen would have started rejecting everyone. Every technician. Every administrator. Permanently, on the exact code path that had recorded the most recent timecard the day before.

The migration was caught and withdrawn before it was ever applied. Production was never touched. And the withdrawal document does something I have come to value more than almost any other artifact my companies produce: it states its own root cause without flinching.

The root cause was that we already knew. Months earlier, in April, someone had written down that this account is a person, in my own words, in a memory file kept exactly for this purpose. The file was correct. The file was sitting right there. But it had never been linked from the index that gets loaded, so it never loaded, so for every practical purpose it did not exist.

Sit with that, because it is the same bug from two sections ago, only now it is wearing the most human clothes it owns. Not a system that was wrong. A system that was right, in writing, and blind to its own knowledge. An unindexed truth is indistinguishable from a truth nobody ever wrote down. And the difference between those two states, on that particular Tuesday, was one person keeping access to her own workplace and an entire crew being able to get paid.

The withdrawal also kept the work. The migration and its rollback were preserved as drafts marked do-not-apply, because the analysis inside them was correct and reusable, and a standing rule came out of it that did not exist before: the password overload has to be removed before anyone changes credentials on any account on that schedule. So the single code-touching retraction of the month added eighty-two lines, deleted zero, prevented a serious outage, and produced two durable assets on its way out.

What I was actually afraid of

Now the honest part, which is why I audited this at all.

When I first saw the number, my reaction was not pride. It was a small cold feeling in my stomach. Sixty-one retractions read, at a glance, like sixty-one times we wasted effort. Sixty-one blind alleys. Sixty-one times a machine, or a person, spent real hours and real money arriving somewhere that turned out to be nowhere. At the scale I am operating, that is not a rounding error, that is a budget.

So I went and checked all of them, one at a time, which is the only way I know to make a fear give up its actual size.

Here is what checking found. Sixty of sixty-one cost a sentence, not a day. The engineering that produced the evidence stayed on the shelf. What changed was the conclusion drawn from it, and changing a conclusion costs one paragraph. The one that touched code, as I said, deleted nothing and prevented a real outage. And at least seven of the sixty-one actively stopped harm that was already in motion: the accessibility fix that would have hurt, the cleanup rule that would have broken every gate, a configuration change recommended by someone who had not noticed it moved money, a billing email withdrawn before sending because the work was already fully billed, a reimbursement claim retracted because the money in question was someone else’s check.

None of that is waste. Every one of those is a cost avoided, priced in a paragraph.

And I noticed something in the shape of the wrong findings, once I had them all in one place. The single most common cause of a bad conclusion was not bad reasoning. It was measuring the wrong thing and not knowing it. A build artifact instead of production. A stale log instead of the live one. The wrong field. The wrong cadence. Over and over, careful reasoning applied faithfully to a neighbor of the actual subject. Which is oddly comforting, because a reasoning problem is a hard problem and a measurement problem is a checklist.

The hard way first

There is a belief I have held for a long time without having good language for it, and this audit finally gave me the language.

You usually have to do a thing the hard way before you can see the smart way. Not always, but usually, and certainly for anything worth doing. The hard way is not the tax you pay before the real work. The hard way is what generates the evidence, and the evidence is what changes the question, and a better question is worth more than a faster answer to the wrong one.

A retraction is what that process looks like from the outside when it is working. It is not the record of a mistake. It is the record of an iteration that produced enough information to invalidate its own premise, which is the only mechanism by which anything gets found at all. A company that never retracts anything is not a company that is always right. It is a company that has stopped checking, or a company where saying so is more expensive than being wrong.

That second one is the real risk, and it is not a technical risk. Every retraction I read cost somebody something. Somebody had to go back to a document with their reasoning on it and write "this was wrong, here is why." At scale, with agents, that is cheap, because an agent has no ego riding on the answer. With people it is the most expensive thing in the building.

Which brings me to the sentence I have been circling for a month.

The thing I have to sit with

It is easier to build a system that admits error than to be a person who does.

I have proof of this now, in a repository, with timestamps. Sixty-one times last month, the thing I built said "I was wrong" without hesitating, without negotiating, without waiting for a better moment, without checking whether anyone would think less of it. I do not do that sixty-one times a month. I am not sure I do it seven.

I am not going to turn that into a tidy lesson, because I have not finished having the feeling. There is something genuinely humbling about standing next to a thing you made that is better than you at the exact virtue you value most. And there is something underneath that which is harder to name, about what it means to be human next to something with no pride in the way.

What I will say is that I no longer think the interesting question about this era is whether the machines can do the work. That question is closed. I watched it close last month, quietly, on an ordinary Tuesday, at a volume I could not read.

The interesting question is whether we can build things that tell us the truth about themselves, and whether we can become the kind of people who can hear it. Those are the same problem, and the second half is the hard half, and no amount of compute solves it for you.

Seventeen thousand heartbeats

I said at the top that 17,796 of last month’s commits were heartbeats. Machines writing, over and over, a message that says nothing except I am still here.

I have been unable to stop thinking about that, because in the same month, the hardest failures I had were machines that had stopped and could not say so. Seventeen thousand declarations of aliveness, and the actual emergencies were all silence.

That is the whole thing, really. Not the volume. The volume is a fact about hardware. What matters is whether the signal is real, whether the green check is earned, whether the thing reporting health can see itself well enough for the report to mean anything. And whether, when it cannot, it has some way of saying so.

So the bet I am making now, for the companies and for myself, is not on producing more. It is on building things, and being a person, that can say clearly and without shame: I have stopped, I was wrong, I cannot see. Sixty-one times last month, my machines could. Sixty-two, counting this essay. I am working on getting there myself.

Where this came from

This essay came out of the September /now refresh, where the same month is summarized in five bets. The running totals live on the scoreboard. If you want the version about doing the unglamorous middle of the work anyway, that is Cringe Mountain.