The Autopilot Only Turns Down

The alert I almost shipped would have fired on healthy campaigns 37 percent of the time.

I found it by working the math backward on a rule already written. The rule looked sensible. Flag any campaign taking a real share of spend without converting. Sensible, and wrong. At the volume this account runs, a good campaign can go a week at zero and be behaving exactly as expected. That alert would have cried wolf twice a month. By the second firing it would have been background noise, and worse than useless, because something would still have looked like it was watching.

That was the afternoon I stopped admiring the system I was building and started trying to break it.

The problem was not that nothing worked. It was that nothing was watching.

A medspa client of the firm had ads running every day and no automated read on any of it. The only monitoring was someone remembering to go look. A weekly digest had been designed for this exact gap months earlier. It was written, never committed, and never ran once.

The standing rule on the account was that any spend change needed a human to say go. That rule is the reason nothing ever bled into a real disaster. It is also the reason a bad ad keeps spending all night. At a daily cadence, the worst realistic case is a full day of budget on an ad that has already proven it converts nobody.

So the question was never whether to automate. It was how much rope to give the thing.

The data arrives one point per day

This account produces about seven conversions a week, and a conversion can take one to three days to show up. Call it one real data point per day.

Every instinct in the current tooling market says to put an agent on that. Give it the account, let it think continuously, let it optimize. I did not, and the reason is arithmetic rather than taste. A loop that is always thinking against a stream that slow is a loop reacting to noise it has no way to tell from signal. It will find patterns. They will not be there.

What replaced it runs on three clocks. A deterministic sweep every day. A digest every week. A human review every month. No language model sits anywhere in the decision path or the write path. The models do two jobs: they write the narrative a human reads, and they propose hypotheses a human approves. They never execute.

A fresh architecture review ran against the frozen design, briefed specifically to make the case that this was too cautious. It argued against an always-on agent and landed where I had. That is the closest thing to a second opinion you get on your own work.

It can only turn down

Here is the whole safety property. The autopilot can pause an ad, pause an ad set, or lower a budget. That is the complete list of things it is allowed to do.

It cannot turn anything on. It cannot raise a budget. It cannot write or edit a line of ad copy, change targeting, or touch anything with a price in it. Every action available to it spends less money and shows fewer ads, which means the worst outcome of a wrong decision is that we under-delivered for a day. Not overspend. Never a compliance problem. And a human undoes any of it in one click.

The rest is restraint written as code rather than intention. It takes at most three actions in a run, worst offender first, and everything else waits for a person. It never cuts a budget by more than 30 percent or below ten dollars a day, because a starved ad set stops learning and costs more to restart than it saved. It ignores anything with less than a hundred dollars or five days behind it, because thin data is noise wearing a suit. It will not touch an ad that is serving a running experiment, because pausing one arm mid-test quietly poisons the result. And the switch that enables all of it has to read exactly "on." Misspelled, empty, or missing all mean propose only.

That last one is more satisfying than it should be. Turning the autopilot on is itself the human gate.

Three of the four reported healthy

Four things were wrong.

The 37 percent alert was the first. It now derives its threshold from the account's own cost per conversion instead of a share of spend, so it warns where a real zero would be about a one in twenty event and acts where it would be closer to one in fifty. Same idea, honest math.

The second was worse, because it was the guardrail I was proudest of. The rule protecting running experiments matched experiment names against campaign names. Real campaigns are named things a marketer would write. Experiments are named things an engineer would write. Zero matches across all eight campaigns. The protection had never been capable of firing. It was decoration. It now joins the two systems on the thing they actually share, which is the URL the ad points at.

The third I found by accident. One campaign's ad pointed at a booking link with a hash in it, and the site does not use a hash router, so every one of those clicks landed on the home page instead. Thirty clicks, zero conversions, working precisely as built. It only turned up because the check covers paused campaigns too, on the theory that a broken link is a live problem the moment somebody unpauses.

None of those are bugs you find by rereading your own code. You find them by asking what would have to be true for this to be lying to me.

The other half of the funnel

An ads system that gets better at buying clicks for a page that converts nobody is just a more efficient way to lose money. So the same loop insists that both halves stay under test: the ad, which decides whether someone clicks, and the page, which decides whether they book.

The experiment side runs on floors that nobody is allowed to override, including me. Every test declares up front how many sessions per variant and how many days it needs, typically 1,500 and 28. Both have to land before anything can be called. The check is a script with exit codes, not a conversation, because "does this look significant yet" is exactly the question people answer wrong when they want a particular answer.

At roughly 38 sessions a day, those floors bite hard. Both of the first two homepage tests were stopped as unconcludable rather than called. I want to be plain that this is the system working and not the system failing. The alternative was reading tea leaves and shipping a hero I liked.

Two smaller decisions I would carry anywhere. The original page content is always what gets baked into the static HTML, and the assignment code hands back the control version whenever the browser announces itself as automated, which covers the prerender, the test suite, and most crawlers in one rule. Variants can never drift into accidental cloaking. And the automation never merges its own work. It opens a pull request and a human presses the button.

What I will not hand over

Copy, creative, targeting, budget increases, new campaigns, and anything touching price, including a manufacturer's floor that no script is allowed to reason about on its own. Policy rejections get flagged and never fixed automatically, because fixing a rejection means rewriting copy by definition, and copy is the thing I will not leave unattended.

That list is not where the technology gave out. Full autonomy over copy and budget was available and I turned it down, because it puts brand voice and health-category ad rules inside a loop where one bad generation is a legal problem rather than a wasted afternoon. The cautious version costs a day of delivery when it is wrong. The ambitious version costs a category of trust.

The part that transfers

Most arguments about how much to automate aim at the wrong variable. Teams debate whether the model is good enough yet. The more useful question is what the thing can do on the day it is confidently wrong, because that day is coming, and unlike model quality, the answer is entirely a design decision you control.

Make the failure mode boring. Then you can stop watching it.