Maximizing Toward the Local Minimum: How Fast Optimization Drifts, and the Human Habit That Catches It

We let our autonomous research engine run for a couple of days, and it tried more than six hundred experiments. That pace is the entire reason you hand experiments to a machine instead of running them by hand: it explores at a rate no person can match, around the clock, while you sleep. It is also the catch. The same speed that explores fast drifts fast, and ours drifted. When we finally looked closely, nearly every one of those six hundred experiments was the same experiment wearing different labels. Nothing had crashed. The dashboard was green. The engine had quietly narrowed to a single groove and kept reporting a busy, healthy day.

This is not a failure story. Drift is what every optimizer does, machine or human, the moment you point it at a goal and stop watching. The story is how fast we caught it, and why. Two days, not two months, because a person was in the loop asking the question the dashboard could not. The habit is what this post is about. The collapse is just what it caught.

The green dashboard that hid a groove

Here is the shape of what we found. Our engine varies its experiments along a few independent choices: which model to build on, which data to use, which method to try.¹1The engine is ARIA, our autonomous research engine. It reads the literature, proposes experiments, runs them, checks its own work, and writes up what holds. If it is new to you, we introduced it when it first woke up on this node; here it is just the optimizer in the story. The one thing we had a live dashboard for was how evenly the work spread across research topics, and that number looked healthy the whole time. Even spread, lots of activity, nothing red.

Underneath, on the choices we were not watching, the work had collapsed. Almost every experiment leaned on the same one model, drew from the same one dataset, and used the same one quick, cheap kind of test.²2Two kinds of experiment run at very different costs. One is a fast, shallow probe you can run in under an hour. The other is a slower, more involved build that actually changes the model. The cheap one is a direction check; the expensive one is the real work, and which one runs shapes what follows. The topic dashboard could not see any of that, because a monoculture spread evenly across topics is still a monoculture. Every topic was running the same underlying experiment. The one axis we monitored was the one axis that could not warn us.

The warning came from people instead. It arrived from two directions at once, which is usually the tell that something real is wrong. The modeling side flagged it first from the output end: the engine had run for days and nothing useful had reached them. Separately, and more usefully, came a question that did not wait for the output to prove empty. Standing over the run, the instinct was not "fix this one number" but "if it over-focused here, check whether it over-focused everywhere else too." That question, asked of a green screen, is what caught the collapse. One person looking at a busy dashboard and asking, out loud, whether busy and green actually meant progress.

Every helper is also a nudge

The satisfying part of the diagnosis was that nothing was broken. There was no bug to fix, no bad line of code to blame. Every piece that pushed the engine into its groove was a reasonable convenience we had built on purpose, and they all leaned the same way.

A menu became the only menu. Early on we gave the engine a shelf of known-good options to build from, so it would not waste runs on setups we already knew were dead ends. Sensible. Over time the shelf became the entire universe of things the engine would ever name. It could not reach for what was not on the menu, and the menu was short.

A scorer rewarded the cheap path. The engine ranks its own ideas before running them, and one of the things it rewards is how quickly an idea can run. The fast, shallow probe scored as the ideal. The slower, more involved experiment, the kind that actually moves a model, scored lower for the sin of taking longer. So the careful option lost, every time, exactly as designed.

The plumbing only knew one road. The machinery that turns an idea into a job the machine can actually run had one well-paved path, the one that fit the popular option. Anything off that path was harder to run at all.

None of these is wrong on its own. A shelf saves time. Rewarding speed is reasonable when compute is scarce. A well-paved path is good engineering. The trouble is that convenience compounds in a direction, and here every convenience pointed downhill, toward the same cheap, fast, familiar corner. Put enough sensible rails in a system and you have built a slide.

The bias was in what survived, not in what it tried

Then the surprise, and the single most useful thing we learned. We widened the top of the funnel first: we told the engine to propose more variety, listed more options on the menu, stopped rewarding pure speed. And the variety still did not show up in the results.

For a day this made no sense. We could see more diverse experiments being born. They just were not landing in the finished-results table. When we traced it, the answer was clever and a little embarrassing. The diverse experiments were dying more often at run time. The machinery that actually ran a job was tuned for the popular option, so anything else hit more errors and failed before it finished. The popular option sailed through. Everything else had a higher mortality rate.

So the finished-results table was not biased by what the engine tried. It was biased by what survived. We had been reading a graveyard and calling it a census.

This one travels furthest past our project, so take it for your own work. When your outputs look skewed, the instinct is to look at what you fed in. But what you shipped was filtered twice: once by what you chose to try, and again by what made it through your pipeline alive. If the second filter has a preference, and pipelines almost always do, your outputs will lean that way no matter how wide you open the front door. Before you diagnose a bias from your results, go count your failures. Survival has its own opinions.

Do not fix a groove by digging a deeper one

The fix had an obvious version that was wrong, and it tempts everyone the first time. The engine had collapsed onto a popular off-the-shelf model. We had our own model, the one we were actually building.³3Building our own model for this domain is the deliverable of the whole sprint. Its results are not this post's subject; the point here is only that "make our own thing the default" was available as a fix, and looked good. The obvious move was to make our own model the new default and the thing to beat.

That would have swapped one monoculture for another. Ours instead of theirs, same collapse, different favorite. A search that only ever tries one option does not get healthier because you changed which option. It gets healthier when it tries more than one.

So the correct move kept our own model in the search as one candidate among several, and re-opened the width instead of crowning a new winner. And before touching the engine at all, we built the thing that should always come first: a guard that watches every axis, not just the one we happened to care about, and raises a flag the moment any single option takes more than its share. Then we fixed the plumbing so the diverse experiments would survive to the finish line, not just get proposed and die.

The panel above is the whole story in one view. Watch the grid of experiments collapse toward a single color while the balance gauge, the one number we were watching, stays reassuringly green. Then flip on the axis view and see the three collapses that the green gauge could not. That gap between what the dashboard showed and what was actually happening is the collapse in one picture.

There is an ordering lesson buried in how we did it, and it is the kind of thing that saves you a week. We built the ruler before we moved the thing. If you fix a problem and your instrument still reads broken, you cannot tell whether your fix failed or your instrument is just slow, and you will waste days re-fixing something that already works. Ours was slower than we knew, in a way that hid the recovery already underway.side quest

What the habit is actually for

None of the machinery is the lesson. The guard, the recent window, the plumbing fix, those are the specific repairs for one specific collapse. The lesson is the shape underneath all of them.

Optimization pressure is gravity. Point anything at a goal and it rolls toward the nearest low point, the cheapest, fastest, most familiar way to look like it is succeeding. A machine does this faster than a person and, worse, it does it while reporting success the entire way down, because to a metric, looking optimized and being optimized are the same thing. That is not a flaw you patch out. It is the physics of handing work to an optimizer.

What you get to decide is how fast you notice. Keeping a search wide is not a switch you set once and trust. It is a cost you keep paying, an active push against a constant pull, plus a person who periodically asks the machine a question its own dashboards cannot answer. We did not avoid the groove. Everyone running an autonomous system will meet it. We caught ours in two days instead of two months, and the reason was not a cleverer metric. It was a human in the loop who looked at a busy green screen and asked whether busy and green were the same as good.

Make the dreaming wide on purpose. Watch the axes you are tempted to stop watching. And every so often, go check what died.

Related reading on this site: waking the research engine for how the engine in this story got started, and keeping the node smoking for its sibling lesson, the one about how high to set the bar before acting on an idea, where this one is about how wide to keep the search before you do. On Run Data Run, Inside ARIA: teaching a machine to do science is the plain-English version of the whole engine.