Choosing the Model Should Not Be Your Job

Here is a decision you make every day and never think about.

You sit down to work. You have several AI subscriptions. Inside each one, several models. Some are fast and cheap to your quota. Some are slow and expensive and genuinely better at hard problems. Before every single task, in principle, you should pick the right combination.

In practice, nobody does that.

You pick one, once, and you leave it. It becomes your default for the rest of the year. And from that moment on you are wrong in both directions all day long: a heavyweight model rewriting a button label, a lightweight one asked to untangle a race condition at four in the afternoon.

That is the irritation the AI Router came from. Not an ambition to build a clever classifier. An irritation about a decision that is asked of you constantly, that has a right answer each time, and that no human being is going to keep making properly.

The choice nobody makes well

The reason the default wins is not laziness. It is arithmetic.

Making the right choice requires knowing, for this specific task, how much reasoning it actually needs. That is a judgement. Judgements take a few seconds. A few seconds, dozens of times a day, across a year, is real time — and the payoff for any single one of them is invisible. You never see the version of the afternoon where you picked better.

So the rational move for a person is to stop choosing. Pick the strongest model you can afford and eat the waste, or pick the cheapest and eat the disappointment.

Both are bad. Both are what everyone does.

What makes this fixable in Open Free Max specifically is that we sit in front of several subscriptions at once. Most tools are built on top of one provider and one bill. We drive the official agent CLIs you already pay for, side by side. The question "which of these should do this task" is one we are uniquely placed to answer, because we are the only thing in the room that can see all of them. The resulting proposal is visible in the AI Router panel before you start anything.

A Tuesday, before and after

It helps to make this concrete, because described abstractly it sounds like a minor convenience.

Take an ordinary Tuesday. You fix a label in the morning. You write a migration note. Someone asks why the nightly export loses rows when two clients import at once, and you spend an hour on it. You clean up a spreadsheet before a meeting. You draft a section of a document you have been avoiding.

Five tasks. Five genuinely different amounts of thinking required. One model setting, chosen in March, applied to all of them.

The label fix ran on something powerful enough to design a distributed system, and burned quota you will want back on Thursday. The export question ran on the same thing, which was correct, but only by accident. The spreadsheet cleanup ran on it too. And if your March choice had gone the other way — a fast, cheap default — then the export question got a model that skimmed it, gave you a plausible answer, and cost you the hour anyway.

Neither version of that Tuesday is a disaster. That is exactly why it never gets fixed. It is a slow, invisible tax: a bit of waste here, a bit of shallowness there, none of it bad enough to make you go and change a setting.

After: you describe each task and stop thinking about it. The label fix goes somewhere small. The export question goes somewhere that will actually sit with it. You did not arbitrate, and you did not spend your Thursday quota on a button label.

What we wanted it to feel like

We wrote the target down early, in one sentence, and it did not change:

You describe the work. The right agent starts, at the right level of thinking.

No dropdown. No model names to memorise. No mental model of which provider is good at what this month. Type what you want done, hit launch, get the appropriate agent.

That sentence turned out to be a good filter. Every later argument — and there were several — came back to it. Anything that made the user think about models again was wrong, even when it was technically better.

The level had to describe the work, not the worker

The first real decision was about vocabulary, and it looked cosmetic until it wasn't.

Our earliest version described difficulty in the terms our own team uses. Refactors. Concurrency. Architecture. It read well to us. It read well to every developer we imagined using it.

Open Free Max is not only used by developers. It is used by people writing books, cleaning datasets, preparing reports, running an administrative pipeline that has nothing to do with code. For all of them, that vocabulary described nothing at all. Their work simply did not contain the words the product was looking for, and the product treated the absence of those words as evidence the work was simple.

That is the worst possible failure mode. Not refusing to answer — answering confidently, wrongly, and silently, to an entire category of user.

So we rebuilt the idea of a level around what a request asks for rather than what field it belongs to. Compare, weigh, explain, audit, restructure: those are hard, whether the object is a codebase, a manuscript or a spreadsheet. Rename, reformat, shorten, replace: those are mechanical, whether you are renaming a variable, a chapter or a column.

"Rename the variable", "rename the chapter" and "rename the column" are the same job. Once we started saying that out loud, the design got much simpler — and it started serving the ghostwriter as well as it served us.

The three levels the product ships with are deliberately plain: quick task, regular work, hard thinking. No jargon. Nothing to learn. A user who has never heard the word "model" can read that panel and know what is about to happen.

The router proposes. You decide.

The second decision was about power, and it was the one we argued about most.

A router that quietly swaps your model is doing something slightly rude. You configured that CLI. You had reasons. Even if your reasons were bad, they were yours, and software that overrides them without a word is software you stop trusting — not loudly, just gradually, until you stop using the feature.

We decided the router would never be invisible.

It shows you the agent it picked, the level it picked, where that decision came from, and the model it implies. Then it waits. Nothing starts until you launch it. If you disagree, the alternatives are right there and one click moves the task to another subscription.

There is a version of this product that skips the panel and just launches the best guess. It would be faster. We did not build it, because the entire value of a routing feature is trust, and trust is not compatible with finding out after the fact.

Two smaller rules came from the same instinct. If you have pinned a model on a provider, the router leaves that provider completely alone — a pin is an explicit instruction and outranks anything automatic. And you can name the level yourself, right inside the task description, in which case the router stops judging and does what you said.

The product's opinion is a default, never a verdict.

One panel, not a settings screen

There was an easier version of this feature, and it would have been much faster to build: expose everything as configuration. Let people define their own rules, their own mappings, their own preferences per project. Ship the machinery, let the user assemble the product.

We have watched that pattern fail often enough to recognise it. A feature delivered as configuration is a feature that works for the twelve people who configure it and for nobody else. Everyone else meets an empty screen, does not know what good values look like, and quietly never comes back.

So the router does something useful the first time you press it, with nothing set up. The settings exist — you can decide which subscriptions may take which levels, and there is a bench where you can paste a task and see what would happen before committing to anything — but they are for tuning, not for switching the feature on. The levels and capabilities guide explains the three plain-language levels in detail.

The test bench turned out to matter more than we expected, for a reason that is about confidence rather than configuration. The fastest way to trust a system that makes decisions for you is to watch it decide about something you already know the answer to. Paste in a task you have done before. See what it says. Now you know what it will do with the way you write.

Nothing changes CLI in the middle of a task

An early idea was to let work move between agents as it ran. It sounds powerful. It is a trap.

A running agent session is a conversation with a history. Move it and you do not move the history, the context, or the state it built up. You get a fresh agent, halfway through a job it did not start, confidently continuing something it cannot see.

So the rule became absolute: routing decisions happen at launch, on a new session, and never once work is underway. Whatever the system decides, it decides it before there is anything to lose.

That single constraint removed an entire family of failure modes we would otherwise have spent a year discovering in production.

The wall that arrives on a Friday

Then there is the other half of the feature, and it comes from a pain that is specific to how our users pay.

If you work through an API, running out of budget is a bill that grows. Unpleasant, but continuous. You see it coming.

Subscriptions do not work like that. They give you a window, and inside that window everything is already paid for, and then at some point the window closes and you are simply stopped. Not slowed. Stopped. Usually at the worst moment, because the worst moment is exactly when you have been working hard enough to exhaust it.

For someone running unattended batches overnight, that is the difference between waking up to finished work and waking up to a job that died at two in the morning and did nothing since.

We could not make the windows bigger. But our users typically own more than one subscription — that is the whole premise of the product. So the answer was sitting there: when one is spent, keep going with another one they already own.

Who gets to decide where your work goes

This is where it got genuinely difficult, and where we ended up overruling our own written plan.

Our original specification said the user must declare, in advance, which subscriptions a project may fall back to. It is the cautious answer. Explicit consent, no surprises.

It is also an answer that guarantees nobody ever uses the feature.

Nobody fills in a form for a capability they have not experienced yet. The setting would sit empty on every installation, the flagship behaviour of the release would be inert, and the only people who would ever benefit are the ones who read the documentation cover to cover. Worse, a form invites you to type in something you have not actually got working, which means the "safety net" could be an agent that fails to start, in the middle of the night, unattended.

So we replaced the declaration with something the user does naturally.

A subscription becomes an eligible fallback on a project once you have already run it there yourself, successfully. That is it. No form.

We like this answer because a single ordinary action proves everything we needed a form to promise: the CLI works on your machine, you are signed in to it, and you have already been willing to let that provider see this particular project. A repository where you have only ever run one agent will never quietly hand itself to a different one.

The scope stays per project on purpose. Trust is not global. You might happily use every agent you own on your own side project and want exactly one of them touching a client's confidential repository.

And because a consent you cannot see is not really consent, the eligible subscriptions for a project are listed on the project header. One click removes one, permanently, and that removal outranks anything the product observed.

Keeping something in hand

There was a temptation, once quota became visible to the system, to squeeze it.

A router that knows exactly how much is left could plan beautifully. Use everything. Land on empty precisely as the window resets. Maximum value extracted from every plan you pay for.

We deliberately did not do that.

Every window keeps a reserve that automatic decisions never touch. Only you can spend it. The reason is simple: the moment a system optimises perfectly, it has no slack, and everything that goes slightly wrong afterwards goes wrong all at once. A provider hiccups, an estimate drifts, a batch runs longer than expected — and with no margin, all of it lands on you simultaneously, at the exact moment you have nothing left.

Leaving something unused feels wasteful on a spreadsheet. It is the difference between a bad afternoon and a stopped one.

When everything is spent, the product would rather step down a level, or park a task that can wait, than eat into what it is holding for you.

Redundancy is a feature we can afford

Most routing systems in this space exist to remove duplication, because for whoever operates them, every extra run is money out the door. Their incentive is to give you one answer, as cheaply as possible.

Ours is the opposite, and it took us a while to notice how unusual that is.

Your plans are already paid for. Running the same question on two of them does not add an OFM API charge. So on a decision that actually matters — an architecture you will live with, a strategy document, a diagnosis you are not sure about — you can tick two subscriptions and get two independent answers to compare.

Each one runs with its own appropriate model, because asking the same question twice with identical settings is just asking it twice.

This is not a trick we invented. It is a structural consequence of building on subscriptions instead of on a meter, and it is probably the clearest example of how the billing model underneath a product shapes what that product is even able to offer you.

What it learns, and what it deliberately ignores

The router pays attention to your project.

Not to everything you do — to the moments you disagreed with it. Override its proposal, or ask a second agent alongside it, and it takes note. Those are deliberate acts. They mean something.

Accepting its suggestion, on the other hand, teaches it nothing at all.

That was a real decision and it goes against the obvious instinct, because acceptance is by far the most abundant signal available. We ignore it because it is ambiguous. When you launch what was proposed, you might be agreeing — or you might have had no opinion whatsoever and simply wanted to get on with your work. Treating that as approval would let the system slowly confirm its own habits using evidence that is mostly noise.

It also forgets at the pace of the work rather than the pace of the calendar. Projects go through phases: exploring, then building, then maintaining. What you wanted from an agent during the first phase stops describing what you want during the third. Preferences that reflect recent work stay useful; preferences anchored to a date go stale while looking authoritative.

And all of it is visible, with a Forget button next to it. Something that shapes your tools and cannot be erased is not a preference. It is an opinion you are stuck with.

The thing we chose not to build

There is one idea we kept coming back to and did not ship, and it is the most interesting one.

Everything the router currently learns is about preference. It knows which agent you reach for. It knows nothing about whether the work was any good.

The thing we actually want to know is whether the result survived. Not whether you accepted the proposal, but whether what the agent produced was still standing a day later, rather than thrown away and done again.

That would be a quality signal instead of a taste signal. It would eventually let the product rely far less on judging your request in advance, and far more on what has genuinely worked for you, on this project, in the past.

We did not build it, because we could not agree on what "survived" means.

Work that gets modified afterwards is not necessarily failed work — most good output gets edited. Work that gets deleted might have been fine but overtaken by a change of direction. Every definition we tried would have quietly punished some perfectly good outcome, and a learning system with a subtly wrong definition of success is worse than one that does not learn at all. It gets confidently better at the wrong thing.

So it stays on the shelf until we have a definition we would defend. We would genuinely like to hear how other people think about this, because we suspect the answer is domain-specific in a way that makes a single rule impossible.

What this means for you

If you are already using Open Free Max, updating to 4.0 changes two things by itself. The 4.0 release news is the short product tour; this piece is the reasoning behind the choices.

The router is on. A task launched through it gets the level of thinking it judges appropriate, rather than the model you had set by hand in that CLI's settings. If you prefer your own setting, turn the router off in Settings and that choice sticks.

Your local state now lives in a folder of its own, per account. Two accounts on the same machine used to share projects, memory and sessions, which made it impossible to keep work and personal use apart. They are now separate. The move happens once, at the first launch after you sign in, and switching account asks for a restart.

Everything else is there when you want it and out of the way when you do not: the gauges showing what is left on each subscription, the fallbacks you can revoke per project, the checklist for asking several agents at once, the Forget button.

The point of all of it is a single sentence we have not changed since the first day of this work. You describe the work. The right agent starts.

You should not have to think about models to get good work out of them. That was always somebody else's job, and we would rather it was ours than yours.