The Two-Speed Stack Is Missing a Gear

In one month, decision models went from a single launch to a whole category. More than half a dozen companies have shipped a model that answers typed questions with probabilities instead of prose. Most of the coverage describes them the same way: as one half of a two-speed stack.

The fast tier is a decision model. It answers yes/no, pick-one and score questions in milliseconds for fractions of a cent. The slow tier is a frontier language model that does the writing, coding and reasoning. Confident decisions go through automatically. Uncertain ones go up to the slow tier, or to a person.

We agree with most of this. Splitting routine calls from real work is the right idea. The part we'd change is the step in the middle.

Escalation is a second system#

In a two-speed stack, "uncertain" is a hand-off. The fast model says 0.52 billing, 0.45 account access, so your code sends the whole ticket to a different model and asks again. That one call adds several costs:

• Another round trip. A frontier model takes seconds to answer, not a tenth of a second. One independent test put a frontier model at about 2.5 seconds per task, against a few hundred milliseconds for decision models.
• Another bill. The same test measured the frontier model at 85 to 170 times the cost per task of the decision models it ran. You pay that rate on the calls that escalate, which are often the ones that matter most.
• Another reading of the input. The second model reads the ticket from scratch. It may weigh it differently, and now you have two answers that don't agree.
• Another format. The frontier model answers in prose, or in JSON with no probabilities attached. You're back to parsing text and guessing how sure it was.
• Another thing to maintain. Two prompts, two vendors, two sets of evaluations, and a threshold between them that someone has to tune.

None of this is a reason to drop the pattern. It does mean the hand-off point is where a two-speed stack spends most of its time and money.

Most close calls don't need a bigger model#

Look at the calls that actually escalate and most of them aren't hard in the way frontier models are good at. They're near-ties.

• A message that fits two queues equally well.
• A delivery date that needs counting, not reading.
• A policy question that turns on one clause on page four.
• A choice where the answer changes when the options are listed in a different order.

These questions don't need a model that can write an essay. They need a second look: a few hundred tokens of working it through, then the same kind of typed answer as before.

System 1.5: think in the same call#

That's the gear we built Wity to fill. In auto mode, Wity answers directly and checks whether the answer holds up: is there a clear winner, and does it survive the options being reordered? Most questions pass and come back in about a tenth of a second.

When an answer doesn't hold up, Wity thinks first. It does this in the same request, with the same model and the same input. It stops as soon as the answer has settled and returns the same typed decision with probabilities. The answer also says that it thought and why: close_call, order_sensitive or forecast.

answer = decide(ticket, questions, reasoning="auto")["team"]
 
if answer["probabilities"][answer["choice"]] >= 0.85:
    assign(ticket, answer["choice"])     # clear, or clear after thinking
else:
    send_to_triage(ticket)               # still close: a person decides

There's no second model in that code, and no second format. The threshold now marks where you hand off to a person, not to another model.

The cost works out like this:

• Easy calls: answered directly in about 100 ms.
• Close calls: usually answered with thinking in one to four seconds, from the same endpoint.
• Price: thinking isn't billed, so a thought answer costs the same as a direct one, $0.042 per million input tokens.

Where the fast tier ends#

This doesn't replace your frontier model. Some work really belongs in the slow tier: writing the full reply to an upset customer, planning a ten-step task, or reasoning over a problem nobody has written options for. Use a large model there.

It also doesn't replace people. Some decisions should go to a human however confident any model is, and probabilities are what let you draw that line clearly.

What changes is what has to leave the fast tier. In a two-speed stack, every close call leaves. With System 1.5, only work that isn't a decision leaves, plus the decisions you've chosen to keep for people.

The numbers so far#

We ran Wity and eight fast, low-cost models through the same 200 real-world cases. Wity's typical call took 109 ms, against 943 ms for the next fastest. 1,000 runs cost $3.29, against $17.95 for the next cheapest. The full results and how they were measured are on the benchmarks.

Try it#

Set "reasoning": "auto" on your next request and log answer.reasoning next to each decision. Questions that often think for close_call usually have two options that overlap. Sharper option descriptions make those questions faster and more accurate. The reasoning docs cover the modes and how to plan around latency.