Reasoning & auto mode

Fast by default, careful when needed#

Most questions in a real workload are easy. The language of a ticket, whether a page has loaded, which of five clearly different queues a message belongs to: a direct read gets them right, and thinking would only add seconds. A few are genuinely close: a message that fits two queues, a policy question that turns on one detail. Those are the ones where thinking pays off.

Most systems make you choose one speed for everything. Wity decides per question. It answers directly when the answer is clear, and spends time only on the questions that need it, even within one request.

The three modes#

Set reasoning at the top level of the request. It applies to every question in that request.

{ "state": "…", "questions": { … }, "reasoning": "auto" }

If one workflow needs different modes (a fast check in the UI and a careful review behind it), send them as two requests with different reasoning values.

How auto mode decides#

Wity first answers directly and checks whether that answer is robust: a clear winner that does not change when the options are presented differently. Most real questions pass, and you get the direct answer in about a tenth of a second. Otherwise it thinks, for one of three reasons:

• close_call: the top options are close. A ticket that sits at 0.51 billing and 0.46 account access deserves a closer look.
• order_sensitive: the answer changes depending on the order the options are presented in. A sound answer shouldn't depend on that, so an answer that does is a sign the question needs more thought, even if one version looked confident.
• forecast: the question asks how likely an outcome is, not which answer is correct. Those are worked out from the evidence. See Forecast questions.

When it thinks, Wity reasons about the state step by step and then answers. It watches its own answer while it thinks and stops as soon as the answer has settled, instead of running a fixed long chain of thought. Most thoughts finish in a few hundred tokens.

Example#

A support inbox asks two things about every ticket: what language it is in (to pick an agent), and which team should own it. This ticket is about a charge after cancelling, which is billing, but the customer also can't log in, which is account access. The account record in the state confirms the plan is cancelled and renews on the 5th.

{
  "state": {
    "ticket": "I cancelled my plan on the 3rd but was still charged on the 5th. I also can't log in any more to check my invoices.",
    "account": { "plan": "Pro monthly", "status": "cancelled", "renewal_day": 5 }
  },
  "questions": {
    "language": {
      "type": "choice",
      "instructions": "Which language is the ticket written in?",
      "criteria": { "en": "English", "de": "German", "es": "Spanish", "other": "Any other language" }
    },
    "team": {
      "type": "choice",
      "instructions": "Which team should own this ticket?",
      "criteria": {
        "billing": "Charges, refunds, invoices, cancellations",
        "account_access": "Log-in problems, locked accounts, passwords",
        "other": "Anything else"
      }
    }
  },
  "reasoning": "auto"
}

In auto mode, the two questions take different paths:

{
  "answers": {
    "language": {
      "type": "choice", "choice": "en", "confidence": 0.99,
      "probabilities": { "en": 0.998, "de": 0.001, "es": 0.0005, "other": 0.0005 },
      "reasoning": { "mode": "auto", "thought": false, "reason": null, "forecast": false, "thought_tokens": null }
    },
    "team": {
      "type": "choice", "choice": "billing", "confidence": 0.41,
      "probabilities": { "billing": 0.74, "account_access": 0.24, "other": 0.02 },
      "direct_probabilities": { "billing": 0.51, "account_access": 0.46, "other": 0.03 },
      "reasoning": { "mode": "auto", "thought": true, "reason": "close_call", "forecast": false, "thought_tokens": 184 }
    }
  },
  "metadata": { "reasoning": "auto", "elapsed_ms": 1840.6 }
}

• language is obvious, so Wity answered directly. thought is false and there is no direct_probabilities, because the answer is the direct one.
• team started as a near tie (0.51 against 0.46), so Wity thought, for close_call. Reasoning through the ticket, the customer's actual complaint is the charge after cancelling, and the log-in problem is secondary. After thinking, billing leads 0.74 to 0.24. direct_probabilities keeps the before-picture.
• metadata.elapsed_ms is for the whole request. It is dominated by the one question that thought. A request where every answer is direct comes back in about a tenth of a second.

Even after thinking, 0.24 on account access is real. A good handler for this ticket fixes the charge and also points the customer to a password reset. The probabilities tell your code both things.

What you see in the response#

In auto and always modes every answer carries a small reasoning record:

When thought is true, the answer also has direct_probabilities (or direct_noul). Act on the main probabilities; the direct ones are for understanding what changed. See Probabilities & confidence.

Using the record

Log the reasoning record next to each decision. Over a day of traffic it shows which questions think, and why:

from collections import Counter
 
reasons = Counter()
for resp in recent_responses:                      # e.g. the last day of logged responses
    for name, ans in resp["answers"].items():
        r = ans.get("reasoning") or {}
        reasons[(name, r.get("reason") or "direct")] += 1
 
for (name, reason), n in sorted(reasons.items()):
    print(f"{name:12s} {reason:16s} {n}")

A question that thinks for close_call on a large share of inputs is telling you something about its options. Two of them probably overlap, and sharper descriptions or a split into two questions will make it both faster and more accurate. A question that often thinks for order_sensitive usually has options that are not clearly distinct.

Latency and cost#

• Direct answers: typically around 0.1 s of model time, plus your network round trip.
• Thought answers: typically 1 to 4 s, depending on how long the question takes to settle.
• A request returns when all of its answers are ready, so one thought answer sets the pace for the whole request.
• Thinking is not billed separately: you pay for your input tokens either way.

Latency budget#

Most questions settle quickly, but a hard one can think for many seconds. When a step has a deadline, pass max_latency_ms (200 to 120000) with the request. Wity then thinks only as long as the budget allows and answers from the reasoning it has so far, the same way it stops early once an answer settles. If there is not enough time to think at all, it answers directly and softens the close call rather than overstating it.

{
  "state": "...",
  "questions": { "...": { "...": "..." } },
  "reasoning": "auto",
  "max_latency_ms": 3000
}

Answers cut short by the budget carry reasoning.budget_limited: true. The budget covers Wity's side of the request; add your network round trip on top. It is a target, not a hard cutoff: the last step can run a little over.