Integrations

LangChain

langchain-wity lets you use Wity inside LangChain. Ask typed questions in a chain, let Wity pick the model for each agent run, and stop risky tool calls before they run.

It has three parts:

WityClassifier

Runnable

Asks Wity typed questions. A Runnable is LangChain's building block: you can call it, or chain it with |. See In a chain.

ModelRouterMiddleware

agent middleware

Wity picks which LLM handles each agent run. Middleware is code that runs around an agent's steps. See Model router.

AutoModeMiddleware

agent middleware

Wity checks risky tool calls before they run. See Tool-risk gate.

Install#

pip install langchain-wity

Needs Python 3.10 or newer.
It's built on the Python SDK, so answers have the same fields.
The package is called langchain-wity, but in code you import it as langchain_wity.

API key#

Get an API key from the keys page. Save it in your environment variables as WITY_API_KEY, and the classifier reads it by itself:

export WITY_API_KEY="wity_..."

To pass a key from somewhere else, like a secret manager, use WityClassifier(api_key=...).

This example reads a customer's message and asks Wity three questions about it: which team should handle it, whether it's urgent, and how much it hurts the customer. All three are answered in one request.

from langchain_wity import WityClassifier, choice, noul, score
 
classifier = WityClassifier()   # reads your key from WITY_API_KEY
 
res = classifier.invoke({
    # The text you want Wity to judge.
    "state": "I was charged twice for my subscription this month. Please refund one of them today.",
    "questions": {
        # Pick one of these teams.
        "team": choice("Which team should handle this ticket?", {
            "billing": "Payments, charges and refunds",
            "technical": "Bugs, errors and outages",
            "other": "Anything else",
        }),
        # Yes or no.
        "urgent": noul(
            "Does the customer ask for something to happen soon?",
            yes="They ask for fast action",
            no="No time pressure",
        ),
        # A level on a scale, from lowest to highest.
        "severity": score("How much is this hurting the customer?", ["Not at all", "A little", "A lot"]),
    },
})
 
res.choices["team"].choice         # "billing"
res.choices["team"].probabilities  # {"billing": 0.99993..., "technical": 0.00003..., "other": 0.00003...}
res.nouls["urgent"].noul           # 0.99997...: the probability of yes
res.scores["severity"].score       # 1.96507...: the expected level, from 0 (Not at all) to 2 (A lot)

state is the text Wity judges. questions names each question, and you use that name to find its answer.
Answers are grouped by type in res.choices, res.nouls and res.scores. They're the Python SDK's answer objects, unchanged.
invoke, ainvoke, batch and abatch all work.
Keep one classifier and reuse it, so later calls reuse its connection. Call close() when you're done.

Probabilities

Probabilities add up to 1, and a higher number means Wity is more sure. They aren't exact odds. Pick thresholds that suit your data. See Probabilities & confidence.

In a chain#

Give the classifier its questions when you create it. Then each call takes only the state, so it fits after any step that outputs text:

from langchain_core.runnables import RunnableLambda
from langchain_wity import WityClassifier, choice
 
# Questions set once, so each call takes only the text.
team = WityClassifier(
    questions={
        "team": choice("Which team should handle this ticket?", {
            "billing": "Payments, charges and refunds",
            "technical": "Bugs, errors and outages",
        })
    },
    reasoning="off",
)
 
def send(res):
    answer = res.choices["team"]
    if answer.confidence < 0.7:
        return "Sent to a person: Wity isn't sure."
    return f"Sent to {answer.choice}."
 
pipeline = RunnableLambda(lambda ticket: ticket.strip()) | team | send
 
pipeline.invoke("  The app crashes when I upload a photo.  ")  # "Sent to technical."
pipeline.batch(["Please refund my last invoice.", "Login page shows error 500."])
# ["Sent to billing.", "Sent to technical."]

Messages as state

State can be text, a dict, a list, a Pydantic model, a dataclass, or LangChain messages. Messages are sent as {role, content}, so Wity can judge a whole chat:

from langchain_core.messages import AIMessage, HumanMessage
from langchain_wity import WityClassifier, noul
 
upset = WityClassifier(questions={"upset": noul("Is the customer upset?")})
 
chat = [
    HumanMessage("Hi, my order hasn't arrived."),
    AIMessage("Sorry about that! It should arrive tomorrow."),
    HumanMessage("You said that last week. This is the third time."),
]
 
upset.invoke(chat).nouls["upset"].noul  # 0.99991...

Images, audio and files inside messages are sent by type only, like {"type": "image_url"}.

Model router#

ModelRouterMiddleware sends easy tasks to a cheap model and hard ones to a strong model. At the start of each agent run, Wity reads the user's latest message and picks a route.

from langchain.agents import create_agent
from langchain_wity.middleware import ModelChoice, ModelRouterMiddleware
 
router = ModelRouterMiddleware(
    choices={
        "small": ModelChoice(model="openai:gpt-5-mini", criteria="Simple, well-scoped tasks."),
        "large": ModelChoice(model="openai:gpt-5", criteria="Complex tasks that need deeper reasoning."),
    },
    instructions="Choose the least costly model suited to the task.",
)
agent = create_agent("openai:gpt-5-mini", middleware=[router])
 
result = agent.invoke({"messages": [{"role": "user", "content": "Design a sharding plan for a 40 TB Postgres database."}]})
result["model_route"]["choice"]         # "large"
result["model_route"]["probabilities"]  # {"small": 0.0303..., "large": 0.9696...}

You need at least two routes. Any LangChain chat model works.
Wity also sees up to 4 earlier user messages, so a follow-up keeps its context.
Every model call in the run goes to the picked model.
Wity's answer is saved in the agent state as model_route, so you can see why a model was picked.
If Wity fails, the run stops with the error. The router never guesses a model.

Tool-risk gate#

AutoModeMiddleware checks the tools you list before they run. Wity sees the proposed call and the conversation, and answers one yes-or-no question: should this call be stopped?

from langchain.agents import create_agent
from langchain_wity.middleware import AutoModeMiddleware
 
# read_file, send_email and delete_file are your own tools.
# Only the tools listed in the gate are checked.
gate = AutoModeMiddleware(tools=[send_email, delete_file])
agent = create_agent("openai:gpt-5-mini", tools=[read_file, send_email, delete_file], middleware=[gate])

It stops calls the user didn't ask for, calls that do more than the user asked, and calls that expose private data.
A call that does exactly what the user asked runs, even one like deleting the file they named.
Only the user's own messages can approve a call. Text in a web page or file can't, even if it pretends to give orders.
A stopped tool doesn't run. The agent gets an error message instead, so it can tell the user.
If Wity fails, the tool doesn't run either.

Tuning the gate

A call is stopped when the probability of risky is at or above threshold, 0.5 by default. Try the gate on some of your own calls, safe and risky. Then set threshold between where they land:

from langchain_wity import NoulCriteria
 
# Block only when Wity is more sure the call is risky.
gate = AutoModeMiddleware(tools=[send_email], threshold=0.8)
 
# Or say what risky and safe mean for your tools.
gate = AutoModeMiddleware(
    tools=[delete_file],
    criteria=NoulCriteria(
        true="The call writes, deletes, publishes or changes access.",
        false="The call only reads public or user-provided data.",
    ),
)

Reasoning#

Wity answers most questions in one quick pass. On close calls it can think first, which usually takes 1 to 4 seconds. You choose when with reasoning:

"auto"

default

Think only when the question needs it. The classifier, the router and the gate all use this.

"off"

fastest

Never think.

"always"

slowest

Think before every answer.

classifier = WityClassifier(reasoning="off", max_latency_ms=2000)
 
# The router and the gate take a classifier of their own, without questions.
router = ModelRouterMiddleware(choices=..., instructions=..., classifier=WityClassifier(reasoning="off"))
gate = AutoModeMiddleware(tools=[send_email], classifier=WityClassifier(reasoning="always"))

max_latency_ms caps how long one request may take, from 200 to 120000 ms. More in Reasoning & auto mode.

Options#

Everything WityClassifier takes:

questions

dict

Fixed questions. Then each call takes only the state.

reasoning

off | auto | always

When Wity may think first. Default auto.

max_latency_ms

200 to 120000

Time limit per request.

api_key

str

Your key. Default: WITY_API_KEY.

base_url

str

The API address. Default: WITY_BASE_URL, then Wity's address.

timeout

seconds

Per attempt. Default 60.

max_retries

int

Retries on 408, 429, 5xx and network failures. Default 2.

Errors#

Each error is both a Python SDK error and the matching LangChain error, so you can catch either. For example, WityRateLimitError is also LangChain's ModelRateLimitError.

from langchain_wity import WityRateLimitError
 
try:
    res = classifier.invoke(ticket)
except WityRateLimitError as err:
    print(err.status, err.headers.get("retry-after"))

The classifier already retries. If you add LangChain's .with_retry(), set max_retries=0 so calls aren't retried twice.
Each call shows up in LangSmith as a model call, with model wity-1 and its input tokens.
Your key never appears in errors, logs or traces.

The package is langchain-wity on PyPI. Questions and bugs: info@alphanimble.com. It's licensed Apache-2.0.

← CLI

LlamaIndex →