Integrations

DSPy

dspy-wity lets Wity answer DSPy signatures. Write a normal signature with yes-or-no, choice or score outputs, and Wity answers it with a probability for every option.

A signature is DSPy's description of a task: its inputs and its outputs. dspy-wity is one class, Wity, that plugs in where DSPy expects a language model. Use it to:

Answer decision outputs with Wity. See Your first request.
Tune when an answer counts, by hand or with DSPy's ReAnchor optimizer. See Tuning.
Judge another model's answers. See Wity as a metric.

Install#

pip install dspy-wity

Needs Python 3.10 or newer and DSPy 3.4 or newer. That's the version with the decision types in dspy.experimental.
It's built on the Python SDK.
The package is called dspy-wity, but in code you import it as dspy_wity.

API key#

Get an API key from the keys page. Save it in your environment variables as WITY_API_KEY, and Wity reads it by itself:

export WITY_API_KEY="wity_..."

To pass a key from somewhere else, like a secret manager, use Wity(api_key=...).

This example reads a customer's message and asks Wity three questions about it: which team should handle it, whether it's urgent, and how much it hurts the customer. All three are answered in one request.

import dspy
from dspy.experimental import Choice, Noul, Score
from dspy_wity import Wity
 
# The options for each answer.
Team = Choice[
    ("billing", "Payments, charges and refunds"),
    ("technical", "Bugs, errors and outages"),
    ("other", "Anything else"),
]
Urgent = Noul[(True, "They ask for fast action"), (False, "No time pressure")]
Severity = Score["Not at all", "A little", "A lot"]
 
 
class Triage(dspy.Signature):
    """Route a customer support ticket."""
 
    ticket: str = dspy.InputField(desc="The customer's message.")
    # Each output's desc is the question Wity answers.
    team: Team = dspy.OutputField(desc="Which team should handle this ticket?")
    urgent: Urgent = dspy.OutputField(desc="Does the customer ask for something to happen soon?")
    severity: Severity = dspy.OutputField(desc="How much is this hurting the customer?")
 
 
dspy.configure(lm=Wity())   # reads your key from WITY_API_KEY
triage = dspy.Predict(Triage)
 
res = triage(ticket="I was charged twice for my subscription this month. Please refund one of them today.")
 
res.team.value             # "billing"
res.team.probabilities     # {"billing": 0.99954..., "technical": 0.00019..., "other": 0.00026...}
res.urgent.value           # True
res.urgent.probability     # 0.99995...: the probability of yes
res.severity.value         # 1.97326...: the expected level, from 0 (Not at all) to 2 (A lot)
res.severity.level         # 2

Choice, Noul and Score are DSPy's decision types. They set the options for each output.
Each output's desc becomes Wity's question.
The signature's docstring and inputs become what Wity reads.
To use Wity for one predictor only, write triage.set_lm(Wity()). Other predictors can keep a chat model for text.

Probabilities

Probabilities add up to 1, and a higher number means Wity is more sure. They aren't exact odds. Pick thresholds that suit your data. See Probabilities & confidence.

Plain Python values

To get a plain bool or string back, annotate it with the decision type:

from typing import Annotated, Literal
 
class PlainTriage(dspy.Signature):
    """Route a customer support ticket."""
 
    ticket: str = dspy.InputField(desc="The customer's message.")
    urgent: Annotated[bool, Urgent] = dspy.OutputField(desc="Does the customer ask for something to happen soon?")
    team: Annotated[Literal["billing", "technical", "other"], Team] = dspy.OutputField(desc="Which team?")
 
 
plain = dspy.Predict(PlainTriage)(ticket="I was charged twice. Please refund one today.")
plain.urgent   # True
plain.team     # "billing"

Which DSPy modules fit#

Wity answers questions with fixed options. It doesn't write text, and it gives one answer per question.

dspy.Predict

use this

The module to use. dspy.Parallel, dspy.Evaluate, dspy.asyncify and your own dspy.Module programs work too.

dspy.ChainOfThought

use Predict

It adds a text output Wity can't write. To have Wity think first, use Predict with Wity(reasoning="always").

dspy.BestOfN, dspy.Refine

not supported

They pick from several sampled answers, and Wity doesn't sample. Read the probabilities instead.

dspy.streamify

not supported

Not for decision outputs.

BootstrapFewShot

configure

It and LabeledFewShot copy your program, which drops set_lm. Use dspy.configure(lm=Wity()) for the whole run instead.

Tuning when an answer counts#

Wity gives the probabilities, and DSPy turns them into a value. Each output has a setting for that:

Noul: a threshold, 0.5 by default.
Score: cuts, where one level ends and the next begins.
Choice: weights, to prefer some options in close calls.

You can set them by hand:

triage.fields["urgent"] = {"threshold": 0.9}            # only say yes when Wity is very sure
triage.fields["severity"] = {"cuts": [0.8, 1.5]}        # where one level ends and the next begins
triage.fields["team"] = {"weights": {"billing": 2.0}}   # prefer billing in close calls

Or let ReAnchor fit them to labelled examples. It makes one Wity call per example, then tries settings on the saved answers:

from dspy.experimental import ReAnchor
 
trainset = [
    dspy.Example(ticket="I want my money back for this month.", urgent=True).with_inputs("ticket"),
    dspy.Example(ticket="How do I download my invoice?", urgent=False).with_inputs("ticket"),
    # ... more labelled tickets
]
 
def metric(example, pred, trace=None):
    return float(pred.urgent.value == example.urgent)
 
optimizer = ReAnchor(metric)
tuned = optimizer.compile(triage, trainset=trainset)
tuned.fields        # the fitted settings, or {} when the defaults already score best
optimizer.report    # the metric before and after

ReAnchor needs DSPy's cache, which Wity uses by default.

Wity as a metric#

DSPy's optimizers and dspy.Evaluate take a metric: a function that scores each answer. Wity can be that judge. Ask it a yes-or-no question about the answer, and use its probability as the score:

Helpful = Noul[(True, "The reply solves the customer's problem"), (False, "It doesn't, or it dodges")]
 
 
class JudgeReply(dspy.Signature):
    """Judge a support agent's reply to a customer."""
 
    question: str = dspy.InputField(desc="What the customer asked.")
    reply: str = dspy.InputField(desc="The agent's reply.")
    helpful: Helpful = dspy.OutputField(desc="Does the reply solve the customer's problem?")
 
 
judge = dspy.Predict(JudgeReply)
judge.set_lm(Wity(reasoning="off"))   # only the judge uses Wity
 
 
def reply_is_helpful(example, pred, trace=None):
    verdict = judge(question=example.question, reply=pred.reply).helpful
    # While an optimizer collects examples (trace is set), return a strict pass or fail.
    return verdict.probability if trace is None else float(verdict.value)

Your program can run on any chat model while Wity judges it. Pass the metric on as usual, for example to dspy.Evaluate or dspy.MIPROv2.

Reasoning#

Wity answers most questions in one quick pass. On close calls it can think first, which usually takes 1 to 4 seconds. You choose when with reasoning:

"auto"

default

Think only when the question needs it.

"off"

fastest

Never think.

"always"

slowest

Think before every answer.

Wity(reasoning="off")
Wity(reasoning="always", max_latency_ms=5000)

max_latency_ms caps how long one request may take, from 200 to 120000 ms. More in Reasoning & auto mode.

Options#

Everything Wity takes:

reasoning

off | auto | always

When Wity may think first. Default auto.

max_latency_ms

200 to 120000

Time limit per request.

api_key

str

Your key. Default: WITY_API_KEY.

base_url

str

The API address. Default: WITY_BASE_URL, then Wity's address.

timeout

seconds

Per attempt. Default 60.

max_retries

int

Retries on 408, 429, 5xx and network failures. Default 2.

cache

bool

Use DSPy's cache. Default True. A repeated call comes from the cache, with no request and no cost.

Saving a program#

tuned.save("triage.json")   # the fitted settings are saved; your key never is
 
loaded = dspy.Predict(Triage)
loaded.load("triage.json", allow_unsafe_lm_state=True)   # only for files you trust

DSPy asks for allow_unsafe_lm_state=True before it loads a language model class from outside DSPy.
Or save with tuned.set_lm(None) first, then call loaded.set_lm(Wity()) after loading.

Errors#

Errors are the Python SDK errors, unchanged, like wity.RateLimitError.
Wity already retries failed calls. DSPy doesn't retry them again.
With dspy.configure(track_usage=True), res.get_lm_usage() counts Wity's input tokens under wity-1.
Your key and your text never appear in errors or logs.

The package is dspy-wity on PyPI. Questions and bugs: info@alphanimble.com. It's licensed Apache-2.0.

← Vercel AI SDK

Writing good questions →