A model call ends in a named outcome and never raises¶
Decided in 0.1.0 (Generator, generate), extended in 0.8.0 (Embedder) and 0.11.0
(Reranker). Recorded in 0.12.0.
Context and problem statement¶
A call to a language model fails in many ways that are not bugs: the server is down, the gateway
refuses, the request is too large, the reply hits the token cap, the reply does not parse. A
caller that receives these as exceptions has to know every SDK's exception tree (pydantic-ai wraps
openai wraps httpx wraps a transport), and a forgotten except turns a model outage into a crashed
request. A caller that receives None cannot tell "the model is down" from "the model said
nothing".
Decision drivers¶
- A caller's fallback policy should be a
matchon a small, closed set of names. - The reason a call failed must be recordable without leaking a provider's response body.
- A failure must stay distinguishable from an empty answer.
Considered options¶
- Raise library exceptions, one class per failure kind.
- Return
Noneon failure. - Return a result whose
outcomeis one of a closed set of names.
Decision outcome¶
Chosen option 3. Generator.generate, Embedder.embed and Reranker.rerank return a result with
outcome in ok | gated | unavailable | request_fault | truncated | invalid and a one-line
reason. The adapter classifies the exception chain, root cause first:
- a gate refusal anywhere in the chain wins (
gated); - a
lengthfinish is a budget failure (truncated) even when it surfaces as a parse error; - then status codes (
request_faultfor a refused request such as 400/413/422,unavailableotherwise); - then the parse failure itself (
invalid).
A partial result is invalid, never a short list: a batch with one vector missing would shift
every later vector onto the wrong row. failure.failure_reason keeps the exception class, status
code and the first line of the message, cut before any body: payload, within 200 characters.
Judge is the exception. It raises when the judging model cannot answer, and the caller records
the sample as could_not_run. An unjudged sample must never read as a pass, and a Judge result
with a failed outcome would invite exactly that default.
Consequences¶
- Good: fallback policy is a
matchonres.outcome; a dead model is a recorded row, not a stack trace. - Good: one taxonomy across three ports; the embedder reuses the generator's classifier.
- Bad: a caller whose contract is that failure propagates must opt in (
GenerationFailed, raised by the facade for that caller). - Neutral: adapters retry nothing by default (
max_retries=0), because a gateway in front usually retries and two stacked policies multiply.
More information¶
- A provider error's full text once reached a user-facing feed through a persisted reason field:
a 300-character
ModelHTTPError: status_code: 500, model_name: ..., body: {...}, one row per scheduler tick. A proxy error can also embed a keyed URL. Hence the body-safe cut. - With a judge asked for its verdict as a tool call (pydantic-evals' default), a self-hosted model
answered in a mix of its XML tool-call syntax and JSON: 400 of 854 judged rows carried a reason
ending
", "pass": false, "score": 0.0or a<parameter=pass>tag. The judge adapter now asks for the native structured-output grammar, and a reply it cannot read raises rather than being guessed.