A result names what was measured, what it cost and who answered¶
Decided in the release after 1.22.0.
Context and problem statement¶
Through 1.22.0 a Generated reported usage as {"input", "output"} from the last reply on a
failed run and the run's total on a successful one, model as the id the request named, and
finish_reason as a provider string. A consumer that needed more rebuilt it from the wire: its
own capture for cached and reasoning tokens, its own price lookup, its own reading of which
deployment a gateway alias resolved to. Each rebuild coerced an unknown to zero somewhere
(tokens or 0, cost or 0.0), so an unpriced call looked free and a USD cap never fired on it.
The question: what does a result say about the call it describes, so that no consumer re-derives it, and so that an unknown stays visible as unknown?
Decision drivers¶
- Unknown is None at every level. A side the provider did not report is never 0.
- A call is every request it made: output re-asks, tool steps, wire retries that returned a body.
- A gateway in front knows the deployment and its negotiated price; the body only knows the alias.
- 1.x is additive: no public name is removed, and the 1.x fields keep their 1.22.0 values.
Decision¶
Usage(ports) has inclusive buckets:inputincludes cached tokens,outputincludes reasoning tokens;cache_read,cache_writeandreasoningare the parts.requestscounts wire requests andunmeasuredthose that reported no usage.None + n = n, andcompleteis false while any request was unmeasured. An adapter for a provider that reports exclusive buckets normalises them; the OpenAI wire is already inclusive.Generated.usage_v2is the sum over every request of the call. The capture keeps every 2xx reply (WireReply), not the last; the framework's ownModelResponseusage is the fallback when the call did not rideopenai_compatible.Cost(usd, source, priced_model, reason)with precedence: a gateway's cost header, then aPricer, then unknown.usdis None when unknown, with a reason (no_usage,unknown_model,pricer_error).Costitself is None only when no pricer and no gateway cost exist. A self-hosted model costs what the caller says (overrides), never free by default.Answered(requested, model, deployment, response_id, ...)says who answered:modelis what the response body reported,deploymentwhat the gateway's header reported.Finish(unified, raw): a closed vocabulary withotherfor an unknown string;rawis never rewritten, and absent is None, neverstop.- A setting the model class declares it ignores (
Model.unsupported_settings, read before the call) is aCallWarning(kind="unsupported")on the result. - Two seams, both constructor arguments:
Pricer(adapters.genai_prices.GenaiPricesPriceron genai-prices' dated table) andGatewayReader(adapters.litellm_proxy, no vendor import). - The 1.x fields stay:
usageis derived fromusage_v2on a returned run and keeps the last reply's on a failed one (as 1.22.0 did);finish_reasonisfinish.raw;modelstays the requested id. A golden fixture written against 1.22.0 pins all of them.
Considered options¶
- Report
Generated.modelas the model that answered. Rejected for 1.x: it is a third behaviour change to a field consumers key ledgers on.answered.modelcarries it;modelis deprecated in its docstring. - A price table inside the library. Prices change weekly; genai-prices ships a dated, Decimal table and its own refresh.
- Capture setting warnings with
warnings.catch_warnings. The filter is process-global, so concurrent calls would see each other's warnings. pydantic-ai declares the ignored settings on the model class; reading that before the call is exact and needs no global state. - Sum usage in the 1.x
usagefield on failure too. More correct, but a change to a 1.x value the golden fixture forbids;usage_v2is the correct sum.
Consequences¶
- A consumer deletes its own capture, price lookup and zero-coercions and reads the result.
- An unpriced model is visible:
llm_cost_unknownonce per model per process,cost.usd is None. - A call with no usage block is
unmeasured, reported per call asllm_usage_unmeasured. - On the framework path (a fixed
model=), pydantic-ai's sub-buckets are plain counts: a provider that reports no cached tokens reads 0 there; the wire path keeps None. - A unit test greps
src/foror 0in code and fails unless the line carries# zero-ok: <why>.