llm_ports.ports¶
llm_ports.ports
¶
The ports: Protocols and the frozen value types that cross them. Zero I/O.
outcomes holds the named-outcome vocabulary: reasons, GenerationFailed, and the HTTP-status
mapping every adapter shares.
ChatRequest
dataclass
¶
ChatRequest(
model: str,
messages: tuple[Message, ...],
system: str = "",
tools: tuple[Tool, ...] = (),
tool_choice: str = "auto",
max_steps: int = 8,
parallel_tools: bool = True,
settings: Mapping[str, Any] = dict(),
deadline: Deadline | None = None,
timeout_s: float = 120.0,
concurrency_group: str | None = None,
cache_system: bool = False,
cache_tools: bool = False,
label: str = "chat",
self_hosted: bool = True,
output_retries: int = 1,
policy: WirePolicy | None = None,
tool_output_chars: int | None = None,
streamed: bool = True,
)
One chat call, already resolved from a role.
Attributes:
-
model(str) –The model id.
-
messages(tuple[Message, ...]) –The history, oldest first; the last is normally the user's turn.
-
system(str) –The system prompt; empty for none. Never trimmed.
-
tools(tuple[Tool, ...]) –The tools the model may call.
-
tool_choice(str) –auto,none,required, or one tool's name. -
max_steps(int) –The most model requests in the call, tool rounds and re-asks included. Reaching it with a tool call still pending is
truncated(provider_code="max_steps"). -
parallel_tools(bool) –Let the model ask for several tool calls in one step.
-
settings(Mapping[str, Any]) –Sent to the model verbatim (max_tokens, temperature, ...).
-
deadline(Deadline | None) –The call's clocks; None means
Deadline(total_s=timeout_s). -
timeout_s(float) –The call's bound when
deadlineis None, in seconds. -
concurrency_group(str | None) –The
Limitergroup; None means the model id. -
cache_system(bool) –Ask for a cache breakpoint after the system prompt (ADR-0024).
-
cache_tools(bool) –Ask for a cache breakpoint after the tool definitions.
-
label(str) –The call's name in an adapter's logs.
-
self_hosted(bool) –True for a self-hosted tier.
-
output_retries(int) –Re-asks for an output or a tool call the model got wrong.
-
policy(WirePolicy | None) –How the adapter retries a wire request; None is the adapter's default.
-
tool_output_chars(int | None) –The request's tool-output budget, for a tool that sets none; None for no cut.
-
streamed(bool) –Stream each model reply as it is generated. False (a model that cannot stream) sends each step's whole text as one
TextDelta.
Raises:
-
ValueError–max_stepsis below 1.
Finished
dataclass
¶
Finished(result: Generated[T])
The stream's last event, exactly once, in every case: success, failure, deadline.
Message
dataclass
¶
Message(
role: ChatRole,
parts: tuple[Part, ...],
cache: bool = False,
)
One turn of a conversation.
Attributes:
-
role(ChatRole) –user,assistant(text, reasoning and tool calls) ortool(tool results). -
parts(tuple[Part, ...]) –The turn's parts, in order.
-
cache(bool) –Ask for a prompt-cache breakpoint after this message (ADR-0024). A request, not a guarantee:
Usage.cache_readon the result says whether it was served.
text
property
¶
text: str
The message's text parts, joined; reasoning and tool parts are not included.
Part
dataclass
¶
Part(
kind: PartKind,
text: str | None = None,
data: bytes | None = None,
media_type: str | None = None,
call_id: str | None = None,
name: str | None = None,
args: Mapping[str, Any] | None = None,
untrusted: bool = False,
)
One piece of a message.
Attributes:
-
kind(PartKind) –What the part holds.
-
text(str | None) –The text of a
text,reasoningortool_resultpart. -
data(bytes | None) –An image's bytes.
-
media_type(str | None) –An image's MIME type.
-
call_id(str | None) –The id that pairs a
tool_callwith itstool_result. -
name(str | None) –The tool's name, on a
tool_callor atool_result. -
args(Mapping[str, Any] | None) –A
tool_call's arguments. -
untrusted(bool) –True for a
tool_resultwhose content came from outside (the default for a tool's output); the adapter marks it as data when it sends it.
ReasoningDelta
dataclass
¶
ReasoningDelta(text: str)
A piece of the model's reasoning; never part of the value, never sent back to a provider.
StepFinished
dataclass
¶
Tool
dataclass
¶
Tool(
name: str,
description: str,
parameters: Mapping[str, Any],
handler: ToolHandler,
timeout_s: float = 30.0,
untrusted_output: bool = True,
max_output_chars: int | None = None,
)
A client-side tool the model may call.
Attributes:
-
name(str) –The name the model calls it by.
-
description(str) –When and why to call it, for the model.
-
parameters(Mapping[str, Any]) –The arguments' JSON Schema, of type
object. -
handler(ToolHandler) –Runs one call; a coroutine function.
-
timeout_s(float) –The most one call may take; past it the call is cancelled and the model is shown an error result.
-
untrusted_output(bool) –What the tool returns is data from outside (the default): it is wrapped in a
<tool_output ... untrusted="true">boundary marker. The marker is something a system prompt can name; it is not a defence on its own. -
max_output_chars(int | None) –The longest output the model is shown; None leaves it to the request's
tool_output_chars. A longer one is cut in the middle with a marker.
Raises:
-
TypeError–handleris not a coroutine function. -
ValueError–parametersis not an object schema, ortimeout_sis not positive.
ToolCallFinished
dataclass
¶
ToolCallFinished(
call_id: str,
name: str,
output: ToolOutput,
duration_s: float,
truncated: bool = False,
)
A tool call ended.
Attributes:
-
call_id(str) –The call's id.
-
name(str) –The tool's name.
-
output(ToolOutput) –What the handler returned (before the untrusted marker); an error result when it raised, timed out, or the tool or its arguments were wrong.
-
duration_s(float) –How long the handler ran, in seconds; 0 when it never ran.
-
truncated(bool) –True when the output was cut to its budget.
ToolCallStarted
dataclass
¶
ToolCallStarted(
call_id: str, name: str, args: Mapping[str, Any]
)
The model called a tool; the handler is about to run.
ToolOutput
dataclass
¶
ToolOutput(
content: str,
is_error: bool = False,
meta: Mapping[str, Any] = dict(),
)
What a tool handler returns.
Attributes:
-
content(str) –What the model is shown.
-
is_error(bool) –True when the tool failed; the model sees the content and may recover.
-
meta(Mapping[str, Any]) –For the consumer's UI; never sent to the model.
Failure
dataclass
¶
Failure(
kind: FailureKind,
retry_after_s: float | None = None,
status: int | None = None,
provider_code: str | None = None,
phase: Phase | None = None,
in_stream: bool = False,
reason: str = "",
)
One failed call's kind and the facts a retry or fallback decision reads.
Attributes:
-
kind(FailureKind) –What went wrong.
-
outcome(Outcome) –The coarse outcome; derived from
kindbyOUTCOME_FOR, never passed in. -
retryable(bool) –Whether a retry can help; derived from
kindbyRETRYABLE, never passed in. -
retry_after_s(float | None) –The server's own wait hint, in seconds, when it gave one.
-
status(int | None) –The HTTP status, when there was one.
-
provider_code(str | None) –The provider's error code or type string, verbatim.
-
phase(Phase | None) –Where a transport or deadline failure happened, when known.
-
in_stream(bool) –True when the failure came after output had started.
-
reason(str) –The one-line reason a caller persists (the 1.x
reason).
Invariants
outcome == OUTCOME_FOR[kind]andretryable == (kind in RETRYABLE).reasonis at mostoutcomes.REASON_MAXcharacters.
Rationale: ADR-0020.
Health
dataclass
¶
Health(
state: HealthState,
models: tuple[str, ...] | None = None,
reason: str | None = None,
latency_s: float | None = None,
)
One probe's answer.
Attributes:
-
state(HealthState) –up,down, orunknownwhen the probe itself failed. -
models(tuple[str, ...] | None) –The model ids the backend lists (empty when its list is empty); None when it was not asked or did not answer with a list.
-
reason(str | None) –One line naming why the state is not
up; None when it is. Never an exception's message: an error can quote a request header, credential included. -
latency_s(float | None) –Seconds the probe took; None when it never reached the backend.
Probe
¶
Chat
¶
Bases: Protocol
A multi-turn, tool-using, streamed conversation: messages in, events and new messages out.
Invariants
streamyieldsFinishedexactly once, last, in every case, and never raises for a model, wire or gate failure (ADR-0001 holds for streams). Closing the iterator or cancelling the task stops the call;asyncio.CancelledErrorpropagates.chatisstreamdrained: it returnsFinished.result. An implementation that subclassesChatinherits thatchatand writes onlystream.- The port is stateless: the caller appends
Generated.messagesto its own history.
Rationale: ADR-0023.
chat
async
¶
chat(
req: ChatRequest,
output: type[T] = str,
*,
validate: Validator[T] | None = None,
retry_message: RetryMessage | None = None
) -> Generated[T]
Run one call to its end and return the result Finished carries.
Parameters:
-
req(ChatRequest) –The resolved call.
-
output(type[T], default:str) –strfor prose, a pydantic model, or aJsonOutputsubclass. -
validate(Validator[T] | None, default:None) –Lists the problems with a parsed value, as for
Generator.generate. -
retry_message(RetryMessage | None, default:None) –Builds a re-ask's message from the problems.
stream
¶
stream(
req: ChatRequest,
output: type[T] = str,
*,
validate: Validator[T] | None = None,
retry_message: RetryMessage | None = None
) -> AsyncIterator[StreamEvent]
Run one call, yielding its events as they happen and Finished last.
Parameters:
-
req(ChatRequest) –The resolved call.
-
output(type[T], default:str) –strfor prose, a pydantic model, or aJsonOutputsubclass. -
validate(Validator[T] | None, default:None) –Lists the problems with a parsed value, as for
Generator.generate. -
retry_message(RetryMessage | None, default:None) –Builds a re-ask's message from the problems.
Citer
¶
Bases: Protocol
Citation-grounded generation: a question and chunks to cited claims.
answer
async
¶
answer(q: str, chunks: Sequence[Chunk]) -> CitedAnswer
Answer q from chunks, each claim citing the ids it rests on.
No chunks yield no claims, not an error.
Invariants
- Every claim's
evidence_idsis a subset of{c.id for c in chunks}, soCitedAnswer.unknown_ids()is empty. A citation is a chunk id, never a URL the model typed.
Counted
dataclass
¶
Counted(
tokens: int | None,
method: CountMethod,
failure: Failure | None = None,
)
An input-token count: exact, or absent.
Attributes:
-
tokens(int | None) –The count; None when this model cannot be counted or the count failed. Never an estimate.
-
method(CountMethod) –How it was measured.
-
failure(Failure | None) –Why the count request itself failed; None when it did not.
Rationale: ADR-0024.
Embedder
¶
Bases: Protocol
Text embeddings: texts to one vector each, all in one named space.
stamp
property
¶
stamp: str
The vector space this embedder writes: model id and dims (e.g. example-embed-v1@1024).
Constant for the embedder's life.
embed
async
¶
embed(
texts: Sequence[str],
*,
input_type: EmbedInput = "document"
) -> Embedded
Embed texts, one vector per text, in input order.
Never raises for a model, wire or gate failure: the failure is the outcome. A wrong
count or a wrong dimension is invalid, never a partial result.
Parameters:
-
texts(Sequence[str]) –The texts to embed.
-
input_type(EmbedInput, default:'document') –"query" lets an asymmetric model treat a search query differently from a stored document; the adapter owns any instruction prefix.
Rationale: ADR-0001.
EndFields
¶
Bases: TypedDict
Every keyword of Generation.end, for an implementation that takes **fields.
An implementation may spell the keywords out or take **fields: Unpack[EndFields]. An absent
key is the port's default: level "DEFAULT", every other key None (ADR-0025).
Evaluator
¶
Bases: Protocol
The eval harness: rows and graders to one board.
run
¶
run(
surface: str,
rows: list[dict[str, Any]],
graders: Mapping[str, Grader],
*,
id_field: str = "id",
monitors: (
Callable[[list[dict[str, Any]]], dict[str, Any]]
| None
) = None
) -> dict[str, Any]
Grade every row with every grader and return one board.
Parameters:
-
surface(str) –The name of the surface under evaluation, echoed on the board.
-
rows(list[dict[str, Any]]) –The rows to grade.
-
graders(Mapping[str, Grader]) –Grader name to grader.
-
id_field(str, default:'id') –The row key that names a row in
examples. -
monitors(Callable[[list[dict[str, Any]]], dict[str, Any]] | None, default:None) –Computes the surface's own rates over all rows; None for none.
Returns:
-
dict[str, Any]–A board with the keys
surface,n,rows_failed,by_graderandmonitors. -
dict[str, Any]–by_gradermaps each grader name topass,failandexamples(at most three -
dict[str, Any]–{id, reason}failures).monitorsholds the callable's result verbatim, or{} -
dict[str, Any]–when none was passed.
Invariants
- Every adapter emits exactly this shape, and every key is always present.
GatewayReader
¶
Bases: Protocol
What a gateway in front of the model reported in a response's headers.
A module of two functions satisfies it (llm_ports.adapters.litellm_proxy).
cost_from_headers
¶
cost_from_headers(
headers: Mapping[str, str],
) -> Decimal | None
Return the cost the gateway reported for the response, or None when it reported none.
Raises:
-
ValueError–The cost header is present but not a number. A caller treats it as absent and reports the bad header.
answered_from_headers
¶
answered_from_headers(
headers: Mapping[str, str],
) -> tuple[str | None, str | None]
Return (deployment id, upstream api base) as the gateway reported them; None for each it did not.
Generation
¶
Bases: Protocol
One open generation observation.
Invariants
- Neither method raises.
Rationale: ADR-0007.
end
¶
end(
*,
output: Any = None,
usage: Mapping[str, int | None] | None = None,
level: str = "DEFAULT",
status_message: str | None = None,
metadata: Mapping[str, Any] | None = None,
usage_v2: Usage | None = None,
cost: Cost | None = None,
finish: Finish | None = None,
answered: Answered | None = None,
failure: Failure | None = None
) -> None
Close the observation.
Parameters:
-
output(Any, default:None) –What the call produced.
-
usage(Mapping[str, int | None] | None, default:None) –The raw pair
{"input": int | None, "output": int | None}as the provider reported it, or None when it reported neither. The caller decides what unmeasured means; the adapter decides the sink's key names. -
level(str, default:'DEFAULT') –"DEFAULT"; "ERROR" for a failed call; "CANCELLED" for a call its caller cancelled (or interrupted).
-
status_message(str | None, default:None) –The error line carried with a level other than "DEFAULT".
-
metadata(Mapping[str, Any] | None, default:None) –Extra fields for the sink.
-
usage_v2(Usage | None, default:None) –The call's usage summed over every request (ADR-0019); None when unknown.
-
cost(Cost | None, default:None) –What the call cost, with its source; None when not priced.
-
finish(Finish | None, default:None) –How the last reply ended.
-
answered(Answered | None, default:None) –Who answered.
-
failure(Failure | None, default:None) –The failure's kind and detail; None for a call that succeeded.
A 1.x implementation without the five keyword-only arguments added in 1.24 keeps
working: observability passes them only to an end that accepts them.
Invariants
- A None field is omitted from what the sink receives, never sent as null.
- An unmeasured count is never written as 0.
score
¶
score(
name: str,
value: float,
comment: str | None = None,
metadata: Mapping[str, Any] | None = None,
) -> None
Attach a numeric score to this observation. Never raises.
Parameters:
-
name(str) –The score's name.
-
value(float) –Its value.
-
comment(str | None, default:None) –A one-line reason, shown beside the score.
-
metadata(Mapping[str, Any] | None, default:None) –Extra fields stored on the score itself (who gave it, for a label); omitted from what the sink receives when None.
Generator
¶
Bases: Protocol
Typed structured generation: a prompt and an output type to a value or a named outcome.
generate
async
¶
generate(
req: GenRequest,
output: type[T],
*,
validate: Validator[T] | None = None,
retry_message: RetryMessage | None = None
) -> Generated[T]
Run one call and parse the reply into output.
Never raises for a model, wire or gate failure: the failure is the outcome, with a
one-line reason.
Parameters:
-
req(GenRequest) –The resolved call.
-
output(type[T]) –A pydantic model, a
JsonOutputsubclass, orstrfor prose. -
validate(Validator[T] | None, default:None) –Lists the problems with a parsed value. A non-empty list earns
req.output_retriesre-asks, after which the value is kept with its problems. -
retry_message(RetryMessage | None, default:None) –Builds a re-ask's message from the problems.
Rationale: ADR-0001.
Grounder
¶
Bases: Protocol
Groundedness: one claim checked against the spans it cites.
verdict
¶
verdict(
claim: Claim, spans: Sequence[Chunk]
) -> GroundingVerdict
Judge
¶
Bases: Protocol
LLM-as-a-judge: a prompt, the reply it drew and a rubric to a graded verdict.
judge
async
¶
judge(
*, prompt: str, output: Any, rubric: str
) -> JudgeVerdict
Grade output, the reply to prompt, against rubric.
Raises when the judging model could not answer; the caller decides what an unjudged sample means.
Rationale: ADR-0001.
Limiter
¶
Bases: Protocol
In-process concurrency: at most N calls at once to one group of models.
A call takes a slot INSIDE its deadline and OUTSIDE its retry loop: waiting spends the caller's deadline, and a retry keeps the slot it holds. Cross-process limits stay with whatever is in front of the model (a gateway answers 429 and the wire policy waits).
Rationale: ADR-0021.
slot
¶
slot(group: str) -> AbstractAsyncContextManager[float]
Hold one slot of group for the block; the value is the seconds waited for it.
Raises:
-
SlotWaitExceeded–The wait passed the limiter's bound; the adapter reports
rate_limited/local_limiter.
Pricer
¶
Bases: Protocol
A price table: a call's usage and who answered become what it cost.
PromptSource
¶
Bases: Protocol
Prompt management: a name and a label to a source template and its provenance.
Rendering is the caller's; the port never compiles a template.
get
¶
get(name: str, *, label: str = 'production') -> Prompt
Return the prompt name as served under label.
Synchronous, as prompt-management SDKs are; an async caller uses asyncio.to_thread.
A managed store that is down is not an error: the result is the repo seed, with
source == "repo". An adapter with no seed for name has nothing to serve and raises.
Invariants
textis never empty.
Reranker
¶
Bases: Protocol
Cross-encoder reranking: a query and documents to one order-only score per document.
rerank
async
¶
rerank(query: str, documents: Sequence[str]) -> Reranked
Score each document against query, in input order.
No documents return an ok with no scores. Never raises for a model, wire or gate
failure: the failure is the outcome, and the caller keeps the order it already had. A
reply that does not score every document exactly once is invalid, never a partial
ranking.
Rationale: ADR-0001.
Retriever
¶
Bases: Protocol
RAG retrieval: a query to ranked chunks with provenance.
Hybrid search and reranking are the adapter's concern; the port promises order and provenance.
search
async
¶
search(
q: str,
*,
k: int = 8,
date_from: date | None = None,
date_to: date | None = None
) -> list[Chunk]
Return up to k chunks for q, best first.
An empty or blank q returns []. Finding nothing is an empty list, never an exception.
Parameters:
-
q(str) –The query.
-
k(int, default:8) –The most chunks to return.
-
date_from(date | None, default:None) –Lower bound on
Chunk.as_of; None leaves it open. -
date_to(date | None, default:None) –Upper bound on
Chunk.as_of; None leaves it open.
TokenCounter
¶
Bases: Protocol
Counts a request's input tokens before it is sent.
Rationale: ADR-0024.
count
async
¶
count(
model: str,
*,
system: str,
messages: Sequence[Message],
tools: Sequence[Tool] = ()
) -> Counted
Count what model would read for this request.
Never raises: a model that cannot be counted is Counted(None, "unsupported"), a count
request that failed is Counted(None, method, failure=...).
Parameters:
Tracer
¶
Bases: Protocol
LLM observability: one generation observation per model call.
Synchronous, as tracing SDKs are; the async wrapper around a model call is
observability.observe. Fail-open.
start_generation
¶
start_generation(
*,
name: str,
model: str,
provider: str,
input: Any,
metadata: Mapping[str, Any],
session_id: str | None,
prompt_name: str | None
) -> Generation
Open one generation observation.
Never raises: a sink that is down or refusing returns a no-op Generation.
Parameters:
-
name(str) –The category a cost view groups by; the adapter stores it verbatim.
-
model(str) –The model id.
-
provider(str) –The provider name.
-
input(Any) –What the model was sent.
-
metadata(Mapping[str, Any]) –Extra fields for the sink.
-
session_id(str | None) –When set, groups the generation under that session.
-
prompt_name(str | None) –When set, links the managed prompt of that name.
Deadline
dataclass
¶
Deadline(
total_s: float,
attempt_s: float | None = None,
first_event_s: float | None = None,
started: float | None = None,
)
How long one call may take, on the event loop's monotonic clock.
Attributes:
-
total_s(float) –The whole call: slot wait, every wire request and re-ask, every backoff, every fallback tier.
-
attempt_s(float | None) –One wire request; None means
total_s. -
first_event_s(float | None) –Time to the first streamed event; None means not enforced.
-
started(float | None) –When the call started, on the loop's clock; None until
startstamps it.
Raises:
-
ValueError–A bound is not a positive finite number.
Rationale: ADR-0021.
at
property
¶
at: float
When the call must be over, on the loop's clock.
Raises:
-
ValueError–The deadline was never started.
start
¶
start(now: float) -> Deadline
This deadline, started at now; one already started is returned as is.
remaining
¶
remaining(now: float) -> float
Seconds left at now, never below 0; total_s when not started.
attempt_bound
¶
attempt_bound(now: float) -> float
The bound on one wire request started at now: attempt_s, never past the deadline.
RetryRule
dataclass
¶
RetryRule(
attempts: int,
wait_initial_s: float,
wait_max_s: float,
multiplier: float = 2.0,
jitter: float = 0.1,
honour_retry_after: bool = True,
retry_after_max_s: float = 120.0,
)
How one failure kind is retried.
Attributes:
-
attempts(int) –Tries of one wire request for this kind, the first included; 1 means no retry.
-
wait_initial_s(float) –The wait before the second try, in seconds.
-
wait_max_s(float) –The cap on one wait, in seconds.
-
multiplier(float) –Each later wait is the previous one times this.
-
jitter(float) –A fraction of each wait added or removed at random; 0 disables it.
-
honour_retry_after(bool) –Wait what the server's
retry-after-ms/Retry-Aftersays instead of the computed wait. -
retry_after_max_s(float) –A server hint longer than this ends the retry; the
Failurekeeps the hint so the caller can reschedule.
Raises:
-
ValueError–attemptsis below 1, a wait is negative, the cap is below the initial wait,multiplieris below 1, orjitteris outside [0, 1].
wait_s
¶
wait_s(n: int, *, spread: float = 0.0) -> float
The wait after the n-th failed try (from 1), capped, with spread in [-1, 1] of jitter. Pure.
SlotWaitExceeded
¶
SlotWaitExceeded(group: str, waited_s: float)
Bases: Exception
Raised by a Limiter whose wait for a slot passed its max_wait_s.
An adapter turns it into rate_limited with provider_code="local_limiter", not retried.
Attributes:
-
group–The concurrency group.
-
waited_s–How long the call waited, in seconds.
WirePolicy
dataclass
¶
WirePolicy(
rules: Mapping[FailureKind, RetryRule] = dict(),
max_total_attempts: int = 60,
)
Which failure kinds one wire request is retried on, and how.
Attributes:
-
rules(Mapping[FailureKind, RetryRule]) –Failure kind to its
RetryRule; a kind not listed is not retried. -
max_total_attempts(int) –Wire requests one call may make across every kind and every request, a hard ceiling.
Raises:
-
ValueError–A rule's key is not in
RETRYABLE, ormax_total_attemptsis below 1.
Rationale: ADR-0021.
default
classmethod
¶
default() -> WirePolicy
rate_limited 3 tries; overloaded, server, transport, timeout 2 tries; waits 0.5-8 s.
none
classmethod
¶
none() -> WirePolicy
No retry: every request is sent once (an interactive first-token path).
from_mapping
classmethod
¶
from_mapping(
raw: Mapping[str, Any], *, path: str = "policy"
) -> WirePolicy
Read a policy written as data: {<kind>: {<RetryRule field>: value}, max_total_attempts: n}.
Parameters:
-
raw(Mapping[str, Any]) –The policy, e.g. a tier table's
[policies.<name>]section. -
path(str, default:'policy') –Where the policy sits, named in an error.
Raises:
-
ValueError–An unknown or non-retryable kind, an unknown field, or a value a
RetryRulerefuses; the message starts with the offending path.
Chunk
dataclass
¶
Chunk(
id: str,
text: str,
title: str = "",
source: str = "",
url: str = "",
as_of: date | None = None,
score: float | None = None,
meta: Mapping[str, Any] = dict(),
)
One retrievable unit of text, with its provenance.
Attributes:
-
id(str) –The id a
Claimcites. -
title(str) –Display title; empty when the source has none.
-
source(str) –The corpus or feed the chunk came from (e.g. "wiki", "tickets", "web").
-
url(str) –The chunk's address; empty when it has none.
-
as_of(date | None) –The date the content speaks for; None when undated.
-
score(float | None) –Adapter-relative relevance (a cross-encoder's raw logit, a fused rank score).
-
meta(Mapping[str, Any]) –Adapter extras, read by name.
Invariants
scoreis order-only: it ranks one adapter's results and is never a threshold or compared across adapters.
CitedAnswer
dataclass
¶
CitedAnswer(
question: str,
claims: tuple[Claim, ...],
chunks: Mapping[str, Chunk],
model: str = "",
extras: Mapping[str, Any] = dict(),
)
What a Citer returns: the cited claims and the chunks they may cite.
Carrying the chunks lets a Grounder read the cited text without a second retrieval.
Attributes:
-
question(str) –The question answered.
-
claims(tuple[Claim, ...]) –The claims, in the order written.
-
chunks(Mapping[str, Chunk]) –Chunk id to chunk; the universe the claims may cite.
-
model(str) –The model the adapter reports; empty when the adapter has none (a stub).
-
extras(Mapping[str, Any]) –The adapter's own per-answer report (retries, latency, ...). A consumer reads a key by name; the port defines none.
unknown_ids
¶
unknown_ids() -> tuple[str, ...]
Return the cited ids that are not in chunks.
A non-empty result means the citer broke the Citer contract.
Claim
dataclass
¶
Claim(
text: str,
evidence_ids: tuple[str, ...] = (),
meta: Mapping[str, Any] = dict(),
)
A sentence the writer asserts, with the ids of the chunks it rests on.
Attributes:
-
evidence_ids(tuple[str, ...]) –Ids of the cited chunks; empty means the claim is ungrounded.
-
meta(Mapping[str, Any]) –Adapter extras for this claim, read by name.
Embedded
dataclass
¶
Embedded(
vectors: tuple[tuple[float, ...], ...],
outcome: Outcome,
stamp: str = "",
reason: str | None = None,
input_tokens: int | None = None,
retries: int = 0,
usage_v2: Usage | None = None,
cost: Cost | None = None,
answered: Answered | None = None,
failure: Failure | None = None,
)
Bases: _Outcomed
What an Embedder returns: one vector per input, in input order, all in one space.
Attributes:
-
vectors(tuple[tuple[float, ...], ...]) –One vector per input;
()exactly whenoutcome != "ok". -
outcome(Outcome) –How the call ended.
-
stamp(str) –The vector space (model and dims). It belongs on every stored vector.
-
reason(str | None) –One line naming the failure; None on
ok. Derived fromfailurewhen left None. -
input_tokens(int | None) –Tokens the wire reported; None when it reported no usage (unknown, never 0).
-
retries(int) –Requests the adapter re-sent after a retryable failure (its
RetryPolicy), counted whether the call then landed or not. -
usage_v2(Usage | None) –Token counts summed over every request; None when no request was made.
-
cost(Cost | None) –What the call cost; None only when no
Pricerwas configured. -
answered(Answered | None) –Who produced the vectors.
-
failure(Failure | None) –The failure's kind and detail; None exactly when
outcome == "ok".
Invariants
- Two stamps never compare: a store holding two is a reindex in progress, not an index.
- A batch that half-landed is a failed batch; no partial result is returned.
Rationale: ADR-0001, ADR-0019.
Generated
dataclass
¶
Generated(
value: T | None,
outcome: Outcome,
model: str = "",
reason: str | None = None,
retries: int | None = 0,
problems: tuple[Any, ...] = (),
usage: Mapping[str, int | None] | None = None,
sources: tuple[str, ...] | None = None,
finish_reason: str | None = None,
stamp: Mapping[str, Any] = dict(),
usage_v2: Usage | None = None,
cost: Cost | None = None,
finish: Finish | None = None,
answered: Answered | None = None,
attempts: tuple[Attempt, ...] = (),
warnings: tuple[CallWarning, ...] = (),
failure: Failure | None = None,
messages: tuple[Message, ...] = (),
steps: int | None = None,
)
What a Generator returns.
Attributes:
-
value(T | None) –The parsed value. None when
outcome != "ok", excepttruncated, which keeps a length-cut reply that still parsed so a caller that accepts cut text can read it. -
outcome(Outcome) –How the call ended.
-
model(str) –The model id the request named. Deprecated:
answeredsays who answered. -
reason(str | None) –One line naming the failure; None on
ok. -
retries(int | None) –Re-asks used; None when the run died mid-flight and the count is unknown, never 0.
-
problems(tuple[Any, ...]) –The validator's findings on the kept value once the re-asks ran out. The caller prunes the value or drops it.
-
usage(Mapping[str, int | None] | None) –{"input", "output"}token counts; None, or a None count, is unmeasured. Deprecated: derived fromusage_v2. -
sources(tuple[str, ...] | None) –URLs the model server fetched while answering (a research-server extension); None when the reply carried none.
-
finish_reason(str | None) –The provider's finish reason, when it sent one. Deprecated: derived from
finish.raw. -
stamp(Mapping[str, Any]) –Extras the call path attaches by name (e.g.
scores,generation). -
usage_v2(Usage | None) –Every bucket, summed over every wire request; None when no request was made.
-
cost(Cost | None) –What the call cost; None only when no
Pricerwas configured and no gateway reported a cost. -
finish(Finish | None) –How the last reply ended; None when the provider said nothing.
-
answered(Answered | None) –Who produced the result.
-
attempts(tuple[Attempt, ...]) –Every wire request or fallback step that did not answer, in order.
-
warnings(tuple[CallWarning, ...]) –Settings dropped or altered on the way to the model.
-
failure(Failure | None) –The failure's kind and detail; None exactly when
outcome == "ok". -
messages(tuple[Message, ...]) –The new messages a
Chatcall produced (assistant turns and tool results, in order), ready to append to the history;()for aGeneratorcall. -
steps(int | None) –The model requests a
Chatcall made; None for aGeneratorcall.
Invariants
- When left None,
usageisusage_v2.as_mapping(),finish_reasonisfinish.rawandreasonisfailure.reason.
Rationale: ADR-0001, ADR-0019, ADR-0020.
GenRequest
dataclass
¶
GenRequest(
model: str,
prompt: str,
system: str = "",
images: tuple[tuple[bytes, str], ...] = (),
settings: Mapping[str, Any] = dict(),
self_hosted: bool = True,
timeout_s: float = 120.0,
output_retries: int = 1,
label: str = "chat",
accept_truncated: bool = False,
deadline: Deadline | None = None,
concurrency_group: str | None = None,
policy: WirePolicy | None = None,
stream_internally: bool = False,
cache_prefix: bool = False,
)
One model call, already resolved from a role: model id, rendered prompt and wire settings.
Attributes:
-
model(str) –The model id.
-
prompt(str) –The rendered user prompt.
-
system(str) –The rendered system prompt; empty for none.
-
images(tuple[tuple[bytes, str], ...]) –(bytes, MIME type) pairs for a vision model.
-
settings(Mapping[str, Any]) –Sent to the model verbatim (max_tokens, temperature, seed, top_p, extra_body).
-
self_hosted(bool) –True for a self-hosted tier. An adapter may take a lease on that model group, and a role sends it the thinking switch.
-
timeout_s(float) –The call's bound, in seconds:
Deadline(total_s=timeout_s)whendeadlineis None. -
output_retries(int) –Re-asks allowed when the validator finds problems.
-
label(str) –A name for the call in an adapter's logs;
call.request_forsets the role name. -
accept_truncated(bool) –Return a length-cut reply that parsed as
ok, with aCallWarning, instead oftruncated. Off by default. -
deadline(Deadline | None) –The call's clocks: total, per wire request, first event. None means
Deadline(total_s=timeout_s). A started deadline (call.generatestarts one) carries what an earlier step already spent. -
concurrency_group(str | None) –The
Limitergroup the call takes a slot in; None means the model id. -
policy(WirePolicy | None) –How the adapter retries a wire request; None means the adapter's own default.
-
stream_internally(bool) –Request a stream and assemble it, so a cancelled or timed-out call closes a connection the server can act on. Off by default.
-
cache_prefix(bool) –Ask for a prompt-cache breakpoint after the system text, the call's stable prefix (ADR-0024). A request, not a guarantee:
Usage.cache_readsays if it was served.
Rationale: ADR-0021, ADR-0024.
GroundingVerdict
dataclass
¶
GroundingVerdict(
supported: bool,
reason: str = "",
score: float | None = None,
checker: str = "",
)
What a Grounder returns for one claim.
Attributes:
-
supported(bool) –The gate.
-
reason(str) –One line saying why.
-
score(float | None) –The adapter's own scale (an entailment model's probability); None for a rule-based check that has none. Reported, never compared across adapters.
-
checker(str) –The adapter that produced the verdict (e.g. "number_law", "hhem", "lettucedetect").
JsonOutput
¶
Bases: dict[str, Any]
A hand-written JSON schema used as a Generator output type.
The value is a plain dict, and the schema reaches the model unchanged. Callers use
json_output, which mints one subclass per schema; each Generator adapter translates that
subclass into its framework's structured-output object (pydantic-ai: StructuredDict), so the
facade (call.json_output) imports no adapter.
Attributes:
-
json_schema(Mapping[str, Any]) –The schema sent to the model.
-
schema_name(str) –The schema's title.
JudgeVerdict
dataclass
¶
JudgeVerdict(
passed: bool,
score: float,
reason: str = "",
judge: str = "",
)
What a Judge returns.
Attributes:
-
passed(bool) –The gate.
-
score(float) –The judge's own reading, from 0 to 1.
-
reason(str) –The judge's one- or two-sentence justification, recorded as the score comment.
-
judge(str) –The adapter that produced the verdict (e.g. "pydantic_evals", "stub").
Prompt
dataclass
¶
Prompt(
name: str,
text: str,
version: int | None = None,
source: str = "repo",
config: Mapping[str, Any] = dict(),
system: str = "",
)
What a PromptSource returns: a source template and its provenance.
Attributes:
-
name(str) –The prompt's name.
-
text(str) –The source template. The caller renders it; an adapter never compiles it.
-
version(int | None) –The managed copy's integer version; None for the repo seed.
-
source(str) –"repo" for the repo seed, otherwise the managed copy's origin (e.g. "langfuse:v7").
-
config(Mapping[str, Any]) –The adapter's per-version extras (content sha, pack version, model role, ...), read by name.
-
system(str) –The system template of a two-message (chat) prompt; empty for a text prompt.
Invariants
version is Noneexactly whensource == "repo". The repo seed is served when the managed copy is absent, unreachable or disagrees with the seed.
Reranked
dataclass
¶
Reranked(
scores: tuple[float, ...],
outcome: Outcome,
model: str = "",
reason: str | None = None,
answered: Answered | None = None,
failure: Failure | None = None,
attempts: tuple[Attempt, ...] = (),
)
Bases: _Outcomed
What a Reranker returns: one score per document, in input order.
Attributes:
-
scores(tuple[float, ...]) –One score per document;
()exactly whenoutcome != "ok". -
outcome(Outcome) –How the call ended.
-
model(str) –The model that scored.
-
reason(str | None) –One line naming the failure; None on
ok. Derived fromfailurewhen left None. -
answered(Answered | None) –Who scored.
-
failure(Failure | None) –The failure's kind and detail; None exactly when
outcome == "ok". -
attempts(tuple[Attempt, ...]) –Every failed request the reranker's wire policy re-sent, in order;
()without a policy.
Invariants
- Scores are model-relative and order-only (one model emits raw logits, another 0..1). Rank by them; never threshold them or compare them across models.
Seed
dataclass
¶
Seed(
text: str,
sha: str,
config: Mapping[str, Any] = dict(),
tags: Sequence[str] = (),
system: str = "",
kind: str = "",
)
What the repo says a prompt is.
A PromptSource serves a seed as a Prompt with source == "repo", and a managed store
mirrors it.
Attributes:
-
text(str) –The source template.
-
sha(str) –The content sha that names
text. -
config(Mapping[str, Any]) –Carried beside the text (pack version, model role, ...).
-
tags(Sequence[str]) –Tags for the managed copy.
-
system(str) –The system template; non-empty makes the seed a chat prompt (system + user).
-
kind(str) –"chat" or "text" forces the managed prompt's type; empty derives it from
system.
Answered
dataclass
¶
Answered(
requested: str,
model: str | None = None,
provider: str | None = None,
tier: str | None = None,
deployment: str | None = None,
response_id: str | None = None,
)
Who produced a result.
Attributes:
-
requested(str) –The model id the request carried (a route or an alias is fine).
-
model(str | None) –The model name the response reported; None when it reported none. Never an echo of
requested. -
provider(str | None) –The provider that served it, when the caller's provider table names it.
-
tier(str | None) –The tier that produced this result, when the caller's routing names it.
-
deployment(str | None) –The gateway deployment id, when a gateway reported one.
-
response_id(str | None) –The response's own id, when it had one.
Rationale: ADR-0019.
Attempt
dataclass
¶
Attempt(
n: int,
tier: str | None,
model: str,
failure: Failure,
waited_s: float,
usage: Usage | None = None,
)
One wire request or fallback step that did not answer, and the wait that followed it.
Attributes:
-
n(int) –The attempt's number, from 1.
-
tier(str | None) –The tier it ran on, when known.
-
model(str) –The model it asked.
-
failure(Failure) –How it failed.
-
waited_s(float) –The backoff slept before the next attempt, in seconds.
-
usage(Usage | None) –What it consumed, when it reported anything.
Rationale: ADR-0019.
CallWarning
dataclass
¶
CallWarning(kind: WarningKind, setting: str, detail: str)
A setting that was dropped or altered on its way to the model.
Attributes:
-
kind(WarningKind) –What happened to it.
-
setting(str) –The setting's name.
-
detail(str) –One line saying what happened.
Rationale: ADR-0019.
Cost
dataclass
¶
Cost(
usd: Decimal | None,
source: CostSource,
priced_model: str | None = None,
reason: str | None = None,
)
What a call cost, in US dollars, and where that number came from.
Attributes:
-
usd(Decimal | None) –The cost; None when unknown, never 0 for unknown.
-
source(CostSource) –Where
usdcame from. -
priced_model(str | None) –The model reference the price was looked up under.
-
reason(str | None) –Why the cost is unknown:
no_usage,unknown_model,no_pricerorpricer_error.
Rationale: ADR-0019.
Finish
dataclass
¶
Finish(unified: UnifiedFinish, raw: str | None)
How the last reply of a call ended.
Attributes:
-
unified(UnifiedFinish) –The closed-vocabulary reading of
raw;otherfor a value no mapping knows. -
raw(str | None) –The provider's own string, unmodified.
Rationale: ADR-0019.
Usage
dataclass
¶
Usage(
input: int | None = None,
output: int | None = None,
cache_read: int | None = None,
cache_write: int | None = None,
reasoning: int | None = None,
requests: int = 0,
unmeasured: int = 0,
details: Mapping[str, int] = dict(),
)
Token counts summed over every wire request of one call.
Attributes:
-
input(int | None) –All input tokens, cached included; None when no request reported it.
-
output(int | None) –All output tokens, reasoning included; None when no request reported it.
-
cache_read(int | None) –The part of
inputserved from a provider cache. -
cache_write(int | None) –The part of
inputwritten to a provider cache. -
reasoning(int | None) –The part of
outputspent reasoning. -
requests(int) –Wire requests folded in (output re-asks, tool steps, retries that returned a body).
-
unmeasured(int) –Of those, how many reported no usage at all.
-
details(Mapping[str, int]) –Provider extras under their own keys, summed key by key.
Invariants
cache_read <= inputandreasoning <= outputwhenever both sides are known.
Rationale: ADR-0019.
failure
¶
failure(
kind: FailureKind,
*,
retry_after_s: float | None = None,
status: int | None = None,
provider_code: str | None = None,
phase: Phase | None = None,
in_stream: bool = False,
reason: str = ""
) -> Failure
Return the Failure of kind with the given detail; the one constructor src/ uses.
Pure. The arguments are the Failure attributes of the same name; reason defaults to the
kind itself, so a failure always names something.
Raises:
-
ValueError–kindis not aFailureKind.
json_output
¶
json_output(
schema: Mapping[str, Any], name: str
) -> type[JsonOutput]
Return a JsonOutput subclass that carries schema and name.
Pure.
Parameters:
-
schema(Mapping[str, Any]) –The JSON schema, sent to the model unchanged.
-
name(str) –The schema's title, and the name a test stub dispatches on.
llm_ports.ports.outcomes¶
Named failure reasons for a failed model call, and the HTTP-status rule every adapter shares.
outcome_for_status names the outcome an HTTP error status maps to: a status in REQUEST_FAULT
blames the request, any other the server. failure_reason turns an exception into the one reason line a caller persists: the exception
class, the status code when there is one, and the head of the message. A provider's error payload
belongs in a log line, not in a persisted reason, and a proxy error can embed a keyed URL.
Invariants
- Pure: no I/O.
- A reason line never carries the response body and never exceeds
REASON_MAXcharacters.
Rationale: ADR-0001.
GenerationFailed
¶
GenerationFailed(outcome: str, reason: str | None)
Bases: RuntimeError
A non-ok, non-gated outcome raised for a caller whose contract is that a failure propagates.
The facade raises it for a caller that expects an exception (a Citer, or a caller that records
a failed row from an except block).
Attributes:
-
outcome–The
Outcomethe call ended with. -
reason–The one-line reason, or None.
outcome_for_status
¶
outcome_for_status(status: int) -> Outcome
Return the outcome an HTTP error status maps to.
Pure. A status in REQUEST_FAULT is request_fault: retrying the same input fails the same
way. Any other status is unavailable: the server, not the input, failed.
failure_reason
¶
failure_reason(
exc: BaseException, *, with_cause: bool = False
) -> str
Return the persisted reason line for exc.
The line is the exception class, the status code when the exception carries one, and the
first line of its message cut before any body: payload.
Parameters:
-
exc(BaseException) –The exception to name.
-
with_cause(bool, default:False) –Append
<- <the cause's reason>, cut by the same rule. A wrapper's own line can name nothing ("Exceeded maximum output retries") while its cause ("Invalid JSON: EOF while parsing a string") names the fix.
Returns:
-
str–A line of at most
REASON_MAXcharacters; the cause keeps up to half of it.
exception_chain
¶
exception_chain(
exc: BaseException, limit: int = CHAIN_LIMIT
) -> Iterator[BaseException]
Yield exc, then its __cause__ (else __context__), and so on down the chain.
SDK layers (pydantic-ai, openai, httpx, a transport) each wrap the one below, so the root cause is never on top.
Parameters:
-
exc(BaseException) –The outermost exception.
-
limit(int, default:CHAIN_LIMIT) –The most exceptions to yield. A cycle ends the chain early.