llm_ports.adapters¶
llm_ports.adapters
¶
Adapters: every module that imports a vendor library lives here, one extra each.
Import an adapter by its path (llm_ports.adapters.pydantic_ai, llm_ports.adapters.langfuse,
...); nothing here is imported by the rest of the package.
llm_ports.adapters.pydantic_ai¶
Adapters on pydantic-ai: generator, chat, embedder, output schemas, replay, the openai_compatible kind.
PydanticAIGenerator, PydanticAIChat, PydanticAIEmbedder, replay, and OpenAICompatibleFactory, the
openai_compatible provider kind.
The failure-classification table every adapter shares (classify_core, WireFacts) and the
pydantic-ai categories over it (classify_failure) are exported here too.
Needs the pydantic-ai extra (pip install 'llm-ports[pydantic-ai]').
IndexOrderedTransport
¶
IndexOrderedTransport(
inner: AsyncBaseTransport | None = None,
)
Bases: AsyncBaseTransport
An httpx transport that re-sorts a 200 reply's index-keyed list (INDEXED_LISTS) by index.
Parameters:
-
inner(AsyncBaseTransport | None, default:None) –The transport that sends the request; None uses httpx's own.
Invariants
- A non-200 reply, a path not in
INDEXED_LISTS, an unreadable body, or a list already in index order is returned unchanged.
Classifier
¶
Bases: Protocol
An extra classifier: knowledge a gateway or provider adds ahead of the shared table.
WireFacts
dataclass
¶
WireFacts(
status: int | None = None,
headers: Mapping[str, str] = dict(),
finish_raw: str | None = None,
error_code: str | None = None,
error_type: str | None = None,
error_message: str | None = None,
refusal: bool = False,
)
What the wire showed of a call's last response, with no body.
Attributes:
-
status(int | None) –The HTTP status, when a response arrived.
-
headers(Mapping[str, str]) –The response headers, names lower-cased.
-
finish_raw(str | None) –The reply's finish reason, verbatim.
-
error_code(str | None) –The provider's error code, verbatim.
-
error_type(str | None) –The provider's error type, verbatim.
-
error_message(str | None) –The provider's error text, at most
ERROR_MESSAGE_MAXcharacters. Never the prompt or the completion. -
refusal(bool) –True when the reply carried an explicit refusal field.
CaptureLike
¶
Bases: Protocol
What PydanticAIGenerator reads from a capture after a call; the type of its capture=.
A capture may also carry replies (a sequence of WireReply) and last (the latest
WireFacts); the generator reads them when present and falls back to pydantic-ai's own
messages when not.
usage
property
¶
usage: Mapping[str, int | None]
input and output token counts; None for a count the server did not report.
WireReply
dataclass
¶
WireReply(
usage: Usage,
finish_raw: str | None = None,
model: str | None = None,
response_id: str | None = None,
refusal: bool = False,
headers: Mapping[str, str] = dict(),
)
One 2xx reply of a call, as the wire showed it.
Attributes:
-
usage(Usage) –The reply's usage, one request; unmeasured when it carried no usage block.
-
finish_raw(str | None) –The first choice's
finish_reason, verbatim. -
model(str | None) –The
modelthe body reported. -
response_id(str | None) –The body's
id. -
refusal(bool) –True when the first choice's message carried a non-empty
refusal. -
headers(Mapping[str, str]) –The response headers, names lower-cased.
PydanticAIChat
¶
PydanticAIChat(
model: Model | None = None,
*,
model_for: (
Callable[[R], AbstractAsyncContextManager[Model]]
| None
) = None,
capture: Callable[
[], AbstractContextManager[CaptureLike]
] = capture,
gated: tuple[type[BaseException], ...] = (),
wrap: Callable[[Model], Model] | None = None,
pricer: Pricer | None = None,
gateway: GatewayReader | None = None,
extra: Sequence[Classifier] = (),
on_event: EventHook | None = None,
policy: WirePolicy | None = None,
limiter: Limiter | None = None
)
Bases: AdapterCore[ChatRequest], Chat
The Chat port on pydantic-ai; the module docstring states the contract.
Parameters:
-
model(Model | None, default:None) –A fixed pydantic-ai
Model. -
model_for(Callable[[R], AbstractAsyncContextManager[Model]] | None, default:None) –A per-request model factory (e.g.
openai_compatible(...), the same object aPydanticAIGeneratortakes, so both share one client). -
capture(Callable[[], AbstractContextManager[CaptureLike]], default:capture) –The capture factory; the default is filled by
openai_compatible's response hook. -
gated(tuple[type[BaseException], ...], default:()) –Exception types that classify as
gated. -
wrap(Callable[[Model], Model] | None, default:None) –Applied to each model
model_foryields, e.g. areplay.Recorder. -
pricer(Pricer | None, default:None) –Prices a call the gateway did not price.
-
gateway(GatewayReader | None, default:None) –Reads a gateway's response headers: its cost and deployment.
-
extra(Sequence[Classifier], default:()) –Classifiers consulted before the shared table, in order.
-
on_event(EventHook | None, default:None) –Receives the generator's events (
llm_call_internal_error,llm_cost_unknown,llm_usage_unmeasured,llm_wire_retry,llm_deadline_expired, ...). -
policy(WirePolicy | None, default:None) –How a failed model request is retried when the request carries none.
-
limiter(Limiter | None, default:None) –Caps concurrent calls per
ChatRequest.concurrency_group(else per model id).
Raises:
-
ValueError–When neither or both of
modelandmodel_forare given.
stream
async
¶
stream(
req: ChatRequest,
output: type[T] = str,
*,
validate: Validator[T] | None = None,
retry_message: RetryMessage | None = None
) -> AsyncGenerator[StreamEvent]
Run one call, yielding its events as they happen and Finished last, exactly once.
The result's stamp carries first_event_s (seconds to the first event; None when there
was none), limiter_wait_s and wire_requests.
Raises:
-
CancelledError–The consumer was cancelled; the call's task is cancelled first.
OpenAICompatible
¶
OpenAICompatible(
base_url: str,
api_key: str | None,
*,
max_retries: int,
transport: (
AsyncBaseTransport | AsyncBaseTransport | None
),
profile: ModelProfileSpec | None,
settings: ModelSettings | None,
client: AsyncClient | None
)
A ModelFor over one OpenAI-wire chat-completions endpoint, on one reused client per loop.
Built by openai_compatible, which documents the arguments. The provider (and the SDK client
it wraps) is built once per event loop, with the SDK's own retries at max_retries; each
request gets an OpenAIChatModel for its model id on it.
PydanticAIEmbedder
¶
PydanticAIEmbedder(
model: EmbeddingModel | None = None,
*,
stamp: str,
dims: int | None = None,
model_for: EmbedModelFor | None = None,
batch_size: int = 16,
query_instruct: str = "",
gated: tuple[type[BaseException], ...] = (),
retry: RetryPolicy | None = None,
pricer: Pricer | None = None,
gateway: GatewayReader | None = None,
on_event: EventHook | None = None,
policy: WirePolicy | None = None
)
The Embedder port on pydantic-ai; the module docstring states the contract.
Parameters:
-
model(EmbeddingModel | None, default:None) –A fixed pydantic-ai
EmbeddingModel. -
stamp(str) –The name of the space the vectors live in; an unnamed space cannot be stored.
-
dims(int | None, default:None) –When given, every vector must be this long, or the call is
invalid. -
model_for(EmbedModelFor | None, default:None) –A per-call model factory.
-
batch_size(int, default:16) –Inputs per request.
-
query_instruct(str, default:'') –The task line prefixed to
input_type="query"inputs; empty sends queries plain. -
gated(tuple[type[BaseException], ...], default:()) –Exception types that classify as
gated. -
retry(RetryPolicy | None, default:None) –Deprecated:
policy=retry.wire()is the same. Pass at most one of the two. -
policy(WirePolicy | None, default:None) –Which failed requests are re-sent, and how; None (and no
retry) sends each request once. -
pricer(Pricer | None, default:None) –Prices a call the gateway did not price; None leaves
costNone unless a gateway reported one. -
gateway(GatewayReader | None, default:None) –Reads a gateway's response headers (e.g. the module
adapters.litellm_proxy). -
on_event(EventHook | None, default:None) –Receives
llm_pricer_failed,llm_gateway_header_invalidandllm_wire_retry; without one they go to the stdlib loggerllm_ports.adapters.pydantic_ai.
Raises:
-
ValueError–When neither or both of
modelandmodel_forare given, bothretryandpolicyare given,stampis empty, orbatch_sizeis below 1.
embed
async
¶
embed(
texts: Sequence[str],
*,
input_type: EmbedInput = "document"
) -> Embedded
Embed texts in order; a model, wire or gate failure returns a named outcome, never raises.
input_type="query" applies query_instruct. Empty texts returns ok with no vectors
and zero input tokens. A reply that breaks the count or dimension check is invalid. A
failure the generator would call invalid_output, truncated or refused is server
(unavailable) here, since an embeddings reply has no parse step, token ceiling or
refusal; its reason ends with the root cause of the exception chain (refused, timed out,
DNS).
Returns:
-
Embedded–An
Embeddedstamped with this embedder's space.input_tokensis None when any -
Embedded–batch did not report usage, never 0.
Rationale: ADR-0001, ADR-0007, ADR-0019.
OpenAICompatibleFactory
¶
OpenAICompatibleFactory(
*,
gated: tuple[type[BaseException], ...] = (),
probe_deadline_s: float = 5.0
)
The openai_compatible provider kind.
Parameters:
-
gated(tuple[type[BaseException], ...], default:()) –Exception types the generator classifies as
gated(a refusal raised by the caller's transport). -
probe_deadline_s(float, default:5.0) –The probe's deadline, in seconds.
Attributes:
-
kind(str) –openai_compatible.
build
¶
build(
spec: ProviderSpec,
*,
url: str | None,
secret: str | None,
models: Sequence[ModelInfo],
deps: BuildDeps
) -> Built
Build the provider's Generator, Embedder source and Probe.
Raises:
-
TableInvalid–The provider names no
base_url, or sets an option this kind does not take.
PydanticAIGenerator
¶
PydanticAIGenerator(
model: Model | None = None,
*,
model_for: (
Callable[[R], AbstractAsyncContextManager[Model]]
| None
) = None,
capture: Callable[
[], AbstractContextManager[CaptureLike]
] = capture,
gated: tuple[type[BaseException], ...] = (),
wrap: Callable[[Model], Model] | None = None,
pricer: Pricer | None = None,
gateway: GatewayReader | None = None,
extra: Sequence[Classifier] = (),
on_event: EventHook | None = None,
policy: WirePolicy | None = None,
limiter: Limiter | None = None
)
Bases: AdapterCore[GenRequest]
The Generator port on pydantic-ai; the module docstring states the contract.
Parameters:
-
model(Model | None, default:None) –A fixed pydantic-ai
Model. -
model_for(Callable[[R], AbstractAsyncContextManager[Model]] | None, default:None) –A per-request model factory.
-
capture(Callable[[], AbstractContextManager[CaptureLike]], default:capture) –The capture factory; the default is filled by
openai_compatible's response hook. -
gated(tuple[type[BaseException], ...], default:()) –Exception types that classify as
gated. -
wrap(Callable[[Model], Model] | None, default:None) –Applied to each model that
model_foryields (not to a fixedmodel), e.g. areplay.Recorder. -
pricer(Pricer | None, default:None) –Prices a call the gateway did not price (e.g.
adapters.genai_prices.GenaiPricesPricer); None leavescostNone unless a gateway reported one. -
gateway(GatewayReader | None, default:None) –Reads a gateway's response headers (e.g. the module
adapters.litellm_proxy): its cost, which takes precedence overpricer, and its deployment id. -
extra(Sequence[Classifier], default:()) –Classifiers consulted before the shared table, in order (e.g.
adapters.litellm_proxy.classify_gateway). -
on_event(EventHook | None, default:None) –Receives
llm_call_internal_error,llm_cost_unknown,llm_usage_unmeasured,llm_pricer_failed,llm_gateway_header_invalid,llm_wire_retryandllm_deadline_expired; without one they go to the stdlib loggerllm_ports.adapters.pydantic_ai. -
policy(WirePolicy | None, default:None) –How a failed model request is retried when the request carries no policy of its own; None is
WirePolicy.default().WirePolicy.none()sends every request once. -
limiter(Limiter | None, default:None) –Caps concurrent calls per
GenRequest.concurrency_group(else per model id); None runs every call at once.
Raises:
-
ValueError–When neither or both of
modelandmodel_forare given.
generate
async
¶
generate(
req: GenRequest,
output: type[T],
*,
validate: Validator[T] | None = None,
retry_message: RetryMessage | None = None
) -> Generated[T]
Run one generation; a model, wire or gate failure returns a named outcome, never raises.
Parameters:
-
req(GenRequest) –The call's prompt, model and limits.
-
output(type[T]) –strfor prose, a pydantic model, or aJsonOutputsubclass. -
validate(Validator[T] | None, default:None) –Returns the problems in a parsed value. A non-empty list triggers a re-ask while
req.output_retriesremain; after that the value is kept with its problems. An exception it raises isinternal, never a model outage. -
retry_message(RetryMessage | None, default:None) –Renders the problems into the re-ask message; by default they are joined with "; ".
Returns:
-
Generated[T]–A
Generatedwhoseoutcomeisokor a named failure. Itsattemptshold every -
Generated[T]–retried request;
stamp["limiter_wait_s"]the seconds waited for a slot (None without -
Generated[T]–a limiter);
stamp["wire_requests"]the requests sent.
Raises:
-
CancelledError–The caller cancelled; the slot is released first.
run
async
¶
run(
model: Model,
req: GenRequest,
output: type[T],
validate: Validator[T] | None,
retry_message: RetryMessage | None,
) -> Generated[T]
Run one agent on model and classify the result; the arguments are as for generate.
A subclass that supplies model on its own transport calls this to share the run, the
validator and the classification.
classify_core
¶
classify_core(
exc: BaseException | None,
facts: WireFacts,
*,
finish: UnifiedFinish | None = None,
extra: Sequence[Classifier] = (),
gated: tuple[type[BaseException], ...] = (),
categorize: Categorize | None = None
) -> Failure | None
Return the failure a call ended with, or None for a success; the module docstring gives the order.
Parameters:
-
exc(BaseException | None) –The exception the call raised; None when it returned.
-
facts(WireFacts) –What the wire showed of the last response.
-
finish(UnifiedFinish | None, default:None) –The unified reading of
facts.finish_raw, from the adapter's own mapping. -
extra(Sequence[Classifier], default:()) –Classifiers consulted first, in order.
-
gated(tuple[type[BaseException], ...], default:()) –Exception types that mean the caller's gate refused the call.
-
categorize(Categorize | None, default:None) –The framework's own categories (step 4).
Raises:
-
CancelledError–Re-raised when
excis one; a cancellation is never an outcome.
Rationale: ADR-0020.
is_context_overflow
¶
is_context_overflow(
codes: Sequence[str | None],
messages: Sequence[str | None],
) -> bool
True when a provider code or message names a context overflow (step 5). Pure.
kind_for_status
¶
kind_for_status(status: int) -> FailureKind
Return the kind an HTTP error status maps to (step 6). Pure.
A status not in STATUS_KINDS is server (outcome unavailable), as 1.x mapped it: only the
listed 4xx say the input itself was refused.
raised_in_provider
¶
raised_in_provider(e: BaseException) -> bool
True when e was raised by code in PROVIDER_MODULES (step 9). Pure.
The owner is the innermost frame of e's traceback whose package is neither the standard
library nor a neutral helper (pydantic, anyio): a KeyError from json inside a caller's
validator is owned by the validator; an IndexError in pydantic_ai.models.openai by the
provider client.
retry_after_s
¶
retry_after_s(headers: Mapping[str, str]) -> float | None
Return the server's wait hint in seconds, from retry-after-ms or Retry-After.
Pure. Retry-After may be seconds or an HTTP date. A hint in the past, unreadable or not
finite is None.
worst
¶
Return the worst of failures by SEVERITY; the first of equals. Pure.
Raises:
-
ValueError–failuresis empty.
parse_usage
¶
parse_usage(block: Any) -> Usage
Return one reply's Usage from its OpenAI-wire usage block. Pure.
prompt_tokens and completion_tokens are the inclusive totals; prompt_tokens_details.cached_tokens
(OpenAI, llama.cpp) or cache_read_input_tokens (a gateway relaying Anthropic) is the cached
part; cache_creation_input_tokens the written part; completion_tokens_details.reasoning_tokens
the reasoning part. Other integer keys go to details under their own names (nested ones as
parent.key). No block at all is one unmeasured request.
classify
¶
classify(
exc: BaseException,
finish_reason: str | None = None,
*,
gated: tuple[type[BaseException], ...] = ()
) -> tuple[Outcome, str]
Name the outcome a failed run maps to, with a one-line reason (the 1.x signature).
A thin wrapper over classify_failure, which also returns the kind and its detail. Pure.
Parameters:
-
exc(BaseException) –The exception the run raised.
-
finish_reason(str | None, default:None) –The last reply's
finish_reason, when the wire reported one. -
gated(tuple[type[BaseException], ...], default:()) –Exception types that mean the caller's gate refused the call.
Returns:
-
tuple[Outcome, str]–The outcome and its reason.
Rationale: ADR-0001, ADR-0020.
classify_failure
¶
classify_failure(
exc: BaseException | None,
facts: WireFacts,
*,
extra: Sequence[Classifier] = (),
gated: tuple[type[BaseException], ...] = ()
) -> Failure | None
Return the failure a pydantic-ai call ended with, or None when it succeeded.
Parameters:
-
exc(BaseException | None) –The exception the run raised; None when it returned.
-
facts(WireFacts) –What the wire showed of the last response (its finish reason, status, error code).
-
extra(Sequence[Classifier], default:()) –Classifiers consulted first, in order (e.g.
adapters.litellm_proxy.classify_gateway). -
gated(tuple[type[BaseException], ...], default:()) –Exception types that mean the caller's gate refused the call.
Raises:
-
CancelledError–Re-raised when
excis one.
Rationale: ADR-0020.
openai_finish
¶
openai_finish(raw: str | None) -> UnifiedFinish | None
Return the unified reading of an OpenAI-wire finish_reason; None when there was none. Pure.
openai_compatible_embeddings
¶
openai_compatible_embeddings(
base_url: str,
model_name: str,
api_key: str | None = None,
*,
timeout_s: float = 120.0,
max_retries: int = 0,
transport: (
AsyncBaseTransport | AsyncBaseTransport | None
) = None
) -> EmbedModelFor
Build a model_for over any OpenAI-wire /embeddings endpoint.
Each call opens one httpx2.AsyncClient (the client pydantic-ai's OpenAI provider takes).
Parameters:
-
base_url(str) –The URL up to where
/embeddingsstarts, e.g.http://localhost:4000/v1. -
model_name(str) –The embedding model the endpoint serves.
-
api_key(str | None, default:None) –The bearer key; None sends a placeholder, for servers that need no key.
-
timeout_s(float, default:120.0) –The client timeout per call.
-
max_retries(int, default:0) –The OpenAI SDK's own retry count. Defaults to 0, so retries are not stacked on those of a gateway in front.
-
transport(AsyncBaseTransport | AsyncBaseTransport | None, default:None) –Replaces httpx2's own transport (a caller's admitting or counting transport, or a
MockTransportin a test). An httpx2 transport is used as is; a legacy httpx one is carried across a bridge. It always sits under the index-order re-sort.
Returns:
-
EmbedModelFor–An
EmbedModelForthat yields anOpenAIEmbeddingModel.
Raises:
-
ValueError–api_keyholds whitespace, a control or a non-ASCII character. The value is not shown.
Rationale: ADR-0009.
count_url
¶
count_url(base_url: str) -> str
The gateway's count route for an OpenAI-wire base URL: /utils/token_counter at its root. Pure.
translate_settings
¶
translate_settings(
settings: Mapping[str, Any], info: ModelInfo | None
) -> tuple[dict[str, Any], tuple[str, ...]]
Spell a request's provider-neutral thinking switch the way its model needs. Pure.
Parameters:
-
settings(Mapping[str, Any]) –The request's settings; a schema 2 role puts
thinking: boolin them. -
info(ModelInfo | None) –The request's model, or None when the table does not know it (the settings pass through unchanged, so pydantic-ai's own unified
thinkingapplies).
Returns:
-
dict[str, Any]–The settings to send, and the names of settings the model cannot take (dropped):
-
tuple[str, ...]–unifiedkeepsthinking(pydantic-ai'sModelSettings["thinking"]);chat_template -
tuple[dict[str, Any], tuple[str, ...]]–moves it to
extra_body.chat_template_kwargs.enable_thinking;nonedrops it, and names -
tuple[dict[str, Any], tuple[str, ...]]–it when the role asked to think.
openai_compatible
¶
openai_compatible(
base_url: str,
api_key: str | None = None,
*,
max_retries: int = 0,
transport: (
AsyncBaseTransport | AsyncBaseTransport | None
) = None,
profile: ModelProfileSpec | None = None,
settings: ModelSettings | None = None,
client: AsyncClient | None = None
) -> OpenAICompatible
Build a model_for over any OpenAI-wire chat-completions endpoint.
One httpx2.AsyncClient (the client pydantic-ai's OpenAI provider takes) is built per event
loop, with explicit pool limits, a 5 s connect bound and the capture hook, and reused by every
call; the per-request bound is passed per call. aclose() closes it.
Parameters:
-
base_url(str) –The URL up to where
/chat/completionsstarts, e.g.http://localhost:4000/v1. -
api_key(str | None, default:None) –The bearer key; None sends a placeholder, for servers that need no key.
-
max_retries(int, default:0) –The OpenAI SDK's own retry count. Defaults to 0, so retries are not stacked on those of a gateway in front.
-
transport(AsyncBaseTransport | AsyncBaseTransport | None, default:None) –Replaces httpx2's own transport (a caller's admitting or counting transport, or a
MockTransportin a test). An httpx2 transport is used as is; a legacy httpx one is carried across a bridge, so the provider still gets an httpx2 client. -
profile(ModelProfileSpec | None, default:None) –A pydantic-ai
ModelProfileSpec: a partialModelProfilemerged over the one the provider resolves for the model, or a function from that resolved profile to the profile to use. E.g. it moves an agent whose output type the caller does not construct onto native structured output. None keeps the provider's. -
settings(ModelSettings | None, default:None) –Model-level
ModelSettings, applied under each run's own settings (pydantic-ai merges run settings over model settings, key by key at the top level). None sets none. -
client(AsyncClient | None, default:None) –A caller's own
httpx2.AsyncClient, used as given and never closed here (addrecord_responseto its response hooks to keep the wire capture).
Returns:
-
OpenAICompatible–A
ModelFor(OpenAICompatible) that yields anOpenAIChatModelforreq.model, carrying -
OpenAICompatible–profileandsettings, withstats()andaclose().
Raises:
-
ValueError–api_keyholds whitespace, a control or a non-ASCII character (the HTTP stack would quote it, value and all, when it refused the header). The value is not shown.
Rationale: ADR-0009.
framework_output
¶
framework_output(output: Any) -> Any
Map the port's JsonOutput to pydantic-ai's StructuredDict; any other type passes unchanged.
strict_json_schema
¶
strict_json_schema(
model: type[BaseModel],
) -> dict[str, Any]
Return a response_format: json_schema, strict: true schema that OpenAI and llama.cpp accept.
OpenAI's strict mode refuses what pydantic's model_json_schema() emits: every object needs
additionalProperties: false and every property listed in required (nullable via anyOf
null). llama.cpp's grammar compiler accepts the strict form too, so one shape serves both. The
transform is pydantic-ai's OpenAIJsonSchemaTransformer.
structured_dict
¶
structured_dict(
schema: Mapping[str, Any], name: str
) -> type[dict[str, Any]]
Wrap a hand-written JSON schema as an output type that returns a plain dict.
The schema is compiled into the grammar unchanged, so maxLength, enum, maxItems and
required all survive. name is the schema's title and the name a FunctionModel stub
dispatches on. No shape validation runs beyond what the grammar enforces.
llm_ports.adapters.pydantic_ai.replay¶
Record a live model's replies once and replay them in CI, for pydantic-ai models.
Example
rec = Recorder(real_model) # wraps any pydantic-ai Model; keeps every reply ... # run the pipeline once against the live model json.dump(rec.replies, f) # {key(messages): reply text} model = replay(json.load(f)) # a FunctionModel answering from that dict
key(messages) (version 1) is the sha of the first user prompt plus the attempt number (the count
of model responses already in the conversation, so an output retry's second attempt has its own
key). key_v2 (version 2, 1.24) covers everything that changes the answer: the model, the system
prompt and instructions, the whole message list, the settings (sorted) and the output schema, plus
the attempt; a system-prompt change no longer replays the old answer. Recorder(version=2) writes
{"version": 2, "entries": {...}} (cassette()); replay reads both shapes. A changed request
misses the lookup, the call raises NotRecorded naming the parts that differ from the nearest
entry, and the golden set must be re-recorded. ReplayModel.unplayed() names the entries a run
never asked for (the llm_replay pytest fixture fails a partly played cassette with it).
A version-2 Recorder given secrets refuses (SecretInCassette) to keep a reply containing any
of them. Only the reply text is kept: no headers, no request bodies.
Recording keeps every raw reply parsed in attempts (an agent returns only the last attempt; a
grader wants every one); an unparsable reply is kept as {"_unparsed": head}, never dropped.
Either wrapper plugs into PydanticAIGenerator as its wrap= model wrapper; recording(make) and
recorder(make) build a generator with the wrapper on and return it with the list or dict it fills.
The module depends on pydantic-ai, so it is imported by its path and needs the pydantic-ai extra.
NotRecorded
¶
Bases: RuntimeError
The replay has no reply for this request: it changed since the recording.
SecretInCassette
¶
Bases: ValueError
A reply about to be recorded contains a bound secret; nothing was written.
Recorder
¶
Recorder(
wrapped: Any,
replies: dict[str, Any] | None = None,
*,
version: int = 1,
secrets: Iterable[str] = ()
)
Bases: WrapperModel
A model wrapper that keeps every reply's text in replies.
Version 1 keys by key(messages) and keeps the text; version 2 keys by key_v2 and keeps
{"text", "parts"} (the key's parts, for NotRecorded's diff). Pass replies to share one
dict across several wrapped models.
Parameters:
-
wrapped(Any) –The live model.
-
replies(dict[str, Any] | None, default:None) –The dict to fill; None starts an empty one.
-
version(int, default:1) –1 (the 1.x key, the default) or 2.
-
secrets(Iterable[str], default:()) –Values that must never be written; a reply containing one raises
SecretInCassette(version 2).
cassette
¶
cassette() -> dict[str, Any]
The recording as written to disk: version 2's {"version", "entries"}, version 1's dict.
request
async
¶
request(
messages: list[ModelMessage],
model_settings: ModelSettings | None,
model_request_parameters: ModelRequestParameters,
) -> ModelResponse
Forward to the wrapped model and record the reply under its key.
ReplayModel
¶
Bases: FunctionModel
The model replay returns; unplayed() names the entries no request asked for.
Recording
¶
Recording(wrapped: Any, attempts: list[Any] | None = None)
Bases: WrapperModel
A model wrapper that keeps every raw reply, parsed, in attempts.
Pass attempts to share one list across several wrapped models (one list per run). An
unparsable reply is kept as {"_unparsed": head}, its first UNPARSED_HEAD characters.
request
async
¶
request(
messages: list[ModelMessage],
model_settings: ModelSettings | None,
model_request_parameters: ModelRequestParameters,
) -> ModelResponse
Forward to the wrapped model and append the reply, parsed, to attempts.
key
¶
key(messages: list[ModelMessage]) -> str
Return the recording's lookup key, <sha of the first user prompt>:<attempt>.
key_parts
¶
key_parts(
messages: Sequence[ModelMessage],
*,
model: str,
system: str,
settings: Mapping[str, Any] | None,
output_schema: Any,
attempt: int
) -> dict[str, str]
The digest of each part key_v2 covers (KEY_PARTS). Pure.
key_v2
¶
key_v2(
messages: Sequence[ModelMessage],
*,
model: str,
system: str,
settings: Mapping[str, Any] | None,
output_schema: Any,
attempt: int
) -> str
Return the version-2 lookup key, <sha of every part but the attempt>:<attempt>. Pure.
replay
¶
replay(recorded: Mapping[str, Any]) -> ReplayModel
Return a model that answers from a recording: a version-1 dict or a version-2 cassette.
The returned model raises NotRecorded for a request the recording never saw.
recording
¶
recording(
make: MakeGenerator[G],
) -> tuple[G, list[Any]]
Return a generator whose every raw reply lands, parsed, in the returned list (Recording).
recorder
¶
recorder(
make: MakeGenerator[G], *, version: int = 1
) -> tuple[G, dict[str, Any]]
Return a generator whose every reply lands in the returned dict under its key (Recorder).
llm_ports.adapters.genai_prices¶
A Pricer over genai-prices: what a call cost, from the model that answered and its usage.
genai-prices ships a dated price table for the hosted providers and computes a call's price as
a Decimal from its token counts (input inclusive of cached tokens, as Usage is). This module
adapts it to ports.Pricer and adds the two things the table cannot know:
- a self-hosted model costs what the caller says:
overridesmaps a model id to a per-request price (oftenDecimal(0)for a local box, stated, never assumed); - a model the table does not know is
Cost(None, "unknown", reason="unknown_model"), never free.
Invariants
pricenever raises: a failing lookup isCost(None, "unknown", reason="pricer_error").- Unknown is None: a call with unmeasured usage is
reason="no_usage", never priced from a partial count. - No I/O: the bundled table is read; genai-prices' own
UpdatePricesrefresh is the caller's choice.
Example
gen = PydanticAIGenerator(model_for=..., pricer=GenaiPricesPricer(overrides={"local-8b": Decimal(0)}))
Rationale: ADR-0019.
GenaiPricesPricer
¶
GenaiPricesPricer(
*,
provider_for: (
Callable[[Answered], str | None] | None
) = None,
overrides: Mapping[str, Decimal] | None = None
)
Price a call from genai-prices' table, with caller overrides for models it cannot know.
Parameters:
-
provider_for(Callable[[Answered], str | None] | None, default:None) –The genai-prices provider id for a call (e.g.
"openai"), or None to let genai-prices match the model id across providers. -
overrides(Mapping[str, Decimal] | None, default:None) –A per-request price in US dollars by model id, looked up under the requested id first and then the answering model's; it wins over the table.
Rationale: ADR-0019.
llm_ports.adapters.litellm_proxy¶
What a LiteLLM proxy in front of the model says about a call, read from its response.
A LiteLLM proxy knows things the response body does not: which deployment served an alias
(x-litellm-model-id, x-litellm-model-api-base), what the call cost at that deployment's price
(x-litellm-response-cost), and how many retries and fallbacks it spent
(x-litellm-attempted-retries, x-litellm-attempted-fallbacks). It also refuses a call in two
ways that look alike on the wire: a 429 because its parallel-request limit is full (comes back in
seconds), and a 429 because a budget or an upstream quota is spent (does not).
The module satisfies ports.GatewayReader (pass the module itself as gateway=), and
classify_gateway is an extra classifier for the generator. It imports no LiteLLM and no HTTP
client: it reads header mappings and WireFacts.
Invariants
- Pure: no I/O.
- A cost header that is present but not a plausible dollar amount (a finite number, 0 or
between
COST_MIN_USDandCOST_MAX_USD) raisesValueErrorfromcost_from_headers; the generator treats it as absent and reports it. classify_gatewaynames onlyquota; a plain 429 is left to the shared table (rate_limited, retryable, with the server'sRetry-After).
Example
gen = PydanticAIGenerator(model_for=openai_compatible(url, key), gateway=litellm_proxy, extra=(litellm_proxy.classify_gateway,))
Rationale: ADR-0019, ADR-0020.
cost_from_headers
¶
cost_from_headers(
headers: Mapping[str, str],
) -> Decimal | None
Return the cost the proxy reported (x-litellm-response-cost), or None when it reported none.
Pure.
A reported zero (0, -0, 0.000) is Decimal(0).
Raises:
-
ValueError–The header is present but not a finite number of dollars that is 0 or between
COST_MIN_USDandCOST_MAX_USD(exclusive of the maximum).
answered_from_headers
¶
answered_from_headers(
headers: Mapping[str, str],
) -> tuple[str | None, str | None]
Return (deployment id, upstream api base) as the proxy reported them; None for each it did not. Pure.
attempts_from_headers
¶
attempts_from_headers(
headers: Mapping[str, str],
) -> tuple[int | None, int | None]
Return (retries, fallbacks) the proxy spent on the call; None for each it did not report. Pure.
retry_hint_s
¶
retry_hint_s(text: str) -> float | None
Return a provider's "Please retry in 8h26m23.4s" hint in seconds; None when the text has none. Pure.
classify_gateway
¶
Name a 429 that no wait will clear as quota; leave everything else to the shared table.
A 429 is quota when its error names a budget (the proxy's budget for a key, team or
provider), a per-day limit, or asks for a retry longer than QUOTA_HINT_MAX_S (an upstream
provider's spent quota). A 429 from the proxy's parallel-request limit is left alone: the
shared table makes it rate_limited, retryable, with the server's Retry-After. Pure.
Rationale: ADR-0020.
llm_ports.adapters.gepa¶
Tune a prompt's text components with GEPA, reflective prompt evolution (extra [gepa]).
optimize(seed, train, val, evaluate=..., reflect=...) runs the gepa core engine over a
candidate: a mapping of named text components (an instruction block, a system prompt) to their
text. The caller owns everything domain-specific through two callables:
evaluate(candidate, items)runs the program with the candidate's texts on each item and returns oneRolloutper item: the output, a score in [0, 1] (higher is better) and the feedback a reflection model should read (grader reasons, validation problems).reflect(prompt)is the reflection model: prompt text in, completion text out.
GEPA evaluates candidates on minibatches of train, asks reflect to rewrite one component from
the rollouts' feedback, keeps a rewrite only when it beats its parent on the same minibatch, and
tracks every kept candidate's mean score on val. Optimized reports the best candidate by that
mean next to the seed's own, so a caller promotes only when improved is true and val was held
out from anything the caller tuned by hand.
Invariants
- Never raises: too few items is
Insufficient; anevaluatethat raises, a rollout count that does not match its batch, a GEPA engine error, areflectthat failed on every call, or one that failedmax_reflection_failurestimes in a row isUnavailable. - A reflection failure never passes silently: GEPA itself logs a failed reflection and moves
on, so
optimizecounts everyreflectcall and failure and reports both onOptimized. - GEPA's own log lines are kept (
Optimized.log_tail,Unavailable.log_tail), never printed. - A non-finite score is scored 0.0 and its feedback says so; a score outside [0, 1] is clamped.
- The seed candidate is never mutated;
Optimized.candidateis a fresh dict. max_metric_callsbounds the item evaluations GEPA spends (its own budget accounting).- The search always ends without the budget: GEPA swallows a failed reflection and an
evaluator that caches spends no budget, so a dead reflection model would otherwise spin.
max_reflection_failuresconsecutive failures stop it asUnavailable;timeout_s, when set, stops it on wall-clock time andOptimized.stoppedsays so.
Rationale: ADR-0017.
Rollout
dataclass
¶
Rollout(output: O, score: float, feedback: str = '')
One item run with one candidate, and how it scored.
Attributes:
-
output(O) –What the program produced (opaque to GEPA; shown to the reflection model).
-
score(float) –In [0, 1], higher is better. GEPA sums it on minibatches and averages it on
val. -
feedback(str) –Why the item scored as it did, in words the reflection model can act on.
Optimized
dataclass
¶
Optimized(
candidate: dict[str, str],
seed_score: float,
best_score: float,
improved: bool,
candidates: int,
metric_calls: int,
n_train: int,
n_val: int,
reflections: int,
reflection_failures: int,
log_tail: tuple[str, ...] = (),
stopped: str = "budget",
)
The best candidate GEPA found, scored on the validation items.
Attributes:
-
candidate(dict[str, str]) –component -> textof the best candidate by mean validation score. -
seed_score(float) –The seed candidate's mean validation score.
-
best_score(float) –The best candidate's mean validation score.
-
improved(bool) –best_score > seed_score; a caller promotes only when this is true. -
candidates(int) –Candidates GEPA kept (the seed included).
-
metric_calls(int) –Item evaluations spent.
-
n_train(int) –Train items given.
-
n_val(int) –Validation items given.
-
reflections(int) –reflectcalls made. -
reflection_failures(int) –reflectcalls that raised (GEPA skipped those steps). -
log_tail(tuple[str, ...]) –GEPA's last log lines.
-
stopped(str) –Why the search ended:
"budget"(max_metric_callsspent) or"timeout".
Unavailable
dataclass
¶
Unavailable(reason: str, log_tail: tuple[str, ...] = ())
The search could not run or finish (an evaluator, reflection or engine failure).
Attributes:
-
reason(str) –The exception's type and message, or why no reflection succeeded.
-
log_tail(tuple[str, ...]) –GEPA's last log lines.
RolloutAdapter
¶
RolloutAdapter(
evaluate: Callable[
[Mapping[str, str], Sequence[I]],
Sequence[Rollout[O]],
],
describe: Callable[[I], str] = str,
)
A GEPA adapter over a caller's batch evaluator.
Implements GEPA's GEPAAdapter protocol: evaluate runs the caller's evaluator and returns an
EvaluationBatch; make_reflective_dataset turns each rollout into the record GEPA's
default proposer shows the reflection model (Inputs, Generated Outputs, Feedback).
Parameters:
-
evaluate(Callable[[Mapping[str, str], Sequence[I]], Sequence[Rollout[O]]]) –(candidate, items) -> one Rollout per item. -
describe(Callable[[I], str], default:str) –Renders an item for the reflection model;
strby default.
Raises:
-
ValueError–From
evaluate, when the evaluator returns a rollout count that does not match its batch (GEPA treats it as a systemic failure;optimizenames it).
evaluate
¶
evaluate(
batch: list[I],
candidate: dict[str, str],
capture_traces: bool = False,
) -> EvaluationBatch[dict[str, Any], O]
Run candidate on batch through the caller's evaluator.
make_reflective_dataset
¶
make_reflective_dataset(
candidate: dict[str, str],
eval_batch: EvaluationBatch[dict[str, Any], O],
components_to_update: list[str],
) -> dict[str, list[dict[str, Any]]]
Return the per-component records the reflection model reads.
optimize
¶
optimize(
seed: Mapping[str, str],
train: Sequence[I],
val: Sequence[I],
*,
evaluate: Callable[
[Mapping[str, str], Sequence[I]],
Sequence[Rollout[O]],
],
reflect: Callable[[str], str],
describe: Callable[[I], str] = str,
max_metric_calls: int = DEFAULT_METRIC_CALLS,
minibatch_size: int = 3,
seed_value: int = 0,
run_dir: str | None = None,
min_train: int = MIN_TRAIN,
min_val: int = MIN_VAL,
max_reflection_failures: int = MAX_REFLECTION_FAILURES,
timeout_s: float | None = None
) -> Optimized | Insufficient | Unavailable
Evolve seed's text components with GEPA and return the best by validation score.
Parameters:
-
seed(Mapping[str, str]) –component -> textto start from; every component may be rewritten. -
train(Sequence[I]) –Items GEPA's minibatches and reflections draw from.
-
val(Sequence[I]) –Held-out items every kept candidate is scored on; disjoint from
train. -
evaluate(Callable[[Mapping[str, str], Sequence[I]], Sequence[Rollout[O]]]) –(candidate, items) -> one Rollout per item; score per-item failures 0.0 with feedback rather than raising (a raise ends the search asUnavailable). -
reflect(Callable[[str], str]) –The reflection model, prompt text in, completion text out.
-
describe(Callable[[I], str], default:str) –Renders an item for the reflection model.
-
max_metric_calls(int, default:DEFAULT_METRIC_CALLS) –The item-evaluation budget.
-
minibatch_size(int, default:3) –Train items per reflection step.
-
seed_value(int, default:0) –GEPA's random seed (minibatch sampling, component order).
-
run_dir(str | None, default:None) –A directory GEPA checkpoints to, so a stopped run resumes; None keeps no state.
-
min_train(int, default:MIN_TRAIN) –Train items below which nothing is searched.
-
min_val(int, default:MIN_VAL) –Validation items below which nothing is searched.
-
max_reflection_failures(int, default:MAX_REFLECTION_FAILURES) –Consecutive failed
reflectcalls that end the search asUnavailable(a reflection model that is down or out of quota). -
timeout_s(float | None, default:None) –Wall-clock seconds after which the search stops and reports what it has (
Optimized.stopped == "timeout"); None sets no time bound.
Returns:
-
Optimized | Insufficient | Unavailable–Optimized;Insufficientwhentrainorvalis under its floor orseedhas no -
Optimized | Insufficient | Unavailable–component;
Unavailablewhen the search failed or the reflection model kept failing. -
Optimized | Insufficient | Unavailable–Never raises.
llm_ports.adapters.http¶
Adapters over HTTP (httpx): a reranker, a grounder client, a health probe, a token counter, and an index-ordered transport.
IndexOrderedTransport
¶
IndexOrderedTransport(
inner: AsyncBaseTransport | None = None,
)
Bases: AsyncBaseTransport
An httpx transport that re-sorts a 200 reply's index-keyed list (INDEXED_LISTS) by index.
Parameters:
-
inner(AsyncBaseTransport | None, default:None) –The transport that sends the request; None uses httpx's own.
Invariants
- A non-200 reply, a path not in
INDEXED_LISTS, an unreadable body, or a list already in index order is returned unchanged.
GrounderUnavailable
¶
Bases: Exception
The sidecar did not answer with a verdict: a transport error, a non-200 or a malformed reply.
HttpGrounder
¶
HttpGrounder(
url: str,
*,
timeout_s: float = 20.0,
client: Client | None = None
)
A Grounder whose model is behind {url}/verdict.
Parameters:
-
url(str) –The sidecar's base URL; a trailing slash is dropped.
-
timeout_s(float, default:20.0) –The httpx timeout, in seconds, of the client this grounder builds.
-
client(Client | None, default:None) –A ready httpx client, used instead of building one from
timeout_s.
Attributes:
-
checker(str) –"http" until the first verdict, then the sidecar's own checker name.
Invariants
- Every verdict is the sidecar's, the no-spans rule included (its own
Grounderenforces it): one judge, onecheckername on every verdict. - A dead sidecar raises for a no-spans claim rather than answering for it.
verdict
¶
verdict(
claim: Claim, spans: Sequence[Chunk]
) -> GroundingVerdict
Return the sidecar's verdict on claim against spans.
Raises:
-
GrounderUnavailable–A transport error, a non-200, or a malformed reply.
HttpProbe
¶
HttpProbe(
url: str,
*,
secret: str | None = None,
deadline_s: float = PROBE_DEADLINE_S,
transport: AsyncBaseTransport | None = None,
clock: Callable[[], float] = monotonic
)
A Probe that GETs one URL.
Parameters:
-
url(str) –The URL to ask, e.g.
http://localhost:4000/v1/models. -
secret(str | None, default:None) –Sent as a bearer token; None sends no
Authorizationheader. -
deadline_s(float, default:PROBE_DEADLINE_S) –The whole probe's deadline, connection included.
-
transport(AsyncBaseTransport | None, default:None) –Replaces httpx's own transport (a test's
MockTransport). -
clock(Callable[[], float], default:monotonic) –Monotonic seconds; injectable for tests.
probe
async
¶
probe() -> Health
Ask the URL once. Never raises.
Returns:
-
Health–upwith the listed model ids (empty when the list is empty);downwhen the backend -
Health–could not be reached, timed out or answered with an error status;
unknownwhen the -
Health–probe failed or could not send its request, the credential was refused (reason
-
Health–auth: ...), the backend has no endpoint to judge it by, or a 200 body is not a model -
Health–list. The reason names an exception by its type only.
HttpReranker
¶
HttpReranker(
url: str,
model: str,
*,
api_key: str | None = None,
timeout_s: float = DEFAULT_TIMEOUT_S,
transport: AsyncBaseTransport | None = None,
policy: WirePolicy | None = None,
on_event: EventHook | None = None
)
A Reranker over one /rerank endpoint.
Each call opens its own httpx client, and keeps it across the call's retries.
Parameters:
-
url(str) –The full
/rerankendpoint URL. -
model(str) –The model name sent on every request and reported on every result.
-
api_key(str | None, default:None) –Sent as a bearer token when set.
-
timeout_s(float, default:DEFAULT_TIMEOUT_S) –The per-call httpx timeout, in seconds.
-
transport(AsyncBaseTransport | None, default:None) –Replaces httpx's own transport (a consumer's counting transport, or
httpx.MockTransportin a test). It always sits underIndexOrderedTransport. -
policy(WirePolicy | None, default:None) –Which failed requests are re-sent, and how; None sends each request once.
-
on_event(EventHook | None, default:None) –Receives
llm_wire_retry; without one it goes to the stdlib loggerllm_ports.adapters.http.
Raises:
-
ValueError–At construction, when
urlormodelis empty.
HttpTokenCounter
¶
HttpTokenCounter(
url: str,
*,
api_key: str | None = None,
key: str = "total_tokens",
models: Collection[str] | None = None,
timeout_s: float = COUNT_TIMEOUT_S,
transport: AsyncBaseTransport | None = None
)
A TokenCounter over one count endpoint.
Parameters:
-
url(str) –The full count URL, e.g.
http://gateway:4000/utils/token_counter. -
api_key(str | None, default:None) –Sent as a bearer token when set.
-
key(str, default:'total_tokens') –The reply's integer key holding the count.
-
models(Collection[str] | None, default:None) –The model ids this counter may count; others are
unsupportedwith no request. None counts every model. -
timeout_s(float, default:COUNT_TIMEOUT_S) –The per-count timeout, in seconds.
-
transport(AsyncBaseTransport | None, default:None) –Replaces httpx's own transport (a test's
MockTransport).
Raises:
-
ValueError–At construction, when
urlis empty.
llm_ports.adapters.langfuse.tracer¶
The Tracer port over the Langfuse Python SDK 5 (extra [langfuse]).
Example
tracer = LangfuseTracer(Langfuse(...)) # the caller builds the client async with observe(tracer, name="summarize", model=m, provider="openai", input=prompt) as obs: ...
This adapter owns the generation lifecycle, the usage block, end, score and the managed-prompt
link. The caller owns building the client (keys, host, its OpenTelemetry provider and export
filter) and what goes into the calls (names, metadata).
Langfuse SDK 5 is OpenTelemetry underneath:
- A generation is
start_observation(as_type="generation"), opened as a root in the current OTel trace (trace_context={"trace_id": ...}), so the Langfuse trace id is the id every other span of that request carries. Outside a span a fresh trace id is minted. - Trace-level fields (session, trace name, tags) ride
propagate_attributesaround the open.nameis stored verbatim as the generation name, the trace name and the one tag, so a cost dashboard can group by it. - Usage is
usage_detailswith the measured sides of{"input", "output"}; the server addstotaland prices by model. - Export is a background batch:
enddoes not block on the network.
provider has no Langfuse field; it rides metadata["provider"] unless the caller set that key.
Invariants
- An unreported token count is never sent as 0: only the measured sides go, and with neither measured no usage block is sent.
- Fails open: a client that raises on open yields a
NoopGeneration. - Every failure (open, end, score) emits at most one warning event per
warn_hold_sthroughon_event(default: the stdlib loggerllm_ports.adapters.langfuse.tracer), and never raises into the call it describes. - The prompt link is best effort and silent.
Rationale: ADR-0007.
LangfuseGeneration
¶
LangfuseGeneration(
gen: Any, warn: Callable[[str, Exception], None]
)
Generation over an SDK 5 LangfuseGeneration.
end is the one terminal call: its fields go in a single update, then end() closes the
span. Failures go to warn, never raised.
end
¶
end(**fields: Unpack[EndFields]) -> None
Close the generation with its output, usage, cost and metadata; never raises.
Usage with neither count reported sends no usage block (and an unreported side is
omitted, never 0). cost_details is sent only when the call's cost is known in dollars;
otherwise Langfuse's own model pricing may apply, and metadata.cost_source says which.
level="ERROR" sends ERROR and a cancelled call ("CANCELLED") sends WARNING, each with
status_message. finish, answered and failure already reach the sink as metadata
on the library's own call path; they are accepted here for the port's signature.
score
¶
score(
name: str,
value: float,
comment: str | None = None,
metadata: Mapping[str, Any] | None = None,
) -> None
Attach a numeric score to the generation, with its metadata when given; never raises.
LangfuseTracer
¶
LangfuseTracer(
client: Any,
*,
prompt_label: str = DEFAULT_PROMPT_LABEL,
prompt_cache_ttl_s: int = DEFAULT_PROMPT_CACHE_TTL_S,
warn_hold_s: float = DEFAULT_WARN_HOLD_S,
clock: Callable[[], float] = monotonic,
on_event: EventHook | None = None
)
Tracer whose sink is Langfuse.
Build one per client; the SDK client is per process.
Parameters:
-
client(Any) –A Langfuse SDK 5 client, built by the caller.
-
prompt_label(str, default:DEFAULT_PROMPT_LABEL) –The managed-prompt label a generation links to.
-
prompt_cache_ttl_s(int, default:DEFAULT_PROMPT_CACHE_TTL_S) –The SDK's cache TTL for that prompt fetch.
-
warn_hold_s(float, default:DEFAULT_WARN_HOLD_S) –The minimum seconds between two warning events.
-
clock(Callable[[], float], default:monotonic) –A monotonic clock, injectable for tests.
-
on_event(EventHook | None, default:None) –The warning sink; defaults to the stdlib logger
llm_ports.adapters.langfuse.tracer.
start_generation
¶
start_generation(
*,
name: str,
model: str,
provider: str,
input: Any,
metadata: Mapping[str, Any],
session_id: str | None,
prompt_name: str | None
) -> Generation
Open a generation as a root span in the current OTel trace; never raises.
With prompt_name, the generation links to the prompt's labelled version (a cached fetch),
so the prompt's generation count follows the live version across re-syncs. Any failure to
open returns a NoopGeneration.
usage_details
¶
usage_details(
prompt: int | None, completion: int | None
) -> dict[str, int] | None
Return the Langfuse usage block, or None when the provider reported neither count.
When only one count is reported, only that side is sent: an unreported side is unknown, and booking it as 0 would make a half-measured call look cheaper than it was. Pure.
Rationale: ADR-0007, ADR-0019.
trace_id
¶
trace_id(client: Any) -> str
Return the current OTel trace id as 32 hex digits, or a fresh one from the client outside a span.
llm_ports.adapters.langfuse.prompt_source¶
The PromptSource port over Langfuse prompt management.
Example
source = LangfusePromptSource(Langfuse(...), seed_for) prompt = source.get("summarize") # Prompt(text, version, source="langfuse:v7" | "repo")
The repository seed leads; the managed label mirrors it and never leads. Per get(name, label):
get_prompt(name, label, fallback=<seed>, cache_ttl_seconds=ttl)goes through the SDK's own cache (an expired entry is served and refreshed on a background thread).- If the reply is the fallback, or its
config.content_shais not the seed's sha, or the seed's config carries anidentitythe reply's config does not, the managed copy is absent or stale: the seed is pushed under the label withcreate_prompt(...)and refetched uncached. - If the reply is still the fallback, or its text is not the seed's, the seed is served as
Prompt(source="repo"). Otherwise the result isPrompt(source="langfuse:vN")with the managed version and its stored config.
seed_for(name) -> Seed is injected: the adapter knows nothing about where a repository prompt
lives or how it is fingerprinted (prompts.Pack.seed builds one). With label_from_seed=True, a
seed whose config names a label (a pack's manifest label) is served and mirrored under that
label instead of the label argument. A seed with a non-empty system (or kind="chat") is a chat
prompt: pushed as type="chat" with the message list, fetched as chat, and compared message by
message on role and content. Langfuse refuses a type change on an existing name, so kind can
force chat for a seed with no system message.
The client is duck-typed: anything with the SDK's get_prompt / create_prompt signatures. The
caller builds it, so this module imports no SDK.
Invariants
- Fails soft: an unreachable store, a refused push or a malformed reply each serves the seed
as
source == "repo"and emits a warning event throughon_event(default: the stdlib loggerllm_ports.adapters.langfuse.prompt_source); none of them raises. - The SDK never caches a fallback reply, so the down verdict is held per name for one TTL here: a store that is down costs one round-trip per TTL, not one per call.
Rationale: ADR-0008.
LangfusePromptSource
¶
LangfusePromptSource(
client: Any,
seed_for: Callable[[str], Seed],
*,
cache_ttl_s: int = DEFAULT_CACHE_TTL_S,
clock: Callable[[], float] = monotonic,
on_event: EventHook | None = None,
label_from_seed: bool = False
)
PromptSource whose managed store is Langfuse.
One instance per client; the SDK caches.
Parameters:
-
client(Any) –A Langfuse client, or anything with its
get_prompt/create_promptsignatures. -
seed_for(Callable[[str], Seed]) –Returns the repository seed for a prompt name.
-
cache_ttl_s(int, default:DEFAULT_CACHE_TTL_S) –The SDK cache TTL, and how long a down verdict is held per name.
-
clock(Callable[[], float], default:monotonic) –A monotonic clock, injectable for tests.
-
on_event(EventHook | None, default:None) –The warning sink; defaults to the stdlib logger
llm_ports.adapters.langfuse.prompt_source. -
label_from_seed(bool, default:False) –Opt in to the seed's own label: a seed whose
config["label"]is set is served and mirrored under it, whateverlabelgetis given. Off by default.
get
¶
get(name: str, *, label: str = 'production') -> Prompt
Return prompt name, mirrored to the managed store under label.
Serves the managed version when it matches the seed, else the seed as source == "repo";
the module docstring gives the steps. A store failure never raises. With
label_from_seed, a seed's config["label"] replaces label.
llm_ports.adapters.langfuse.experiments¶
Record a graded evaluation as a Langfuse dataset experiment (extra [langfuse]).
seed writes a golden set into a dataset, idempotently: each item's id is the caller's or a hash
of its input, and Langfuse upserts an item by id, so a re-seed rewrites rather than duplicates.
run drives the SDK's own experiment runner (Langfuse.run_experiment): it opens one trace per
dataset item, links the trace to the item, and books each grade as a score on it. record_rows
records rows already graded offline (a golden-set run that graded a recording) as one run on a
dataset: the task hands back the row's public output plus its id, and the evaluator looks the row
up by that id, because an SDK evaluator sees input, output, expected output and metadata, never the
item.
The SDK logs a task or evaluator that raised and drops or leaves that item ungraded, so a run that
only counted what came back would report a clean ratio over a shrunken denominator. Here a run that
graded fewer items than it was handed is an invalid outcome naming the shortfall.
Example
run = record_rows(client, dataset="summary_golden_v1", run_name="pack-1a2b", rows=rows,
output=lambda r: {"text": r["text"]}, grade=lambda r: [("echo", r["echo"], "NUMERIC")])
if not run.ok:
... # the run did not finish looking; run.reason says why
Invariants
- Never raises for a server or SDK failure: the failure is the outcome.
max_concurrencydefaults to 1: an offline grade drives one model or CLI at a time.
Rationale: ADR-0001.
ExperimentRun
dataclass
¶
ExperimentRun(
outcome: Outcome,
handed: int,
graded: int = 0,
reason: str | None = None,
result: Any = None,
)
What run and record_rows return.
Attributes:
-
outcome(Outcome) –okwhen every item was graded;invalidwhen the SDK dropped or left an item ungraded;unavailablewhen the run itself failed. -
handed(int) –Items the run was handed.
-
graded(int) –Items that came back with at least one grade.
-
reason(str | None) –One line naming the failure; None on
ok. -
result(Any) –The SDK's
ExperimentResult(its run URL, item results); None when the run failed.
Seeded
dataclass
¶
Seeded(
outcome: Outcome,
handed: int,
written: int = 0,
ids: tuple[str, ...] = (),
reason: str | None = None,
)
What seed returns.
Attributes:
-
outcome(Outcome) –okwhen every item was written;invalidwhen some writes failed;unavailablewhen the dataset could be neither created nor read. -
handed(int) –Items
seedwas handed. -
written(int) –Items the store accepted.
-
ids(tuple[str, ...]) –The id of every item written, in input order.
-
reason(str | None) –One line naming the failure; None on
ok.
run
¶
run(
client: Any,
*,
run_name: str,
items: Sequence[Any],
task: Callable[..., Any],
grade: Callable[..., Iterable[Grade]],
metadata: Mapping[str, str] | None = None,
max_concurrency: int = 1
) -> ExperimentRun
Run task over dataset items and book grade's scores on each item's trace.
Parameters:
-
client(Any) –A Langfuse SDK 5 client, built by the caller.
-
run_name(str) –The experiment and run name.
-
items(Sequence[Any]) –Dataset items (
client.get_dataset(name).items), so each trace links to its item. -
task(Callable[..., Any]) –The SDK task: called with
item=(and other keywords), returns the item's output. -
grade(Callable[..., Iterable[Grade]]) –Called with the SDK evaluator's keywords (
output=,expected_output=, ...); returns the item'sGrades. -
metadata(Mapping[str, str] | None, default:None) –Run metadata.
-
max_concurrency(int, default:1) –Items in flight at once.
Returns:
-
ExperimentRun–The run, whose outcome is
okonly when every handed item was graded. Never raises.
record_rows
¶
record_rows(
client: Any,
*,
dataset: str,
run_name: str,
rows: Sequence[Mapping[str, Any]],
output: Callable[
[Mapping[str, Any]], Mapping[str, Any]
],
grade: Callable[[Mapping[str, Any]], Iterable[Grade]],
metadata: Mapping[str, str] | None = None,
id_field: str = "id"
) -> ExperimentRun
Record rows graded offline as one run on dataset.
Only items whose id matches a row's id_field are run; an item with no row is skipped, not
counted.
Parameters:
-
client(Any) –A Langfuse SDK 5 client.
-
dataset(str) –The dataset's name.
-
run_name(str) –The run name.
-
rows(Sequence[Mapping[str, Any]]) –The graded rows, each carrying
id_field. -
output(Callable[[Mapping[str, Any]], Mapping[str, Any]]) –A row to the public output recorded on its trace.
-
grade(Callable[[Mapping[str, Any]], Iterable[Grade]]) –A row to its
Grades. -
metadata(Mapping[str, str] | None, default:None) –Run metadata.
-
id_field(str, default:'id') –The row key that matches a dataset item's id.
Returns:
-
ExperimentRun–The run (see
run);unavailablewhen the dataset could not be read. Never raises.
input_id
¶
input_id(value: Any) -> str
A stable id for a dataset item: the first 32 hex digits of the SHA-256 of its canonical JSON.
Parameters:
-
value(Any) –The item's input; any JSON-serialisable value (a non-JSON leaf is stringified).
Returns:
-
str–The id; equal inputs (key order aside) give equal ids. Never raises.
seed
¶
seed(
client: Any,
*,
dataset: str,
items: Sequence[Mapping[str, Any]],
input: Callable[[Mapping[str, Any]], Any],
expected: (
Callable[[Mapping[str, Any]], Any] | None
) = None,
metadata: (
Callable[[Mapping[str, Any]], Any] | None
) = None,
item_id: (
Callable[[Mapping[str, Any]], str] | None
) = None,
description: str | None = None
) -> Seeded
Write items into dataset, creating the dataset when missing; a re-seed upserts.
Parameters:
-
client(Any) –A Langfuse SDK 5 client.
-
dataset(str) –The dataset's name.
-
items(Sequence[Mapping[str, Any]]) –The golden rows, in any shape the projections read.
-
input(Callable[[Mapping[str, Any]], Any]) –A row to the item's input.
-
expected(Callable[[Mapping[str, Any]], Any] | None, default:None) –A row to the item's expected output; omitted, none is written.
-
metadata(Callable[[Mapping[str, Any]], Any] | None, default:None) –A row to the item's metadata; omitted, none is written.
-
item_id(Callable[[Mapping[str, Any]], str] | None, default:None) –A row to the item's id; omitted,
input_idof the item's input, so the same input is the same item across seeds. -
description(str | None, default:None) –The dataset's description, written on creation.
Returns:
-
Seeded–okwith every id when every item was written;invalidnaming the shortfall when some -
Seeded–writes failed;
unavailablewhen the dataset could be neither created nor read. Never -
Seeded–raises.
Invariants
- Idempotent: seeding the same rows twice leaves one item per id.
- A failed write never stops the rest; the outcome counts it.
llm_ports.adapters.langfuse.labels¶
Labelled model replies as Langfuse scores, with a Langfuse annotation queue per role (extra [langfuse]).
LangfuseExampleStore is a learn.WritableExampleStore and an evals.labels.LabelQueue for
topics llm:<role> (evals.labels.topic). An example is a score on the labelled generation,
named by source and role:
label-<role>: a HUMAN label against a categorical score config of the same name (good/bad), given in Langfuse's annotation-queue UI or relayed byputfrom another surface that names the person (Example.by). A relayed label completes that reply's queue item, as the annotation UI does.label.model-<role>: a stronger model's label, written byput.label.judge-<role>: the production judge's verdict as a label, written byput.
The separator is - because Langfuse refuses a score-config name outside CONFIG_NAME (no :);
reads also accept the earlier label:<role> spelling, so scores written under it stay readable.
Every role gets one score config and one annotation queue, both named label-<role>, created on
first enqueue. enqueue skips a reply the queue already holds (pending or done), and queued
reads what it holds, so a nightly draw neither duplicates items nor grows the backlog unbounded.
put writes each score under a deterministic id (source, role, subject, and the person for a
human label), so a relabel replaces the earlier score instead of adding a second one. The SDK is
synchronous; each call runs in a worker thread.
Example
store = LangfuseExampleStore(Langfuse(...)) # the caller builds the client await store.enqueue(topic("summarize"), [(observation_id, trace_id)]) read = await store.examples([topic("summarize")])
Invariants
- Never raises for a server or SDK failure: the failure is the outcome.
- A score this adapter cannot read as an example (another name, an unknown value, no observation subject) is skipped, never guessed.
- A topic that is not
llm:<role>holds no examples here and is refused byput/enqueue. - A role whose label name Langfuse would refuse (
CONFIG_NAME) is refused here as arequest_faultbefore any write, never sent to fail remotely. - A human label is written only with the person who gave it (
by); anoutcomelabel never. - A failure reason carries the server's own validation message when it sends one.
Rationale: ADR-0010, ADR-0011, ADR-0014.
LangfuseExampleStore
¶
LangfuseExampleStore(
client: Any,
*,
description: str = "Is this reply good? (labelled data)"
)
Labelled model replies stored as Langfuse scores, queued for humans in annotation queues.
Parameters:
-
client(Any) –A Langfuse SDK 5 client, built by the caller.
-
description(str, default:'Is this reply good? (labelled data)') –The description given to each created score config and queue.
enqueue
async
¶
enqueue(
topic: str, subjects: Sequence[tuple[str, str]]
) -> Written
Add each (observation id, trace id) the role's annotation queue does not hold yet.
Returns:
-
Written–How many were newly queued, or a failure outcome. Never raises.
queued
async
¶
queued(topic: str) -> Queued
Read the observation ids the role's annotation queue holds, pending and done. Never raises.
A role with no queue yet reads ok and empty; nothing is created by reading.
put
async
¶
Write labels as categorical scores. Never raises.
A human label is one a person gave on another surface and code relays: it is written
under label-<role> against the role's score config (created when missing), keyed by the
person as well, and the role's queue item for that reply is then marked completed.
Returns:
-
Written–How many scores landed;
request_fault(none written) when any example is an -
Written–outcomelabel, ahumanlabel with noby, not on anllm:<role>topic, not -
Written–good/bad, or has noref["trace"].unavailablewhen the server failed; a -
Written–failure while completing a queue item comes after every score landed, so
written -
Written–then counts them all.
examples
async
¶
examples(
topics: Sequence[str],
*,
since: datetime | None = None,
source: Source | None = None
) -> ExamplesRead
Read every source's labels on the replies of topics. Never raises.
A topic that is not llm:<role> contributes nothing; an outcome source reads empty.
score_name
¶
score_name(source: Labeller, role: str) -> str
Return the score name a source's label on a role reply is stored under.
parse_name
¶
parse_name(name: str) -> tuple[Labeller, str] | None
Return (source, role) for a label score name in either spelling, or None for any other score.
llm_ports.adapters.optuna¶
Tune a combiner's hyperparameters with Optuna, walk-forward (extra [optuna]).
tune(build, events, space) splits the resolved events in time (forecast.walk.holdout_start):
the tuner only ever sees the TRAIN events, those forecast before the holdout starts. Each trial
builds a fresh combiner from its parameters (build(params)) and replays it over the train events
with walk_forward; its objective is the mean Brier score over the resolved train events, with
an abstention scored as 0.5 (Brier 0.25), so a configuration cannot win by answering less. The
best parameters are then replayed over EVERY event, learning through the holdout as they would
live, and scored on the holdout only: Tuned.holdout is out of sample.
The study goes to Optuna's storage (an RDB URL, an Optuna storage object, or None for memory)
under study_name; a reused name adds trials to that study. The TPE sampler is seeded.
SqlJournal(engine, table) is an Optuna journal backend on one SQL table the caller owns
(journal_storage(...) wraps it in Optuna's JournalStorage). Optuna's own RDB storage creates
its tables wherever the connection's search path points and checks its schema through Alembic,
which reads the default schema; behind a transaction-mode pooler (PgBouncer) a session-level
search path does not hold, so the RDB storage cannot be pinned to one schema there. A journal is
one append-only table: the caller creates it (a migration; create=True for tests and SQLite),
names it schema-qualified, and every statement carries the name.
dashboard_app(storage) / serve_dashboard(storage) put optuna-dashboard over any of those
storages, a SqlJournal included, on demand (extra [optuna-dashboard]): the stock dashboard
reads only an RDB URL or a journal file, so a journal table needs this launcher.
Invariants
- Never raises: too few events is
Insufficient; a storage or Optuna failure isUnavailable; a trial whose build or replay raises is pruned as failed, not fatal. - The objective never reads a holdout event.
Rationale: ADR-0015.
Range
dataclass
¶
Range(low: float, high: float, log: bool = False)
One hyperparameter's search range.
Attributes:
-
low(float) –The smallest value tried.
-
high(float) –The largest value tried.
-
log(bool) –Sample on a log scale (a learning rate);
lowmust then be > 0.
Tuned
dataclass
¶
Tuned(
params: dict[str, float],
study_name: str,
trials: int,
failed_trials: int,
holdout_start: datetime,
n_train: int,
train_brier: float,
holdout: Score,
)
The best parameters a study found, and how they scored out of sample.
Attributes:
-
params(dict[str, float]) –name -> valueof the best trial. -
study_name(str) –The Optuna study the trials went to.
-
trials(int) –Trials completed in this call.
-
failed_trials(int) –Trials whose build or replay raised in this call.
-
holdout_start(datetime) –The first holdout event's forecast time.
-
n_train(int) –Resolved train events the objective scored.
-
train_brier(float) –The best trial's objective (mean Brier over the train events).
-
holdout(Score) –The best parameters'
Scoreon the resolved holdout events they answered.
Unavailable
dataclass
¶
Unavailable(reason: str)
The study could not run (storage unreachable, every trial failed, an Optuna error).
Attributes:
-
reason(str) –The exception's type and message, or why no trial completed.
SqlJournal
¶
SqlJournal(
engine: Engine, table: str, *, create: bool = False
)
Bases: BaseJournalBackend
An Optuna journal backend on one SQL table (n bigint primary key, log text).
Log n is the journal's 0-based log number. An append reads the next number and inserts the
whole batch in one transaction; the primary key makes a concurrent append that read the same
number fail, and it retries (APPEND_RETRIES) from the new end, so the journal stays gapless
and in order across processes. Any SQLAlchemy dialect (PostgreSQL behind a pooler, SQLite).
Parameters:
-
engine(Engine) –A synchronous SQLAlchemy engine.
-
table(str) –The table,
nameorschema.name. -
create(bool, default:False) –Create the table if it is missing (tests, SQLite); a production table comes from the caller's migration.
Raises:
-
ValueError–tableis not one or two plain identifiers.
objective_brier
¶
objective_brier(
preds: Sequence[float | None], events: Sequence[Event]
) -> float | None
Return the mean Brier score over the resolved events, an abstention scored as ABSTAIN_P; None when none resolved.
tune
¶
tune(
build: Callable[[dict[str, float]], Combiner],
events: Sequence[Event],
space: Mapping[str, Range],
*,
study_name: str,
storage: Any = None,
n_trials: int = 40,
holdout: float = 0.2,
seed: int = 0,
min_train: int = MIN_TRAIN
) -> Tuned | Insufficient | Unavailable
Search space for the combiner parameters with the lowest walk-forward Brier score.
Parameters:
-
build(Callable[[dict[str, float]], Combiner]) –Makes a fresh combiner from one trial's
name -> value. -
events(Sequence[Event]) –Every event, resolved and open, in any order.
-
space(Mapping[str, Range]) –name -> Rangeto search. -
study_name(str) –The Optuna study name; trials join an existing study of that name.
-
storage(Any, default:None) –An Optuna storage (URL or object); None keeps the study in memory.
-
n_trials(int, default:40) –Trials to run in this call.
-
holdout(float, default:0.2) –The share of the resolved events held out, in (0, 1).
-
seed(int, default:0) –The TPE sampler's seed.
-
min_train(int, default:MIN_TRAIN) –Resolved train events below which nothing is searched.
Returns:
-
Tuned | Insufficient | Unavailable–Tuned;Insufficientwhen the train split has fewer thanmin_trainresolved events; -
Tuned | Insufficient | Unavailable–Unavailablewhen the study could not run. Never raises.
journal_storage
¶
journal_storage(
engine: Engine, table: str, *, create: bool = False
) -> Any
Optuna's JournalStorage over a SqlJournal; pass it to tune(storage=...).
dashboard_app
¶
dashboard_app(storage: Any) -> Any
Return optuna-dashboard's WSGI app over storage (extra [optuna-dashboard]).
storage is anything Optuna opens: a journal_storage(...), an RDB URL, an Optuna storage
object. The app reads the studies live; nothing is copied. Mount it under any WSGI server.
Raises:
-
ModuleNotFoundError–The
optuna-dashboardextra is not installed (the message names it).
serve_dashboard
¶
serve_dashboard(
storage: Any,
*,
host: str = "127.0.0.1",
port: int = 8080
) -> None
Serve optuna-dashboard over storage until interrupted (extra [optuna-dashboard]).
An on-demand study UI: run it while a person looks, stop it after. It binds host (loopback
by default; reach it through a tunnel) and blocks.
Raises:
-
ModuleNotFoundError–The
optuna-dashboardextra is not installed (the message names it).
llm_ports.adapters.otel¶
OpenTelemetry GenAI spans for model calls (extra [otel]).
OtelTracer is the Tracer port over OpenTelemetry: compose it like any other sink
(TeeTracer(LangfuseTracer(client), OtelTracer())) and every call the library makes is a span.
The two helpers below are for a call the library does not make.
Example
with gen_ai_span(tracer, operation="chat", model=m, provider="openai") as rec: reply = await client.call(...) rec.set_usage(reply.in_tok, reply.out_tok)
record_gen_ai(tracer, operation="embeddings", model=m, provider="openai", latency_ms=840)
A span records the shape of a model call (who, which model, how long, how many tokens) in the
trace the rest of the request already lives in, named and attributed by the OpenTelemetry GenAI
semantic conventions. Attribute keys come from opentelemetry-semantic-conventions' constants,
so an upstream rename arrives with a version bump.
tracer is any OTel Tracer, or None when tracing is off; every function then does nothing, so a
call site needs no conditional. attributes carries the caller's own dimensions under its own
prefix, outside the reserved gen_ai.* namespace.
gen_ai_spanopens a child of the ambient context (start_as_current_span), so the model call lands inside the existing job or request span. Only span creation is guarded: an exception from the caller's block propagates, and the SDK records it on the span.record_gen_airecords a call after it is over, for a call that learns its latency and usage only then. The span is back-dated by the measuredlatency_ms, so its duration is the call's.gen_ai.systemis no longer emitted (1.24): the conventions removed it in v1.37.0 forgen_ai.provider.name.SYSTEMstays exported for 1.x imports; nothing sets it.
Invariants
- Prompt and completion bodies are never carried, and no switch turns them on.
- None is not zero: a provider that reported neither count gets the
unmeasured_attrflag and no token attributes; a reported zero stays zero.
Rationale: ADR-0007, ADR-0025.
OtelGeneration
¶
OtelGeneration(
span: Any,
meters: _Meters | None,
base: Mapping[str, Any],
clock: Callable[[], float],
*,
tracer: Any | None = None
)
The Generation an OtelTracer opens; ends its span once and never raises.
end
¶
end(
*,
output: Any = None,
usage: Mapping[str, int | None] | None = None,
level: str | None = None,
status_message: str | None = None,
metadata: Mapping[str, Any] | None = None,
usage_v2: Usage | None = None,
cost: Cost | None = None,
finish: Finish | None = None,
answered: Answered | None = None,
failure: Failure | None = None
) -> None
Set the result's attributes, record the metrics, end the span; output is never carried.
tool
¶
tool(
*,
name: str,
call_id: str,
duration_s: float,
is_error: bool
) -> None
Write one tool execution as an INTERNAL child span execute_tool {name}; never raises.
score
¶
score(
name: str,
value: float,
comment: str | None = None,
metadata: Mapping[str, Any] | None = None,
) -> None
Do nothing: scores belong to the trace store (an ended span takes no new data).
OtelTracer
¶
OtelTracer(
tracer_provider: Any | None = None,
meter_provider: Any | None = None,
*,
semconv: Literal["pinned"] = "pinned",
clock: Callable[[], float] = monotonic
)
A Tracer that writes each generation as an OpenTelemetry span; the module docstring states the contract.
Parameters:
-
tracer_provider(Any | None, default:None) –The provider spans come from; None takes the global one.
-
meter_provider(Any | None, default:None) –When given, the duration histogram and token counters are recorded on it.
-
semconv(Literal['pinned'], default:'pinned') –The attribute-name set; only
"pinned"(observability.semconv) exists. -
clock(Callable[[], float], default:monotonic) –A monotonic clock for the duration metric, injectable for tests.
start_generation
¶
start_generation(
*,
name: str,
model: str,
provider: str,
input: Any,
metadata: Mapping[str, Any],
session_id: str | None,
prompt_name: str | None
) -> Generation
Open a CLIENT span for one call; input is never carried. A failure to open yields a no-op.
GenAiRecorder
¶
GenAiRecorder(
span: Any, *, unmeasured_attr: str = UNMEASURED
)
Attaches a call's results to its span once the call returns.
A no-op over span=None. Never raises.
set_usage
¶
set_usage(
input_tokens: int | None, output_tokens: int | None
) -> None
Set the token counts as reported; when neither is reported, set the unmeasured flag, never zeros.
span_name
¶
span_name(operation: str, model: str) -> str
Return the conventions' span name, {operation} {model}; the call's identity lives in attributes.
attributes_for
¶
attributes_for(
*,
operation: str,
model: str,
provider: str,
attributes: Mapping[str, Any] | None = None
) -> dict[str, Any]
Return the convention attributes plus the caller's own, with None values dropped. Pure.
gen_ai_span
¶
gen_ai_span(
tracer: Any | None,
*,
operation: str,
model: str,
provider: str,
attributes: Mapping[str, Any] | None = None,
unmeasured_attr: str = UNMEASURED
) -> Iterator[GenAiRecorder]
Open a child span around one live model call; the module docstring states the contract.
Yields:
-
GenAiRecorder–A
GenAiRecorderover the span, or a no-op recorder whentraceris None or span -
GenAiRecorder–creation failed.
record_gen_ai
¶
record_gen_ai(
tracer: Any | None,
*,
operation: str,
model: str,
provider: str,
latency_ms: int,
input_tokens: int | None = None,
output_tokens: int | None = None,
error: str | None = None,
attributes: Mapping[str, Any] | None = None,
unmeasured_attr: str = UNMEASURED
) -> None
Record a span for a call that already finished, back-dated by latency_ms; never raises.
A non-empty error sets the span status to ERROR, with the message cut to ERROR_MAX
characters; without one, the status is left unset.
llm_ports.adapters.pluggy¶
Plugins found by entry point and loaded with pluggy (extra [market]): action packs and provider kinds.
Two entry-point groups share one loader:
llm_ports.market(market.ENTRY_POINT_GROUP): action packs. A module declares its actions on its ownActionRegistryand returns it from amarket_actionshook;load_packsadds each pack to the scheduler's registry.-
llm_ports.providers(call.providers.ENTRY_POINT_GROUP): provider kinds. A module callsregister(factory)from aregister_providershook;load_providersreturns the kinds.PACK = ActionRegistry("reports")
@PACK.action(feature="report", action="weekly", cadence="7d", lane=3) async def weekly(session, work_key): ...
@hookimpl def market_actions() -> ActionRegistry: return PACK
pyproject.toml¶
[project.entry-points."llm_ports.market"] reports = "reports_pkg.market"
The process calls each loader once, at its composition root, passing the value of its pin setting
(PACKS_PIN_ENV, PROVIDERS_PIN_ENV by convention; the library reads no environment).
Invariants
- The hook specs are added before any plugin registers, so pluggy refuses a hook implementation whose arguments the spec does not declare.
- Built-ins register first, by explicit name, needing no distribution metadata; entry points follow, sorted by name. No order derives from a set or from install order.
- The pin: None loads the built-ins and every installed plugin;
""the built-ins only;"a,b"the built-ins and exactly those, and a pinned name that is not installed raises. - Strict loading (the default) raises
PluginLoadError(PackLoadErrorfor packs) on any failure, naming the group, the plugin and the stage:import,register(pluggy refused it, or it contributed a wrong object),hook(its hook raised),duplicate(a key or a name another plugin holds; both are named),pin,pending(an unknown hook name, or no hook).load_packs(strict=False)records each failure as aPackLoadinstead, for a diagnostic. - A registry is frozen after load: a strict
load_packsthat finishes freezes theActionRegistryit loaded into, andload_providersreturns itsProviderRegistryfrozen, so a registration after the composition root raises. - Every load emits
plugins_loaded(group, names, origins) once it finishes. market_pack_failedcarriesstagein thePackLoad.stagevocabulary (PackStage) andplugin_stagein thePluginLoadError.stagevocabulary (PluginStage).
Rationale: ADR-0018 (packs), ADR-0022 (providers).
Loaded
dataclass
¶
Loaded(
group: str,
plugins: tuple[str, ...],
origin: Mapping[str, str] = dict(),
)
Which plugins a process loaded into one registry, and from where.
Attributes:
-
group(str) –The entry-point group.
-
plugins(tuple[str, ...]) –Plugin names in load order: built-ins as given, then entry points sorted by name.
-
origin(Mapping[str, str]) –Plugin name to
builtin,caller, or<distribution>==<version>.
PluginLoadError
¶
PluginLoadError(
message: str,
*,
group: str,
plugin: str,
stage: PluginStage
)
Bases: ImportError
A plugin that did not load; raised at the composition root, so the process does not start.
Attributes:
-
group(str) –The entry-point group.
-
plugin(str) –The plugin's name (comma-joined for several pinned-but-missing names).
-
stage(PluginStage) –Where it failed.
PackLoadError
¶
PackLoadError(
message: str,
*,
group: str,
plugin: str,
stage: PluginStage
)
PackSpec
¶
The hooks a pack module may implement.
ProviderHooks
¶
The hooks a provider plugin may implement.
register_providers
¶
register_providers(
register: Callable[[ProviderFactory], None],
) -> None
Call register(factory) once per provider kind the plugin contributes.
PackLoad
dataclass
¶
PackLoad(
name: str,
dist: str | None,
outcome: PackOutcome,
actions: tuple[str, ...] = (),
stage: PackStage | None = None,
reason: str | None = None,
origin: str | None = None,
)
One pack, loaded or not.
Attributes:
-
name(str) –The entry point's name (the pack's name, and its pluggy plugin name).
-
dist(str | None) –The distribution that declares it, or None when unknown (and for a built-in).
-
outcome(PackOutcome) –ok,failedorblocked. -
actions(tuple[str, ...]) –The
feature.actionnames the pack added; empty unlessok. -
stage(PackStage | None) –Where a failed pack stopped; None unless
failed. -
reason(str | None) –Why it failed; None unless
failed. -
origin(str | None) –builtin,<dist>==<version>, or None when unknown.
load_packs
¶
load_packs(
registry: ActionRegistry,
*,
group: str = ENTRY_POINT_GROUP,
block: Iterable[str] = (),
entry_points: EntryPoints | None = None,
on_event: EventHook | None = None,
builtins: Sequence[tuple[str, object]] = (),
pin: str | None = None,
strict: bool = True
) -> tuple[PackLoad, ...]
Load the built-in packs, then every pack listed under group, into registry.
Parameters:
-
registry(ActionRegistry) –The scheduler's registry; each pack joins it through
include. Understrict, a load that finishes freezes it (ActionRegistry.freeze), so a declaration after the composition root raises; declare the caller's own actions first, or pass them asbuiltins. A diagnostic load (strict=False) leaves it open. -
group(str, default:ENTRY_POINT_GROUP) –The entry-point group to read.
-
block(Iterable[str], default:()) –Pack names to skip (pluggy's blocked set); for tests. Production pins instead.
-
entry_points(EntryPoints | None, default:None) –The entry points to load instead of the installed ones (or a callable that returns them).
-
on_event(EventHook | None, default:None) –Receives
market_pack_loaded,market_pack_failed(stageasPackLoad.stage,plugin_stageasPluginLoadError.stage),market_pack_blockedandplugins_loaded; without one, the stdlib loggerllm_ports.market. -
builtins(Sequence[tuple[str, object]], default:()) –(name, module)packs that ship with the caller, registered first, in order. -
pin(str | None, default:None) –The pin setting's value; None when it is unset (see the module docstring).
-
strict(bool, default:True) –Raise on the first failure. False records each failure and loads the rest.
Returns:
-
tuple[PackLoad, ...]–One
PackLoadper pack, in load order.
Raises:
-
PackLoadError–Under
strict, the first pack that failed, naming its stage. -
RuntimeError–registryis already frozen; nothing is loaded.
packs_loaded
¶
The Loaded record of the packs that joined: names in load order, with their origins.
failed
¶
The loads that failed (a blocked pack is a decision, not a failure).
builtin
¶
builtin(
name: str, *factories: ProviderFactory
) -> tuple[str, object]
Return a built-in provider plugin that registers factories, for load_providers(builtins=...).
Parameters:
-
name(str) –The plugin's name.
-
factories(ProviderFactory, default:()) –The kinds it contributes.
load_providers
¶
load_providers(
*,
builtins: Sequence[tuple[str, object]] = (),
pin: str | None = None,
entry_points: EntryPoints | None = None,
on_event: EventHook | None = None
) -> ProviderRegistry
Load the built-in provider plugins, then every one listed under llm_ports.providers.
Parameters:
-
builtins(Sequence[tuple[str, object]], default:()) –(name, plugin)pairs registered first, in order (builtin(...)makes one). -
pin(str | None, default:None) –The pin setting's value; None when it is unset (see the module docstring).
-
entry_points(EntryPoints | None, default:None) –The entry points to load instead of the installed ones (or a callable that returns them).
-
on_event(EventHook | None, default:None) –Receives
plugins_loaded; without one, the stdlib loggerllm_ports.call.
Returns:
-
ProviderRegistry–The kinds, frozen, with
loadednaming each plugin and its origin.
Raises:
-
PluginLoadError–The first plugin that failed, naming its stage; a kind registered by two plugins is
duplicateand names both.
llm_ports.adapters.pydantic_evals¶
The Judge port implemented on pydantic-evals' LLMJudge.
pydantic-evals' LLMJudge renders the Input/Output/Rubric prompt and binds the verdict to its
GradingOutput grammar (reason, pass, score); this adapter picks the model and translates the
verdict into a JudgeVerdict. Install with the evals extra (llm-ports[evals]).
The model source is the caller's, as for the generator and the embedder:
model=: a fixed pydantic-aiModel(tests pass aFunctionModel).model_for=:() -> async context manager yielding a Model, called per request, so the judge's own call goes through the caller's transport (admission control, leases, a work queue) like every other call. Build it withprofile=JUDGE_PROFILE.
The grade uses native structured output. pydantic-evals' agent asks for GradingOutput as a tool
call by default; JUDGE_PROFILE puts the judge on NativeOutput, a strict json_schema
response_format, which is the mode a grammar-compiling server enforces. verdict_of refuses a
reason that still carries tool-call markup, or a leaked verdict that contradicts the parsed one,
so the caller records that the judge could not run, and why.
A call that sent images is judged with them (images=, beside the port's text-only signature),
as pydantic-evals multimodal Input.
Invariants
PydanticEvalsJudge.judgeraises when the judging model could not answer, as the port requires; the caller decides what an unjudged sample means.- No default verdict exists anywhere in this module.
Rationale: ADR-0001.
PydanticEvalsJudge
¶
PydanticEvalsJudge(
role: Role,
*,
model: Model | None = None,
model_for: JudgeModelFor | None = None
)
The Judge port on pydantic-evals; the module docstring states the contract.
role carries the judge's model name and sampling (Role.model_settings()). Exactly one of
model and model_for is required.
Raises:
-
ValueError–When neither or both of
modelandmodel_forare given.
judge
async
¶
judge(
*,
prompt: str,
output: Any,
rubric: str,
images: Sequence[tuple[bytes, str]] = ()
) -> JudgeVerdict
Grade output against rubric, with prompt and any images as the judged input.
A failure of the judging model's call propagates: the caller decides what an unjudged sample means.
Raises:
-
ValueError–When the reply lacks a score or a verdict, or its reason is unreadable (
verdict_of).
Rationale: ADR-0001.
clean_reason
¶
clean_reason(reason: str, passed: bool) -> str
Return the reason cut where the grade's own JSON leaked into it.
Pure.
Parameters:
-
reason(str) –The judge's reason text.
-
passed(bool) –The parsed verdict, checked against any verdict the leaked tail states.
Raises:
-
ValueError–When the reason carries tool-call markup (its verdict is unreadable), or its leaked tail states a verdict that contradicts
passed.
verdict_of
¶
verdict_of(out: dict[str, Any]) -> JudgeVerdict
Translate an LLMJudge.evaluate result into the port's JudgeVerdict.
out holds score and pass, each with the model's reason. Pure.
Raises:
-
ValueError–When either key is missing (the judge did not answer the question), or when
clean_reasonrefuses the reason.
judge_inputs
¶
judge_inputs(
prompt: str, images: Sequence[tuple[bytes, str]] = ()
) -> str | list[Any]
Build the LLMJudge Input: the text alone, or the text then each image as BinaryContent.
pydantic-evals renders a sequence Input part by part, an image as multimodal content.
llm_ports.adapters.sqlglot¶
A read-only AST guard for SQL an agent wrote (extra [sql], sqlglot).
The guard is one layer of a defense stack, not the stack: run its result as a role that cannot
write, in a READ ONLY transaction with a statement timeout. It parses with sqlglot, validates the
tree, and regenerates the SQL from the tree. The returned string is the only thing to execute, so
comments and formatting tricks do not survive the parser.
A frozen SqlGuardPolicy holds the rules' data; the consumer owns its schema allowlist.
Invariants
- An admitted statement is exactly one statement, and it is a SELECT (UNION, CTE and subquery are fine).
- No DML, DDL, transaction or utility node appears anywhere in the tree (
FORBIDDEN_NODES), and there is noSELECT ... INTO. - Every real table is schema-qualified and inside
allowed_schemas. CTE names are exempt, so the system catalogs are denied unless a policy names them. - No function in
deny_functionsis called (POSTGRES_DENY_FUNCTIONS: sleep, file, backend-signal, advisory-lock, dblink and xml-dump functions). - LIMIT is at most
max_rows: injected when absent, lowered when larger. A non-integer LIMIT is refused. - A policy may tighten the structural rules but never loosen them:
forbidden_nodeskeeps every type inFORBIDDEN_NODES, andsingle_statementstays True. Both are refused at construction, so no instance widens what the guard admits. - The guard fails closed: anything unparseable raises.
Example
POLICY = SqlGuardPolicy(allowed_schemas=("reporting",), deny_functions=POSTGRES_DENY_FUNCTIONS, max_rows=200) safe_sql = guard(agent_sql, POLICY) # raises SqlGuardError with a reason the agent can act on
SqlGuardError
¶
Bases: ValueError
The statement was refused; the message is a reason the agent can act on.
SqlGuardPolicy
dataclass
¶
SqlGuardPolicy(
allowed_schemas: tuple[str, ...],
deny_functions: frozenset[str],
max_rows: int,
forbidden_nodes: tuple[
type[Expression], ...
] = FORBIDDEN_NODES,
single_statement: bool = True,
dialect: str = "postgres",
)
What the guard admits.
Attributes:
-
allowed_schemas(tuple[str, ...]) –The queryable schemas, lower-case. An ordered tuple, so the error text is stable.
-
deny_functions(frozenset[str]) –Function names refused anywhere in the tree, lower-case.
-
max_rows(int) –The largest LIMIT an admitted statement carries; at least 1.
-
forbidden_nodes(tuple[type[Expression], ...]) –Node types refused anywhere in the tree; a superset of
FORBIDDEN_NODES. -
single_statement(bool) –Always True.
-
dialect(str) –The sqlglot dialect that parses and regenerates the statement.
Raises:
-
ValueError–At construction, for a policy that loosens the structural rules, a
max_rowsbelow 1, or a schema or function name that is not lower-case.
schemas_text
property
¶
schemas_text: str
The allowed schemas as one comma-separated string, for error text.
guard
¶
guard(sql: str, policy: SqlGuardPolicy) -> str
Validate sql under policy and return the regenerated statement to execute.
Parameters:
-
sql(str) –The statement the agent wrote.
-
policy(SqlGuardPolicy) –The rules to enforce.
Returns:
-
str–The statement regenerated from its parse tree in
policy.dialect, comments stripped, with -
str–a LIMIT of at most
policy.max_rows.
Raises:
-
SqlGuardError–The statement is empty, unparseable, or breaks a rule; the message names the rule.
llm_ports.adapters.lettucedetect¶
LettuceDetect, a token-span hallucination detector, behind the Grounder port.
The detector returns the offending tokens of a claim with a confidence each, and reads a context of up to 4,096 tokens. It costs what a validator may, not what a hot path may.
The lettucedetect extra (llm-ports[lettucedetect]: torch, transformers and the model, hundreds
of MB) is imported lazily, and the detector is injectable, so the port's shape is tested anywhere
with a fake (llm_ports.testing.StubDetector) and the real model only where it is installed.
score is 1 minus the worst flagged span's confidence. It is this adapter's own scale, never
compared with another checker's.
Invariants
scoreis None when any flagged span carries no confidence: unknown is never zero, and 1.0 would read as fully grounded.- A context longer than
MAX_CONTEXT_CHARS(a character proxy for the model's 4,096-token window) is still judged, and the verdict's reason starts withWINDOW_REASON, so a reader can see the judgement rested on a truncated premise.
Detector
¶
Bases: Protocol
The slice of lettucedetect.models.inference.HallucinationDetector this adapter uses.
predict
¶
predict(
*, context: list[str], answer: str, output_format: str
) -> list[dict[str, Any]]
Return the spans of answer that context does not support.
LettuceDetectGrounder
¶
LettuceDetectGrounder(
detector: Detector | None = None, *, model: str = MODEL
)
A Grounder over LettuceDetect token spans.
A claim is supported when the model flags no span of it as unsupported by the cited chunks' text.
Parameters:
-
detector(Detector | None, default:None) –The detector to use; None loads the real model named by
model. -
model(str, default:MODEL) –The checkpoint
load_detectorloads whendetectoris None.
Invariants
- A claim with no spans is unsupported.
verdict
¶
verdict(
claim: Claim, spans: Sequence[Chunk]
) -> GroundingVerdict
Return whether the text of spans supports claim.
llm_ports.adapters.scoringrules¶
Proper scores beyond the binary case, on scoringrules (extra [scoringrules]).
forecast.score scores a binary forecast with the standard library. A range or a level is not
binary: a stated interval is scored by the interval score (width plus a penalty for each miss,
scaled by 2 / alpha), and a stated distribution by the CRPS. Each function scores many
forecasts at once and returns one score per forecast; lower is better. Names and argument order
are scoringrules' own (obs first), so a reader of its documentation reads these unchanged.
Invariants
- Never raises: inputs of unequal length, empty, or that scoringrules rejects give None.
- A returned list is aligned with the inputs, and every score in it is finite.
Rationale: ADR-0013.
brier_score
¶
brier_score(
obs: Sequence[bool], fct: Sequence[float]
) -> list[float] | None
Return the Brier score of each binary forecast (a cross-check of forecast.score.brier_score).
Parameters:
-
obs(Sequence[bool]) –Whether each event happened.
-
fct(Sequence[float]) –The forecast probabilities, aligned with
obs.
Returns:
-
list[float] | None–One score per forecast, or None for unusable input. Never raises.
interval_score
¶
interval_score(
obs: Sequence[float],
lower: Sequence[float],
upper: Sequence[float],
alpha: float,
) -> list[float] | None
Return the interval score of each stated [lower, upper] range at level 1 - alpha.
Parameters:
-
obs(Sequence[float]) –What the world did.
-
lower(Sequence[float]) –Each range's lower end.
-
upper(Sequence[float]) –Each range's upper end.
-
alpha(float) –The range's stated miss rate, in (0, 1) (0.32 for a one-sigma range).
Returns:
-
list[float] | None–One score per range, or None for unusable input or an alpha outside (0, 1). Never raises.
crps_normal
¶
crps_normal(
obs: Sequence[float],
mu: Sequence[float],
sigma: Sequence[float],
) -> list[float] | None
Return the CRPS of each stated normal distribution N(mu, sigma) at what was observed.
Returns:
-
list[float] | None–One score per forecast, or None for unusable input or a sigma <= 0. Never raises.
llm_ports.adapters.sklearn¶
An isotonic Calibrator on scikit-learn (extra [sklearn]).
IsotonicCalibrator is a scikit-learn estimator: fit(X, y, sample_weight=None) runs
sklearn.isotonic.IsotonicRegression and keeps its knots as a forecast.PiecewiseLinear in
map_, so the fitted map is applied and stored without scikit-learn; predict_proba(X) reads it.
Invariants
- Never raises across the port: too few pairs, one outcome only, unaligned inputs or a failed
fit leave
map_None and the reason ininsufficient_, andpredict_probareturns None. - The fitted map is monotone non-decreasing and inside [0, 1].
Rationale: ADR-0013.
IsotonicCalibrator
¶
IsotonicCalibrator(min_pairs: int = MIN_PAIRS)
Bases: Params
Fits a monotone calibration map by isotonic regression.
Parameters:
-
min_pairs(int, default:MIN_PAIRS) –The floor of usable pairs;
MIN_PAIRSby default.
Attributes:
-
min_pairs–As given.
-
map_(PiecewiseLinear | None) –The fitted map, or None when the last fit produced none. Set by
fit. -
insufficient_(Insufficient | None) –Why the last fit produced no map, or None when it did. Set by
fit.
fit
¶
fit(
X: Sequence[float],
y: Sequence[bool],
sample_weight: Sequence[float] | None = None,
) -> Self
Fit the map.
Parameters:
-
X(Sequence[float]) –Stated probabilities.
-
y(Sequence[bool]) –Whether each event happened, aligned with
X. -
sample_weight(Sequence[float] | None, default:None) –Optional weights aligned with
X, to calibrate toward a reweighted population. A pair is dropped when its probability is invalid or its weight is not a finite real > 0. Inputs of different lengths are not aligned (a derived array that lost a row would misalign every pair after it), so nothing is fitted.
Returns:
-
Self–The estimator, with
map_orinsufficient_set. Never raises.
predict_proba
¶
predict_proba(
X: Sequence[float],
) -> list[tuple[float, float]] | None
Return (P(no), P(yes)) per stated probability; None with no fitted map. Never raises.
llm_ports.adapters.venn_abers¶
A Venn-ABERS Calibrator (extra [venn-abers]), for voices with few resolved forecasts.
Venn-ABERS (Vovk, Petej and Fedorova) fits two isotonic regressions per test point, one assuming
each outcome, and returns the pair (p0, p1) that brackets the calibrated probability; its
validity guarantee holds at any sample size, so it is the calibrator for a small history. The
width of (p0, p1) says how much the history can say. VennAbersCalibrator is a scikit-learn
estimator: fit(X, y), predict_proba(X) (the merged probability) and predict_interval(X).
Invariants
- Never raises across the port: too few pairs, one outcome only, or a failed fit leave
model_None and the reason ininsufficient_, and both predictions return None; a failed prediction for one point returns its stated probability unchanged. - Every returned probability is in [0, 1].
Rationale: ADR-0013.
VennAbersCalibrator
¶
VennAbersCalibrator(min_pairs: int = MIN_PAIRS)
Bases: Params
Fits a Venn-ABERS map.
Parameters:
-
min_pairs(int, default:MIN_PAIRS) –The floor of usable pairs;
MIN_PAIRSby default.
Attributes:
-
min_pairs–As given.
-
model_(Any) –The fitted
venn_abers.VennAbers, or None when the last fit produced none. -
n_(int) –The pairs the model was fitted on (0 with no model).
-
insufficient_(Insufficient | None) –Why the last fit produced no model, or None when it did.
fit
¶
fit(X: Sequence[float], y: Sequence[bool]) -> Self
Fit the map.
Parameters:
-
X(Sequence[float]) –Stated probabilities.
-
y(Sequence[bool]) –Whether each event happened, aligned with
X.
Returns:
-
Self–The estimator, with
model_orinsufficient_set. Never raises.
predict_interval
¶
predict_interval(
X: Sequence[float],
) -> list[tuple[float, float]] | None
Return (p0, p1) per stated probability, the bracket around its calibrated value.
Returns:
-
list[tuple[float, float]] | None–The brackets, None with no fitted model. A point with an invalid probability gets
-
list[tuple[float, float]] | None–(0.5, 0.5), a failed prediction(p, p). Never raises.
predict_proba
¶
predict_proba(
X: Sequence[float],
) -> list[tuple[float, float]] | None
Return (P(no), P(yes)) per stated probability, P(yes) = p1 / (1 - p0 + p1).
Returns:
-
list[tuple[float, float]] | None–The rows, None with no fitted model. Never raises.
llm_ports.adapters.river¶
A learned Combiner on River's online logistic regression (extra [river]).
RiverLogitPool regresses the outcome on each voice's log-odds, one feature per voice, plus a
silent flag per voice it has heard before and that did not speak this time: a voice's silence
can carry information. It learns one event at a time, so its weights follow drift without a
nightly refit. Until it has heard warmup outcomes it does not combine ({}), so a caller falls
back to its baseline rather than read an untrained model. It speaks the Combiner protocol,
River's own: predict_proba_one(x) and learn_one(x, y), with x as voice -> probability.
Invariants
- Never raises across the port: a River failure leaves the state unchanged (
learn_one) or returns{}(predict_proba_one). - A combined probability is in [0, 1].
Rationale: ADR-0013.
RiverLogitPool
¶
RiverLogitPool(
lr: float = 0.05, l2: float = 0.0, warmup: int = WARMUP
)
An online logistic logit pool.
Parameters:
-
lr(float, default:0.05) –The SGD learning rate.
-
l2(float, default:0.0) –The L2 penalty on the weights.
-
warmup(int, default:WARMUP) –Outcomes heard before
predict_proba_oneanswers;WARMUPby default.
Attributes:
-
seen(int) –Outcomes heard so far.
-
failures(int) –Updates River refused (state left unchanged); a rising count means the pool is not learning.
-
voices(set[str]) –Every voice heard so far.
predict_proba_one
¶
predict_proba_one(
x: Mapping[str, float],
) -> dict[bool, float]
Return {False: P(no), True: P(yes)}; {} before warmup or with no usable voice. Never raises.
learn_one
¶
learn_one(x: Mapping[str, float], y: bool) -> None
Update the weights on one resolved event; an event with no usable voice is ignored. Never raises.
weights
¶
weights() -> dict[str, float]
Return the current feature weights (voice:<v>, silent:<v>) for a readout.
llm_ports.adapters.mlflow¶
The learning loop's run ledger and model registry on MLflow (extra [mlflow]).
MlflowRunLedger(client) records each RunLedger.append as one MLflow run, so loops land in
MLflow's tracking UI and API next to every other experiment:
- experiment:
prefix + kind(created on first append;artifact_locationsets where its runs' artifacts go). - tags:
llm_ports.kind,llm_ports.subject,llm_ports.decision(the recorded verdict's decision, absent on an ungated run) andmlflow.runName. - params / metrics: every scalar param and every finite numeric metric whose name MLflow
accepts. A verdict, a model or a string metric is not a number MLflow can chart, so the whole
row is also JSON in the run's tags (
ROW_TAG/000,ROW_TAG/001, ... of at most 8000 characters each, thenROW_PARTS_TAG= the count, written last), whichlatestreads back from the search result itself: no artifact store, no download. A row overMAX_ROW_JSONcharacters, or a subject over 8000 (MLflow would cut its tag short), is refused.latestmatches the row's own kind, so two prefixes whose experiment names collide (a+bc,ab+c) never cross-read. Written.ref/LedgerRow.ref: the MLflow run id.
A run ends FINISHED once every part is written, FAILED when a write fails part-way; latest
reads FINISHED runs whose row is complete, so a half-written or foreign run never reads as a
row. Runs end through update_run (MLflow's set_terminated prints a URL to stdout on a
tracking server), so their end_time is unset. Runs are ordered by start time in milliseconds,
stamped strictly increasing per ledger object; two ledger objects writing one kind in the same
millisecond have no defined order.
MlflowModelRegistry(client) maps ModelRegistry onto MLflow's model registry: a registered
model per name, a model version per register, aliases as MLflow aliases. A version's source
is the artifact_uri when one is given (a tracking server refuses a local path as a source, so
use a remote URI such as s3://... there, or runs:/<id>/..., whose run id is passed along as
MLflow's server requires); with none it is a backing run in the experiment
registry_experiment (runs:/<id>/llm_ports), since MLflow requires a source. The fitted
model is JSON in the version tag llm_ports.model (at most MAX_MODEL_JSON characters; a
larger model belongs at an artifact_uri), run_ref in llm_ports.run_ref. register does not
wait for the version to turn READY (MLflow's default waits and logs while it does); a backend that
copies the artifact first, Databricks for one, reports it PENDING_REGISTRATION until then.
The client is the caller's (mlflow.MlflowClient(tracking_uri, registry_uri)): a tracking
server, Databricks, or a local sqlite:/// file. MLflow's client is synchronous; every call runs
in a worker thread, and one ledger or registry object is safe to share between threads. Against
an unreachable server MLflow retries each call before failing (MLFLOW_HTTP_REQUEST_MAX_RETRIES,
MLFLOW_HTTP_REQUEST_BACKOFF_FACTOR, MLFLOW_HTTP_REQUEST_TIMEOUT; minutes by default), so a
loop that must report unavailable promptly sets those lower in its environment.
Invariants
- Never raises: MLflow refusing a value (
INVALID_PARAMETER_VALUE) is arequest_fault; any other failure, a server down included, isunavailable. A param or metric MLflow would refuse (its name, an unconvertible number) is left out of MLflow's columns, never a failed write; it stays in the row. - Its calls write nothing to stdout or stderr.
- Passes
llm_ports.testing.conformance(check_run_ledger,check_model_registryand their broken-storage checks) against a tracking server and a local SQLite store.
Rationale: ADR-0016.
llm_ports.adapters.mlflow.ledger¶
MlflowRunLedger: the learning loop's run ledger on MLflow tracking (see the package docstring).
MlflowRunLedger
¶
MlflowRunLedger(
client: MlflowClient,
*,
prefix: str = "",
artifact_location: str | None = None,
clock: Callable[[], float] = time
)
A RunLedger over MLflow tracking: one run per row (see the module docstring).
Record runs through client.
Parameters:
-
client(MlflowClient) –The caller's MLflow client.
-
prefix(str, default:'') –Prepended to a kind to name its experiment.
-
artifact_location(str | None, default:None) –Where a new experiment's artifacts go; None for the server's default.
-
clock(Callable[[], float], default:time) –Wall-clock seconds, for each run's start time; injectable for tests.
llm_ports.adapters.mlflow.registry¶
MlflowModelRegistry: the model registry port on MLflow's registry (see the package docstring).
MlflowModelRegistry
¶
MlflowModelRegistry(
client: MlflowClient,
*,
registry_experiment: str = "llm_ports.registry"
)
A ModelRegistry over MLflow's model registry (see the module docstring).
Register models through client (its registry URI).
Parameters:
-
client(MlflowClient) –The caller's MLflow client.
-
registry_experiment(str, default:'llm_ports.registry') –The experiment holding the backing run of a version registered with no
artifact_uri.
register
async
¶
register(
name: str,
*,
model: Mapping[str, Any],
artifact_uri: str | None = None,
run_ref: str | None = None,
tags: Mapping[str, str] | None = None
) -> Written
Create an MLflow model version. Never raises.
set_alias
async
¶
set_alias(name: str, alias: str, version: int) -> Written
Set an MLflow alias. Never raises; MLflow refuses latest and v<N> as aliases.
by_alias
async
¶
by_alias(name: str, alias: str) -> VersionRead
Read the version an MLflow alias points at. Never raises.
get
async
¶
get(name: str, version: int) -> VersionRead
Read one MLflow model version. Never raises.