Skip to content

Architecture decision records

Why the library is shaped the way it is, with the evidence that decided it. Each record follows MADR 4. Docstrings state the contract and point here with Rationale: ADR-NNNN.; what changed release by release is in CHANGELOG.md.

The first eight were recorded together in 0.12.0, after the fact: each decision had shipped earlier, and its reasoning lived in docstrings until then. The release that made each decision is named in its record.

ADR Decision
0001 A model call ends in a named outcome and never raises; Judge is the one port that raises
0002 Grounded prose by construction: the model cites a value by id, code prints the value
0003 One number law, comparing figures by value, digits and spelled-out
0004 Context a digit-free writer reads is digit-free too
0005 The digit-free grammar is a character whitelist, not a negated class
0006 Data rights and lineage in PROV-O terms, with one deterministic print gate
0007 One observation lifecycle; unmeasured usage is unmeasured, never zero
0008 Prompt packs live in the repo and are named by a content sha
0009 The pydantic-ai adapter hands its provider an httpx2 client and bridges a legacy httpx transport
0010 A standalone learning-loop core (llm_ports.learn): one gate, one run ledger port, one labelled-example contract
0011 Label score names follow Langfuse's score-config rule (label-<role>); reads accept the old : spelling
0012 A judge earns trust from human labels per judge model (calibration.trust, kappa_ci, score metadata)
0013 Forecast scoring in a standalone core (llm_ports.forecast); fitters and non-binary scores in four adapters
0014 A person's label can be relayed by code (put with by), on the same score config, completing its queue item
0015 A combiner is replayed walk-forward, shrunk toward a prior, and tuned only on the past (forecast.walk, PriorPool, adapters.optuna)
0016 Seams to bigger ML engines: run ids, artifact URIs, a ModelRegistry port, conformance suites, and MLflow as the first engine adapter
0017 Prompt tuning runs on the gepa core behind a caller's evaluator; the study UI is launched on demand
0018 The work market's declaration side is a core subpackage (llm_ports.market); packs from other distributions load through pluggy entry points (adapters.pluggy)
0019 A result names what was measured (Usage, summed over every request), what it cost (Cost: gateway, then a Pricer, then unknown) and who answered (Answered)
0020 A failure has a kind (Failure.kind) under coarse outcomes; refused is the seventh outcome, a cut reply is truncated, a bug is internal
0021 One retry owner (the adapter's loop, keyed on Failure.kind, a WirePolicy) and a total Deadline over the whole call; a cancel propagates; a slot is taken inside the deadline
0022 The model table: schema 2 states each model's capabilities and its provider by name; capability is checked before the wire; providers and packs load through one strict plugin contract; a never-read gate fails closed
0023 A Chat port beside Generator: messages in, events and new messages out; tools run by the adapter under a timeout and an output budget; max_steps bounds the loop; ADR-0001 holds for streams
0024 Context is measured, not estimated: an exact TokenCounter or None; Role.context checks or trims before the wire; unknown sends; turns are trimmed whole; the cache hint follows the stated capability
0025 OpenTelemetry is a Tracer (OtelTracer, composed with TeeTracer); every GenAI name is pinned in observability.semconv and a weekly job diffs it against HEAD; bodies stay off OTel; gen_ai.system is gone; one end payload on every exit, cancel included
0026 A pack has an identity (byte_sha over the manifest and the closed template set) beside its frozen content_sha; CRLF is refused; the manifest is resolved by the library