The model table: schema 2 states what each model can do and who serves it, by name; providers and packs load through one strict plugin contract¶
Decided in the release after 1.22.0. Amends ADR-0018 (duplicate declarations, pack loading).
Context and problem statement¶
A 1.x tier table maps a tier to a model id string and a timeout. Nothing in it says what a model
can do, so a consumer inferred capability from a model's name (a -web suffix), and a request
with images sent to a text-only model failed on the wire, after a trace had opened and the call
had been paid for. The thinking switch was spelled inside Role.model_settings() for one server
family (extra_body.chat_template_kwargs), so a model that spells it differently could not be
listed. Where a model is served, and with which key, lived in the consumer's wiring code, outside
the reviewed table.
The 1.22 pack loader (load_packs) shared a defect class with every plugin loader the estate had
reviewed: it never raised, so a pack that failed to import was a log line and the scheduler ran
without it; load order followed install order; an equal re-declaration with a different handler
replaced the held one silently; and a misspelled hook name loaded as a plugin that did nothing.
Providers by kind are the second entry-point group, and the third use of the estate's plugin
contract, so they and packs should share one loader that fails loudly.
Decision¶
schema_version = 2. A table without the key is version 1 and loads to the same values as 1.22.0 (pinned by a snapshot); a version 1 table that carries a version 2 section is refused, so a half-migrated file is not read as the old shape. Version 2 adds:[providers.<name>]: a registeredkind, and by NAME only abase_urlkey and asecretname. A value holding://is refused: URLs and keys stay out of a reviewed, fingerprinted table.[models.<key>](ModelInfo): the wire id, its provider,supportsfrom a closed capability vocabulary,thinkingstyle, and limits that are None when unknown, never a guessed number.[policies.<name>], carried read-only until the resilience row types them.- Per tier:
policy, orderedfallbacktier names andfallback_onfailure kinds (default: overloaded, server, transport, timeout). Cycles, self-fallback and unknown names are refused. Every refusal isTableInvalid(ValueError)naming the dotted path. - A fingerprint.
TierTable.fingerprint()is the sha256 (first 16 hex digits) of the parsed table's canonical JSON. Each loaded tier carries it, andcall.generatestamps it on every result asstamp["tiers"], so a graded result names the table that routed it. - Capability before the wire.
call.generatederives what a request needs (visionfor images,web,json_schemafor structured output, withjson_objectstanding in) and returnsrequest_faultwith anunsupported: ...reason when the tier's model does not state it, with zero wire requests.TierTable.require(role, *caps)proves at import that a role's tier and every tier it can fall back to state what the role needs. A schema 1 table states nothing and is not checked; onellm_call_warningper process says so. - Provider detail lives in the adapter. A schema 2 role sends
thinking: bool; theopenai_compatiblefactory spells it perModelInfo.thinking(unified: the framework's own setting;chat_template:extra_body.chat_template_kwargs.enable_thinking;none: dropped, with onellm_call_warningper model when a role asked to think). Schema 1 roles keep the 1.22 spelling. - Providers by kind, built at boot.
call.providers.build(table, urls=, secrets=, deps=, factories=)resolves each used provider's names through the caller's mappings and raises at boot for an unknown kind, an unresolved URL key, or a secret that resolves to nothing or to a value no header can carry (whitespace, a control or a non-ASCII character: httpx quotes a refused header, value and all, in its error). It returnsBackends(generator_for(tier),embedder_for(model_key),probe_for(provider), and typed placeholders for the chat and fallback rows), after emittingllm_backends_builtwith the fingerprint and the plugin origins. Onlyopenai_compatibleis built; other kinds are seams until a consumer uses one. Probe. Aports.ProbereturnsHealth(state, models, reason, latency_s).unknownmeans the probe itself failed (including a request it could not send), the credential was refused (401/403: auth, not liveness), the backend has no endpoint to judge, or a 200 body is not a model list;downmeans it could not be reached or answered with an error. A breaker opens ondownonly. A reason names an exception by its type, never its message.adapters.http.HttpProbeis the OpenAI-wire implementation (GET /models, 5 s deadline).- One strict plugin contract for both groups (
llm_ports.market,llm_ports.providers): hook specs added before any plugin registers; built-ins registered first by explicit name; entry points loaded sorted by name; a pin (Noneall,""built-ins only,"a,b"exactly those, a missing pinned name raises);check_pendingafter each registration; any failure raisesPluginLoadErrornaming the group, plugin and stage (load_packs(strict=False)keeps the 1.22 record-and-continue behaviour for a diagnostic); a duplicate key names both plugins; the loader freezes the registry when a strict load finishes;plugins_loadedreports names and origins. The same ten conformance tests run against both groups. - The gate fails closed until read. A
GateCachethat has never completed a read refuses every category withgate_unread;fail_open=Trueadmits instead. Each judgement from never-read gates emitsmarket_gate_unreadwithservingclosedoropen. A gate nobody refreshed is not a gate, and nothing refreshes it but the consumer, at boot and on a schedule.
Considered options¶
- A bundled capability database (LiteLLM's model price/context file). Thousands of entries, megabytes, and stale for a self-hosted model; the consumer knows its ten models.
- Capability booleans on a model class. Cannot differ per model version or per deployment.
- Inferring capability from the model name. The defect this replaces.
- URLs and keys in the table. The table is reviewed in a pull request and fingerprinted; a value there leaks into both.
- Detect an unknown provider kind at
load_tiers. Kinds arrive from plugins, which load after the table may already be parsed; the kind is checked atbuild, which is still boot. - Keep
load_packslenient by default. A producer that silently failed to load is the failure the registry exists to prevent; the lenient mode stays as an opt-in for diagnostics. - A required
on_unreadableargument onGateCache. A required argument breaks 1.x callers;fail_open=Falsekeeps the safe default without breaking a call site.
Consequences¶
- A request a model cannot serve costs nothing and is named; a table edit that removes a needed
capability fails the boot through
require, not a call. - Behaviour changes for 1.22 callers, each stated first in the changelog:
load_packsraises by default; an equal re-declaration with a different spec object raisesDuplicateAction; adesired_ttlneedscache_tableandage_col; a never-readGateCacherefuses; themarketextra needs pluggy 1.7; a strictload_packsfreezes the registry it loaded into; every result carriesstamp["tiers"]. A version 1 table with a[providers]or[policies]section still loads to its 1.22 values (the sections are ignored, with allm_table_ignoredwarning), since 1.x is additive. TierTable.modelsstays key to id string for 1.x;ModelInfolives beside it inmodel_info.- Fallback (the resilience row) and the chat port (the chat row) read
Tier.fallback,Tier.fallback_on,Tier.policyandBuilt.chat, which are typed placeholders here.