llm_ports.market¶
llm_ports.market
¶
The work market's declaration side: background actions, their registry, and the controls a scheduler reads.
A capacity-driven scheduler runs background LLM work when its model server is free: the most valuable due item first, fair across features. This package holds the parts every producer and that scheduler share; the scheduler, its backlog and its store are the consumer's.
ActionSpec,ActionRegistry: a feature declares an action with@registry.action(...); a spec nothing could judge stale is refused at import.KillSwitch: the operator's pausedfeature.actionset, from a setting the consumer reads.GateCache,LastKnown: cached control reads;GateCache.admitis acall.Hooks.admit, and refuses asgate_unreaduntil the consumer's boot has read the gates once.ActionMirror,sync_all: the consumer's store copy of the registry, which an operator tunes.PrunableMirror,prune_mirror: retire the mirror rows of actions the code no longer declares.- Packs: a distribution exposes its actions under the
ENTRY_POINT_GROUPentry-point group, andadapters.pluggy.load_packsadds them to the scheduler's registry (extra[market]).
Events (through each component's on_event hook, else the stdlib logger llm_ports.market):
market_action_refused, market_actions_disabled, market_kill_unknown, market_read_failed,
market_sync_failed, market_gate_unread, market_action_pruned, market_prune_held,
market_prune_failed; the pack loader adds market_pack_loaded, market_pack_failed and
market_pack_blocked.
Invariants
- No store, no table name and no setting name is built in: each is passed in by the consumer.
Rationale: ADR-0018.
Gate
dataclass
¶
Gate(
enabled: bool = True,
call_cap: int | None = None,
used: int | None = None,
)
One category's switch.
Attributes:
-
enabled(bool) –False switches the category off.
-
call_cap(int | None) –The most calls admitted per window; None for no cap.
-
used(int | None) –Calls already admitted this window, as of the read; None when unknown (the cap is then not enforced by the cached check).
GateCache
¶
GateCache(
read: Callable[[], Awaitable[Mapping[str, Gate]]],
*,
window: str | None = None,
ttl_s: float = 10.0,
clock: Callable[[], float] = monotonic,
on_event: EventHook | None = None,
fail_open: bool = False
)
The per-category switchboard, cached: refresh reads it, check and admit judge it.
Parameters:
-
read(Callable[[], Awaitable[Mapping[str, Gate]]]) –Returns every seeded category's
Gate. -
window(str | None, default:None) –Names the cap's window in the refusal (
"daily"); None leaves it out. -
ttl_s(float, default:10.0) –Seconds a read stays fresh.
-
clock(Callable[[], float], default:monotonic) –Monotonic seconds; injectable for tests.
-
on_event(EventHook | None, default:None) –Receives
market_read_failedandmarket_gate_unread; without one, the stdlib loggerllm_ports.market. -
fail_open(bool, default:False) –Admit while the gates have never been read, instead of refusing as
gate_unread. Each judgement from never-read gates emitsmarket_gate_unread, withservingopenorclosed.
refresh
async
¶
refresh() -> Mapping[str, Gate]
The gates, read when the cache is stale. Never raises.
prime
¶
prime(gates: Mapping[str, Gate]) -> None
Set the cached gates and mark them fresh (a warm start, a test).
check
¶
check(category: str | None) -> str | None
Judge category against the cached gates: the refusal reason, or None to admit. Never raises.
No category, or an unknown one, is admitted. Before any read has succeeded, a category is
refused as gate_unread (or admitted, when built fail_open), and market_gate_unread
names which.
LastKnown
¶
LastKnown(
read: Callable[[], Awaitable[T]],
initial: T,
*,
name: str = "value",
ttl_s: float = 10.0,
clock: Callable[[], float] = monotonic,
on_event: EventHook | None = None
)
A value read through read at most once per ttl_s, serving the last known one on a failure.
Parameters:
-
read(Callable[[], Awaitable[T]]) –The store read.
-
initial(T) –The value served until a read succeeds.
-
name(str, default:'value') –Names the value in
market_read_failed. -
ttl_s(float, default:10.0) –Seconds a read (or a failed read) stays fresh.
-
clock(Callable[[], float], default:monotonic) –Monotonic seconds; injectable for tests.
-
on_event(EventHook | None, default:None) –Receives
market_read_failed; without one, the stdlib loggerllm_ports.market.
ActionMirror
¶
Bases: Protocol
A consumer's store copy of the registry.
sync_action
async
¶
sync_action(spec: ActionSpec) -> None
Upsert spec's CODE_OWNED fields; seed OPERATOR_TUNED ones only on first insert.
Raises:
-
Exception–Any store failure;
sync_allcatches and reports it.
SyncReport
dataclass
¶
SyncReport(
synced: tuple[str, ...],
failed: tuple[tuple[str, str], ...],
)
What one sync_all pass did.
Attributes:
-
synced(tuple[str, ...]) –Names of the specs synced.
-
failed(tuple[tuple[str, str], ...]) –(name, reason)for each spec whose sync raised.
MirrorRow
dataclass
¶
MirrorRow(
name: str, enabled: bool, origin: str | None = None
)
One row of the consumer's mirror, as prune_mirror reads it.
Attributes:
-
name(str) –The action's
feature.actionkey. -
enabled(bool) –The operator's switch.
-
origin(str | None) –The pack (or module) that declared it, when the table records one; else None.
PrunableMirror
¶
Bases: ActionMirror, Protocol
An ActionMirror that can list its rows and retire one.
mirrored
async
¶
mirrored() -> Sequence[MirrorRow]
Every row of the mirror.
Raises:
-
Exception–Any store failure;
prune_mirrorcatches and reports it.
retire_action
async
¶
retire_action(name: str) -> None
Retire the row of an action no longer declared: delete it, or set it disabled.
Raises:
-
Exception–Any store failure;
prune_mirrorcatches and reports it.
PruneReport
dataclass
¶
PruneReport(
pruned: tuple[str, ...] = (),
held: tuple[str, ...] = (),
failed: tuple[tuple[str, str], ...] = (),
error: str | None = None,
)
What one prune_mirror pass did.
Attributes:
-
pruned(tuple[str, ...]) –Names of the rows retired.
-
held(tuple[str, ...]) –Names of unregistered enabled rows NOT retired because a pack failed to load and the row's origin is unknown or is that pack's.
-
failed(tuple[tuple[str, str], ...]) –(name, reason)for each row whose retire raised. -
error(str | None) –Why the mirror could not be read; None when it was.
ActionRegistry
¶
ActionRegistry(
name: str = "actions",
*,
clocks: tuple[str, ...] = SILENCE_CLOCKS,
on_event: EventHook | None = None,
sources: Collection[str] | None = None
)
Bases: MutableMapping[Key, ActionSpec]
Insertion-ordered (feature, action) -> ActionSpec table with the declare-time guard.
Parameters:
-
name(str, default:'actions') –Names the registry in errors and events (a pack's name, or the process's).
-
clocks(tuple[str, ...], default:SILENCE_CLOCKS) –The silence clocks a spec may declare.
-
on_event(EventHook | None, default:None) –Receives
market_action_refused; without one, the stdlib loggerllm_ports.market. -
sources(Collection[str] | None, default:None) –The source ids a spec may name (the consumer's lane registry); None checks none.
freeze
¶
freeze() -> None
Close the registry: a later add, include or item write raises RuntimeError.
origin
¶
origin(feature: str, action: str) -> str | None
Who declared feature.action: a pack's name, or the handler's module path; None if unknown.
add
¶
add(
spec: ActionSpec, *, origin: str | None = None
) -> ActionSpec
Declare spec and return it.
Parameters:
-
spec(ActionSpec) –The declaration.
-
origin(str | None, default:None) –Who declares it, named in a duplicate's error; defaults to the handler's path.
Raises:
-
ValueError–The spec is refused (
refusal). -
DuplicateAction–A different spec object holds its key.
-
RuntimeError–The registry is frozen.
action
¶
action(
*,
feature: str,
action: str,
model: str | None = None,
lane: int = 2,
value_weight: float = 1.0,
desired_ttl: str | None = None,
est_cost_s: float = 40.0,
cadence: str | None = None,
cache_table: str | None = None,
age_col: str | None = None,
stale_fn: StaleFn | None = None,
source: str | Iterable[str] | None = None,
max_silence_h: float | None = None,
silence_clock: str = "wall"
) -> Callable[[H], H]
Return a decorator that declares the handler it wraps; the handler is returned unchanged.
The arguments are ActionSpec's fields; source also takes one id as a string.
Raises:
-
ValueError–At decoration, when the spec is refused or another spec holds the key (
DuplicateAction).
include
¶
include(
other: ActionRegistry, *, origin: str | None = None
) -> tuple[ActionSpec, ...]
Add every spec of other (a pack), all or nothing, and return the specs added.
Parameters:
-
other(ActionRegistry) –The pack.
-
origin(str | None, default:None) –Who contributes it, named in a duplicate's error; defaults to
other.name.
Raises:
-
ValueError–A spec of
otheris refused here (a clock or source this registry does not accept); nothing is added. -
DuplicateAction–A different spec object holds one of its keys; nothing is added.
-
RuntimeError–The registry is frozen.
find
¶
find(feature: str, action: str) -> ActionSpec | None
The spec under (feature, action), or None.
empty
¶
empty() -> ActionRegistry
A fresh, open registry with the same name, clocks, sources and event hook (test isolation).
DuplicateAction
¶
Bases: ValueError
A feature.action key declared twice with different spec objects; the message names both origins.
ActionSpec
dataclass
¶
ActionSpec(
feature: str,
action: str,
handler_path: str,
fn: Handler,
model: str | None = None,
lane: int = 2,
value_weight: float = 1.0,
desired_ttl: str | None = None,
est_cost_s: float = 40.0,
cadence: str | None = None,
cache_table: str | None = None,
age_col: str | None = None,
stale_fn: StaleFn | None = None,
max_silence_h: float | None = None,
silence_clock: str = "wall",
source: tuple[str, ...] = (),
)
A background action: a handler, what makes it due, and how it is ranked and judged.
Attributes:
-
feature(str) –The feature that owns the action (the first half of its key).
-
action(str) –The action's name within the feature (the second half of its key).
-
handler_path(str) –module.qualnameof the handler, recorded as provenance. -
fn(Handler) –The handler. Not compared: two imports of one module declare equal specs.
-
model(str | None) –The model alias the handler calls, recorded for the operator; None when it calls none.
-
lane(int) –Priority lane, 0 first; a scheduler claims a lower lane before a higher one.
-
value_weight(float) –Relative worth inside a lane; a fair-share scheduler divides by it.
-
desired_ttl(str | None) –How long an output stays fresh, as an interval literal (
"18h"). -
est_cost_s(float) –Expected seconds per dispatch; a scheduler scales its timeout and lease by it.
-
cadence(str | None) –How often the action is expected to run, as an interval literal.
-
cache_table(str | None) –Where the action's output lands, for a freshness check to read.
-
age_col(str | None) –The column of
cache_tablethat dates an output. -
stale_fn(StaleFn | None) –Yields the work keys currently due; None for an action fed only by events. Not compared.
-
max_silence_h(float | None) –Hours of output silence past which an action with no
desired_ttland nocadenceis stale: a property of how often its input arrives. -
silence_clock(str) –The clock silence is counted on (
SILENCE_CLOCKS, or the registry's own). -
source(tuple[str, ...]) –Ids of the content sources the action reads or writes; a consumer skips the action while any of them is switched off. Empty when it depends on none.
limit_h
property
¶
limit_h: float | None
The declared freshness limit in hours: desired_ttl, else cadence, else None.
None means max_silence_h is the rule.
KillSwitch
¶
KillSwitch(
read: Callable[[], str | None],
*,
default: Iterable[str] = (),
on_event: EventHook | None = None,
registry: ActionRegistry | None = None
)
The paused set, read once from a consumer's setting.
Parameters:
-
read(Callable[[], str | None]) –Returns the raw setting, or None when it is unset.
-
default(Iterable[str], default:()) –The paused set while the setting is unset.
-
on_event(EventHook | None, default:None) –Receives
market_actions_disabledandmarket_kill_unknown; without one, the stdlib loggerllm_ports.market. -
registry(ActionRegistry | None, default:None) –The actions the names must match; each parse runs
unknownagainst it. None checks nothing (callunknownyourself).
unknown
¶
unknown(registry: ActionRegistry) -> frozenset[str]
Paused names that match no action in registry: a typo pauses nothing.
A non-empty result is emitted as market_kill_unknown (WARNING).
reset
¶
reset() -> None
Forget the parsed set; the next read parses the setting again (and emits again).
sync_all
async
¶
sync_all(
registry: ActionRegistry,
mirror: ActionMirror,
*,
on_event: EventHook | None = None
) -> SyncReport
Sync every spec of registry into mirror. Never raises.
Parameters:
-
registry(ActionRegistry) –The declared actions.
-
mirror(ActionMirror) –The consumer's store copy.
-
on_event(EventHook | None, default:None) –Receives
market_sync_failed; without one, the stdlib loggerllm_ports.market.
Returns:
-
SyncReport–The names synced and the failures, in declaration order.
prune_mirror
async
¶
prune_mirror(
registry: ActionRegistry,
mirror: PrunableMirror,
*,
unloaded: Iterable[str] = (),
on_event: EventHook | None = None
) -> PruneReport
Retire each enabled mirror row whose action registry does not hold. Never raises.
Parameters:
-
registry(ActionRegistry) –The actions this process declares (after every pack loaded).
-
mirror(PrunableMirror) –The consumer's store copy.
-
unloaded(Iterable[str], default:()) –The packs that did NOT load in this process (failed or blocked: with
adapters.pluggy.load_packs, thenameof eachPackLoadwhoseoutcomeis notok). Their rows are held; see the module docstring. -
on_event(EventHook | None, default:None) –Receives
market_action_pruned,market_prune_heldandmarket_prune_failed; without one, the stdlib loggerllm_ports.market.
Returns:
-
PruneReport–The rows retired, held and failed, in the mirror's order.
parse_interval
¶
parse_interval(literal: str) -> timedelta
Parse a short interval literal: an integer and one unit of s, m, h or d.
Parameters:
-
literal(str) –"90s","30m","18h","7d"; surrounding whitespace is ignored.
Returns:
-
timedelta–The interval.
Raises:
-
ValueError–literalis not<int><s|m|h|d>.
refusal
¶
refusal(
spec: ActionSpec,
clocks: Collection[str] = SILENCE_CLOCKS,
sources: Collection[str] | None = None,
) -> str | None
Return why spec cannot be declared, or None when it can.
Parameters:
-
spec(ActionSpec) –The declaration.
-
clocks(Collection[str], default:SILENCE_CLOCKS) –The silence clocks the registry accepts.
-
sources(Collection[str] | None, default:None) –The source ids that exist (the consumer's lane registry); None checks none.
Returns:
-
str | None–The reason, naming the field; None for a declarable spec. Never raises.
parse_names
¶
parse_names(raw: str) -> frozenset[str]
Split a comma-separated list of feature.action names; blanks are dropped.
Parameters:
-
raw(str) –The setting's value.
Returns:
-
frozenset[str]–The names, stripped. Never raises.