Skip to content

llm_ports.market

llm_ports.market

The work market's declaration side: background actions, their registry, and the controls a scheduler reads.

A capacity-driven scheduler runs background LLM work when its model server is free: the most valuable due item first, fair across features. This package holds the parts every producer and that scheduler share; the scheduler, its backlog and its store are the consumer's.

  • ActionSpec, ActionRegistry: a feature declares an action with @registry.action(...); a spec nothing could judge stale is refused at import.
  • KillSwitch: the operator's paused feature.action set, from a setting the consumer reads.
  • GateCache, LastKnown: cached control reads; GateCache.admit is a call.Hooks.admit, and refuses as gate_unread until the consumer's boot has read the gates once.
  • ActionMirror, sync_all: the consumer's store copy of the registry, which an operator tunes.
  • PrunableMirror, prune_mirror: retire the mirror rows of actions the code no longer declares.
  • Packs: a distribution exposes its actions under the ENTRY_POINT_GROUP entry-point group, and adapters.pluggy.load_packs adds them to the scheduler's registry (extra [market]).

Events (through each component's on_event hook, else the stdlib logger llm_ports.market): market_action_refused, market_actions_disabled, market_kill_unknown, market_read_failed, market_sync_failed, market_gate_unread, market_action_pruned, market_prune_held, market_prune_failed; the pack loader adds market_pack_loaded, market_pack_failed and market_pack_blocked.

Invariants
  • No store, no table name and no setting name is built in: each is passed in by the consumer.

Rationale: ADR-0018.

Gate dataclass

Gate(
    enabled: bool = True,
    call_cap: int | None = None,
    used: int | None = None,
)

One category's switch.

Attributes:

  • enabled (bool) –

    False switches the category off.

  • call_cap (int | None) –

    The most calls admitted per window; None for no cap.

  • used (int | None) –

    Calls already admitted this window, as of the read; None when unknown (the cap is then not enforced by the cached check).

GateCache

GateCache(
    read: Callable[[], Awaitable[Mapping[str, Gate]]],
    *,
    window: str | None = None,
    ttl_s: float = 10.0,
    clock: Callable[[], float] = monotonic,
    on_event: EventHook | None = None,
    fail_open: bool = False
)

The per-category switchboard, cached: refresh reads it, check and admit judge it.

Parameters:

  • read (Callable[[], Awaitable[Mapping[str, Gate]]]) –

    Returns every seeded category's Gate.

  • window (str | None, default: None ) –

    Names the cap's window in the refusal ("daily"); None leaves it out.

  • ttl_s (float, default: 10.0 ) –

    Seconds a read stays fresh.

  • clock (Callable[[], float], default: monotonic ) –

    Monotonic seconds; injectable for tests.

  • on_event (EventHook | None, default: None ) –

    Receives market_read_failed and market_gate_unread; without one, the stdlib logger llm_ports.market.

  • fail_open (bool, default: False ) –

    Admit while the gates have never been read, instead of refusing as gate_unread. Each judgement from never-read gates emits market_gate_unread, with serving open or closed.

gates property

gates: Mapping[str, Gate]

The last cached gates, without a read.

refresh async

refresh() -> Mapping[str, Gate]

The gates, read when the cache is stale. Never raises.

prime

prime(gates: Mapping[str, Gate]) -> None

Set the cached gates and mark them fresh (a warm start, a test).

invalidate

invalidate() -> None

Mark the cache stale; the next refresh reads.

check

check(category: str | None) -> str | None

Judge category against the cached gates: the refusal reason, or None to admit. Never raises.

No category, or an unknown one, is admitted. Before any read has succeeded, a category is refused as gate_unread (or admitted, when built fail_open), and market_gate_unread names which.

admit

admit(role: Role) -> str | None

call.Hooks.admit: judge the role's category (check). Never raises.

LastKnown

LastKnown(
    read: Callable[[], Awaitable[T]],
    initial: T,
    *,
    name: str = "value",
    ttl_s: float = 10.0,
    clock: Callable[[], float] = monotonic,
    on_event: EventHook | None = None
)

A value read through read at most once per ttl_s, serving the last known one on a failure.

Parameters:

  • read (Callable[[], Awaitable[T]]) –

    The store read.

  • initial (T) –

    The value served until a read succeeds.

  • name (str, default: 'value' ) –

    Names the value in market_read_failed.

  • ttl_s (float, default: 10.0 ) –

    Seconds a read (or a failed read) stays fresh.

  • clock (Callable[[], float], default: monotonic ) –

    Monotonic seconds; injectable for tests.

  • on_event (EventHook | None, default: None ) –

    Receives market_read_failed; without one, the stdlib logger llm_ports.market.

known property

known: bool

Whether a read has ever succeeded (or a value was primed): False means value is initial.

value property

value: T

The last known value, without a read.

fresh property

fresh: bool

Whether the last read (or failed read) is younger than ttl_s.

get async

get() -> T

The value: cached while fresh, else read; the last known one when the read fails. Never raises.

prime

prime(value: T) -> None

Set the value and mark it fresh (a warm start, a test).

invalidate

invalidate() -> None

Mark the value stale; the next get reads.

ActionMirror

Bases: Protocol

A consumer's store copy of the registry.

sync_action async

sync_action(spec: ActionSpec) -> None

Upsert spec's CODE_OWNED fields; seed OPERATOR_TUNED ones only on first insert.

Raises:

  • Exception –

    Any store failure; sync_all catches and reports it.

SyncReport dataclass

SyncReport(
    synced: tuple[str, ...],
    failed: tuple[tuple[str, str], ...],
)

What one sync_all pass did.

Attributes:

  • synced (tuple[str, ...]) –

    Names of the specs synced.

  • failed (tuple[tuple[str, str], ...]) –

    (name, reason) for each spec whose sync raised.

MirrorRow dataclass

MirrorRow(
    name: str, enabled: bool, origin: str | None = None
)

One row of the consumer's mirror, as prune_mirror reads it.

Attributes:

  • name (str) –

    The action's feature.action key.

  • enabled (bool) –

    The operator's switch.

  • origin (str | None) –

    The pack (or module) that declared it, when the table records one; else None.

PrunableMirror

Bases: ActionMirror, Protocol

An ActionMirror that can list its rows and retire one.

mirrored async

mirrored() -> Sequence[MirrorRow]

Every row of the mirror.

Raises:

  • Exception –

    Any store failure; prune_mirror catches and reports it.

retire_action async

retire_action(name: str) -> None

Retire the row of an action no longer declared: delete it, or set it disabled.

Raises:

  • Exception –

    Any store failure; prune_mirror catches and reports it.

PruneReport dataclass

PruneReport(
    pruned: tuple[str, ...] = (),
    held: tuple[str, ...] = (),
    failed: tuple[tuple[str, str], ...] = (),
    error: str | None = None,
)

What one prune_mirror pass did.

Attributes:

  • pruned (tuple[str, ...]) –

    Names of the rows retired.

  • held (tuple[str, ...]) –

    Names of unregistered enabled rows NOT retired because a pack failed to load and the row's origin is unknown or is that pack's.

  • failed (tuple[tuple[str, str], ...]) –

    (name, reason) for each row whose retire raised.

  • error (str | None) –

    Why the mirror could not be read; None when it was.

count property

count: int

How many rows were retired.

ActionRegistry

ActionRegistry(
    name: str = "actions",
    *,
    clocks: tuple[str, ...] = SILENCE_CLOCKS,
    on_event: EventHook | None = None,
    sources: Collection[str] | None = None
)

Bases: MutableMapping[Key, ActionSpec]

Insertion-ordered (feature, action) -> ActionSpec table with the declare-time guard.

Parameters:

  • name (str, default: 'actions' ) –

    Names the registry in errors and events (a pack's name, or the process's).

  • clocks (tuple[str, ...], default: SILENCE_CLOCKS ) –

    The silence clocks a spec may declare.

  • on_event (EventHook | None, default: None ) –

    Receives market_action_refused; without one, the stdlib logger llm_ports.market.

  • sources (Collection[str] | None, default: None ) –

    The source ids a spec may name (the consumer's lane registry); None checks none.

clocks property

clocks: tuple[str, ...]

The silence clocks a spec may declare.

frozen property

frozen: bool

Whether freeze has run.

freeze

freeze() -> None

Close the registry: a later add, include or item write raises RuntimeError.

origin

origin(feature: str, action: str) -> str | None

Who declared feature.action: a pack's name, or the handler's module path; None if unknown.

add

add(
    spec: ActionSpec, *, origin: str | None = None
) -> ActionSpec

Declare spec and return it.

Parameters:

  • spec (ActionSpec) –

    The declaration.

  • origin (str | None, default: None ) –

    Who declares it, named in a duplicate's error; defaults to the handler's path.

Raises:

  • ValueError –

    The spec is refused (refusal).

  • DuplicateAction –

    A different spec object holds its key.

  • RuntimeError –

    The registry is frozen.

action

action(
    *,
    feature: str,
    action: str,
    model: str | None = None,
    lane: int = 2,
    value_weight: float = 1.0,
    desired_ttl: str | None = None,
    est_cost_s: float = 40.0,
    cadence: str | None = None,
    cache_table: str | None = None,
    age_col: str | None = None,
    stale_fn: StaleFn | None = None,
    source: str | Iterable[str] | None = None,
    max_silence_h: float | None = None,
    silence_clock: str = "wall"
) -> Callable[[H], H]

Return a decorator that declares the handler it wraps; the handler is returned unchanged.

The arguments are ActionSpec's fields; source also takes one id as a string.

Raises:

  • ValueError –

    At decoration, when the spec is refused or another spec holds the key (DuplicateAction).

include

include(
    other: ActionRegistry, *, origin: str | None = None
) -> tuple[ActionSpec, ...]

Add every spec of other (a pack), all or nothing, and return the specs added.

Parameters:

  • other (ActionRegistry) –

    The pack.

  • origin (str | None, default: None ) –

    Who contributes it, named in a duplicate's error; defaults to other.name.

Raises:

  • ValueError –

    A spec of other is refused here (a clock or source this registry does not accept); nothing is added.

  • DuplicateAction –

    A different spec object holds one of its keys; nothing is added.

  • RuntimeError –

    The registry is frozen.

registered

registered() -> list[ActionSpec]

Every spec, in declaration order.

find

find(feature: str, action: str) -> ActionSpec | None

The spec under (feature, action), or None.

empty

empty() -> ActionRegistry

A fresh, open registry with the same name, clocks, sources and event hook (test isolation).

DuplicateAction

Bases: ValueError

A feature.action key declared twice with different spec objects; the message names both origins.

ActionSpec dataclass

ActionSpec(
    feature: str,
    action: str,
    handler_path: str,
    fn: Handler,
    model: str | None = None,
    lane: int = 2,
    value_weight: float = 1.0,
    desired_ttl: str | None = None,
    est_cost_s: float = 40.0,
    cadence: str | None = None,
    cache_table: str | None = None,
    age_col: str | None = None,
    stale_fn: StaleFn | None = None,
    max_silence_h: float | None = None,
    silence_clock: str = "wall",
    source: tuple[str, ...] = (),
)

A background action: a handler, what makes it due, and how it is ranked and judged.

Attributes:

  • feature (str) –

    The feature that owns the action (the first half of its key).

  • action (str) –

    The action's name within the feature (the second half of its key).

  • handler_path (str) –

    module.qualname of the handler, recorded as provenance.

  • fn (Handler) –

    The handler. Not compared: two imports of one module declare equal specs.

  • model (str | None) –

    The model alias the handler calls, recorded for the operator; None when it calls none.

  • lane (int) –

    Priority lane, 0 first; a scheduler claims a lower lane before a higher one.

  • value_weight (float) –

    Relative worth inside a lane; a fair-share scheduler divides by it.

  • desired_ttl (str | None) –

    How long an output stays fresh, as an interval literal ("18h").

  • est_cost_s (float) –

    Expected seconds per dispatch; a scheduler scales its timeout and lease by it.

  • cadence (str | None) –

    How often the action is expected to run, as an interval literal.

  • cache_table (str | None) –

    Where the action's output lands, for a freshness check to read.

  • age_col (str | None) –

    The column of cache_table that dates an output.

  • stale_fn (StaleFn | None) –

    Yields the work keys currently due; None for an action fed only by events. Not compared.

  • max_silence_h (float | None) –

    Hours of output silence past which an action with no desired_ttl and no cadence is stale: a property of how often its input arrives.

  • silence_clock (str) –

    The clock silence is counted on (SILENCE_CLOCKS, or the registry's own).

  • source (tuple[str, ...]) –

    Ids of the content sources the action reads or writes; a consumer skips the action while any of them is switched off. Empty when it depends on none.

key property

key: tuple[str, str]

(feature, action), the registry key.

name property

name: str

feature.action, the name a kill switch and an operator use.

ttl property

ttl: timedelta | None

desired_ttl parsed, or None.

cadence_interval property

cadence_interval: timedelta | None

cadence parsed, or None.

limit_h property

limit_h: float | None

The declared freshness limit in hours: desired_ttl, else cadence, else None.

None means max_silence_h is the rule.

KillSwitch

KillSwitch(
    read: Callable[[], str | None],
    *,
    default: Iterable[str] = (),
    on_event: EventHook | None = None,
    registry: ActionRegistry | None = None
)

The paused set, read once from a consumer's setting.

Parameters:

  • read (Callable[[], str | None]) –

    Returns the raw setting, or None when it is unset.

  • default (Iterable[str], default: () ) –

    The paused set while the setting is unset.

  • on_event (EventHook | None, default: None ) –

    Receives market_actions_disabled and market_kill_unknown; without one, the stdlib logger llm_ports.market.

  • registry (ActionRegistry | None, default: None ) –

    The actions the names must match; each parse runs unknown against it. None checks nothing (call unknown yourself).

default property

default: frozenset[str]

The paused set while the setting is unset.

names

names() -> frozenset[str]

The paused feature.action names, parsed on first use.

is_disabled

is_disabled(feature: str, action: str) -> bool

Whether feature.action is paused.

unknown

unknown(registry: ActionRegistry) -> frozenset[str]

Paused names that match no action in registry: a typo pauses nothing.

A non-empty result is emitted as market_kill_unknown (WARNING).

reset

reset() -> None

Forget the parsed set; the next read parses the setting again (and emits again).

sync_all async

sync_all(
    registry: ActionRegistry,
    mirror: ActionMirror,
    *,
    on_event: EventHook | None = None
) -> SyncReport

Sync every spec of registry into mirror. Never raises.

Parameters:

  • registry (ActionRegistry) –

    The declared actions.

  • mirror (ActionMirror) –

    The consumer's store copy.

  • on_event (EventHook | None, default: None ) –

    Receives market_sync_failed; without one, the stdlib logger llm_ports.market.

Returns:

  • SyncReport –

    The names synced and the failures, in declaration order.

prune_mirror async

prune_mirror(
    registry: ActionRegistry,
    mirror: PrunableMirror,
    *,
    unloaded: Iterable[str] = (),
    on_event: EventHook | None = None
) -> PruneReport

Retire each enabled mirror row whose action registry does not hold. Never raises.

Parameters:

  • registry (ActionRegistry) –

    The actions this process declares (after every pack loaded).

  • mirror (PrunableMirror) –

    The consumer's store copy.

  • unloaded (Iterable[str], default: () ) –

    The packs that did NOT load in this process (failed or blocked: with adapters.pluggy.load_packs, the name of each PackLoad whose outcome is not ok). Their rows are held; see the module docstring.

  • on_event (EventHook | None, default: None ) –

    Receives market_action_pruned, market_prune_held and market_prune_failed; without one, the stdlib logger llm_ports.market.

Returns:

  • PruneReport –

    The rows retired, held and failed, in the mirror's order.

parse_interval

parse_interval(literal: str) -> timedelta

Parse a short interval literal: an integer and one unit of s, m, h or d.

Parameters:

  • literal (str) –

    "90s", "30m", "18h", "7d"; surrounding whitespace is ignored.

Returns:

  • timedelta –

    The interval.

Raises:

  • ValueError –

    literal is not <int><s|m|h|d>.

refusal

refusal(
    spec: ActionSpec,
    clocks: Collection[str] = SILENCE_CLOCKS,
    sources: Collection[str] | None = None,
) -> str | None

Return why spec cannot be declared, or None when it can.

Parameters:

  • spec (ActionSpec) –

    The declaration.

  • clocks (Collection[str], default: SILENCE_CLOCKS ) –

    The silence clocks the registry accepts.

  • sources (Collection[str] | None, default: None ) –

    The source ids that exist (the consumer's lane registry); None checks none.

Returns:

  • str | None –

    The reason, naming the field; None for a declarable spec. Never raises.

parse_names

parse_names(raw: str) -> frozenset[str]

Split a comma-separated list of feature.action names; blanks are dropped.

Parameters:

  • raw (str) –

    The setting's value.

Returns:

  • frozenset[str] –

    The names, stripped. Never raises.