Skip to content

llm_ports.prompts

llm_ports.prompts

Prompt packs named by their content sha and their identity, their manifest, its resolution, and RepoPromptSource.

PackManifest

Bases: BaseModel

A validated, frozen pack.yaml.

Attributes:

  • id (str) –

    feature.action. Equals the pack's directory name and the managed prompt's name, so the stamp on an output row, the prompt-store link and the repo path agree by name.

  • version (str) –

    Semver. MAJOR: the output schema changed. MINOR: an instruction or claim-rule changed. PATCH: wording. The content sha, computed by the loader and never written here, says whether the text changed; the version says what kind of change it was.

  • role (str) –

    The model alias or Role name the pack is written for.

  • templates (dict[str, str]) –

    Template name to a .j2 file inside the pack directory. prompt is the primary template and is required. The content sha covers every template in manifest order, and a single-template pack hashes that template alone.

  • output_schema (str | None) –

    module:attr (or module.attr) path of the pydantic class or JsonOutput the output parses as; None for prose. prompts.resolve imports it from the roots its caller names.

  • graders (list[str]) –

    Grader names, looked up by prompts.resolve in the caller's grader registry. A dotted path is accepted as a name. A pack with graders is evaluable and requires golden.

  • golden (str | None) –

    Path of the golden set the graders replay, relative to the pack directory.

  • langfuse_label (str) –

    The label a managed mirror serves under (YAML key langfuse_label or its neutral alias label). The managed copy mirrors the repo; it never leads.

  • includes (list[str]) –

    Files a template reaches by {% include %}, {% import %} or {% extends %} that are not in templates: relative POSIX paths inside the pack directory, any extension (partials/header.j2, rubric.md). With templates, the closed set: no other file is ever rendered.

  • schema_version (int) –

    The manifest schema the pack is written to; only SCHEMA_VERSION loads.

Raises:

  • ValidationError –

    A field is unknown, malformed, or graders lacks golden.

Rationale: ADR-0008.

feature property

feature: str

The feature half of id.

label property

label: str

The managed mirror's label; the neutral name of langfuse_label.

evaluable property

evaluable: bool

True when the pack declares graders and a golden set, so an eval can grade it.

Pack dataclass

Pack(
    manifest: PackManifest,
    dir: Path,
    sources: dict[str, str],
    content_sha: str,
    blobs: Mapping[str, bytes] = dict(),
)

One loaded prompt pack. load builds it.

Attributes:

  • manifest (PackManifest) –

    The validated manifest.

  • dir (Path) –

    The pack's directory.

  • sources (dict[str, str]) –

    Template name to source text, in manifest order.

  • content_sha (str) –

    The content_sha of the sources.

  • blobs (Mapping[str, bytes]) –

    Relative path to the bytes read at load: pack.yaml and every file of the closed set. A pack built by hand without them has an identity over its templates alone.

id property

id: str

The pack id, feature.action.

source property

source: str

The primary template's source; a single-template pack's sha is over this text.

identity cached property

identity: str

byte_sha over the manifest's bytes, then each file of the closed set by sorted path.

Rationale: ADR-0026.

files

files() -> tuple[str, ...]

Return the closed template set: every file a render may read, sorted, relative to dir.

stamp

stamp() -> dict[str, str]

Return the stamp a writer stores beside an output row: {pack, version, sha, identity}.

The version says what kind of change the pack had; sha says whether its template text changed; identity says whether anything that changes its behaviour did.

stamp_str

stamp_str() -> str

Return the stamp as one string, <id>@<version>+<sha>.

render

render(name: str = PRIMARY_TEMPLATE, /, **ctx: Any) -> str

Render the template name of this pack with ctx, in the environment built at load.

Raises:

  • UndefinedError –

    The template reads a variable ctx does not supply.

render_source

render_source(source: str, **ctx: Any) -> str

Render candidate template text under this pack's environment and closed set.

Lets an eval grade a template revision that is not yet the pack's installed source.

Raises:

  • TemplateNotFound –

    The candidate includes a file outside the closed set.

seed

seed(name: str = PRIMARY_TEMPLATE) -> Seed

Return the repo Seed of template name, the text a managed mirror copies.

sha is the pack's content_sha. config carries pack, version, role, template, content_sha, identity and label (the manifest's mirror label), so a mirror holding a stale identity is re-pushed and serves under the manifest's label.

Raises:

  • KeyError –

    The pack has no template name.

resolved

resolved(
    *,
    registry: GraderRegistry | Registry | None = None,
    import_roots: Sequence[str] = ()
) -> ResolvedPack

Return prompts.resolve(self, registry=registry, import_roots=import_roots).

Raises:

  • PromptPackInvalid –

    A manifest pointer does not resolve; the message names the field.

PromptPackInvalid

Bases: ValueError

A pack that cannot be served. The message names the pack and the defect.

RepoPromptSource

RepoPromptSource(seed_for: Callable[[str], Seed])

A PromptSource for a process with no managed store.

Every name is its repo seed (seed_for, the same callable a managed-store adapter mirrors), served as Prompt(repo) with the seed's config and system template: what a managed adapter serves when its store is down, without the round-trip.

get

get(name: str, *, label: str = 'production') -> Prompt

Return the repo seed for name as a prompt.

label is accepted and ignored: a repo has one copy. A name without a seed raises from seed_for (the port's law: no seed, no text).

GraderRegistry

Bases: Protocol

Grader names to graders, owned by the consumer's composition root.

get

get(name: str) -> AnyGrader

Return the grader named name.

Raises:

  • KeyError –

    No grader has that name.

ResolvedPack dataclass

ResolvedPack(
    pack: Pack,
    output: type[BaseModel] | type[JsonOutput] | None,
    graders: Mapping[str, AnyGrader],
    golden: Path | None,
    output_schema_sha: str | None,
)

A pack with its manifest's pointers resolved.

Attributes:

  • pack (Pack) –

    The pack.

  • output (type[BaseModel] | type[JsonOutput] | None) –

    The imported output type, or None for a prose pack.

  • graders (Mapping[str, AnyGrader]) –

    The manifest's grader names, in manifest order, to the graders they resolved to.

  • golden (Path | None) –

    The golden set's path inside the pack directory, or None when the pack has none. Not checked for existence: evals.golden.fire_pack reports a missing file.

  • output_schema_sha (str | None) –

    byte_sha of the canonical JSON schema of output; None with no output.

byte_sha

byte_sha(
    parts: Sequence[tuple[str, bytes]], *, n: int = 16
) -> str

Return sha256 over the named parts, length-prefixed, in order; the first n hex characters.

Parameters:

  • parts (Sequence[tuple[str, bytes]]) –

    (name, data) pairs. The order is part of the input: the caller sorts when the order carries no meaning.

  • n (int, default: 16 ) –

    Hex characters kept, at most 64.

Returns:

  • str –

    The digest, n lowercase hex characters.

Raises:

  • ValueError –

    A part name contains NUL, which would make the encoding ambiguous.

content_sha

content_sha(sources: list[str]) -> str

Return sha over the sources joined by one newline; a single source hashes as itself.

load

load(pack_id: str, root: Path) -> Pack

Read, validate and fingerprint the pack pack_id under root.

Raises:

  • PromptPackInvalid –

    The manifest is missing, unreadable or invalid, or its id is not the directory name; a file of the closed set is missing, not UTF-8 or does not parse; a template references a file outside the closed set or includes dynamically; or a pack file holds a carriage return. The message names the pack and the defect.

registry

registry(
    root: Path,
    *,
    registry: GraderRegistry | Registry | None = None,
    import_roots: Sequence[str] | None = None
) -> tuple[Pack, ...]

Load every pack under root, strictly.

Suits a boot path or a test that every pack loads. Given registry or import_roots, every pack is also resolved with them (scan), so a mistyped grader or output schema fails here. A runtime reader that must keep serving the valid packs uses scan instead.

Raises:

scan

scan(
    root: Path,
    *,
    registry: GraderRegistry | Registry | None = None,
    import_roots: Sequence[str] | None = None
) -> tuple[tuple[Pack, ...], dict[str, str]]

Load every pack directory under root, keeping the ones that fail by name.

A directory with no manifest is not a pack and is not reported. A missing root returns ((), {}). Given registry or import_roots, each loaded pack is also resolved with them (prompts.resolve), and a pack that does not resolve is reported beside the ones that do not load.

Returns:

  • tuple[tuple[Pack, ...], dict[str, str]] –

    The loaded packs, and {id: reason} for each invalid pack.

served_shas

served_shas(
    root: Path,
    *,
    key: Literal["content_sha", "identity"] = "content_sha"
) -> dict[str, str]

Return {pack id: content sha} (or identity) for every loadable pack: the served side of unevaluated.

sha

sha(text: str, *, n: int = 16) -> str

Return the sha256 hex digest of text, cut to n characters.

Invariants
  • The algorithm never changes: stamps built on it are persisted.

unevaluated

unevaluated(
    served: dict[str, str], evaluated: dict[str, str]
) -> list[str]

Return the pack ids whose served name is not the name of their newest pass or fail grade.

A changed pack reads as due until it has been graded. This is the one definition of "unevaluated"; every reader uses it. Both sides name packs the same way: content shas, or identities (served_shas(root, key="identity")), never one of each.

Parameters:

  • served (dict[str, str]) –

    {pack id: content sha or identity} as served (see served_shas).

  • evaluated (dict[str, str]) –

    {pack id: the same} of each pack's newest pass or fail grade.

resolve

resolve(
    pack: Pack,
    *,
    registry: GraderRegistry | Registry | None = None,
    import_roots: Sequence[str] = ()
) -> ResolvedPack

Resolve the pack's output_schema, graders and golden, strictly.

Parameters:

  • pack (Pack) –

    A loaded pack.

  • registry (GraderRegistry | Registry | None, default: None ) –

    Where grader names resolve: a GraderRegistry or a plain mapping. None resolves no grader, so a pack that names one does not resolve.

  • import_roots (Sequence[str], default: () ) –

    Module prefixes an output_schema may be imported from; empty imports nothing.

Returns:

Raises:

  • PromptPackInvalid –

    A pointer does not resolve; the message names the pack and the field.

schema_sha

schema_sha(
    output: type[BaseModel] | type[JsonOutput],
) -> str

Return byte_sha of the output type's JSON schema, serialized canonically (sorted keys, compact).

Pure. A JsonOutput hashes the schema it carries; a pydantic model hashes model_json_schema().