Skip to content

llm_ports.rag

llm_ports.rag

Retrieval helpers: text chunking (chunking) and retrieval metrics (metrics).

llm_ports.rag.chunking

Split a document for embedding, and decide whether it needs re-embedding at all.

Example

chunks = chunk_text(doc, header="Handbook, 'Returns policy': ") if should_ingest(stored_hash, content_hash(header + doc)): ... # else skip: nothing changed budget = max_chars_for_chunks(12) # the longest text guaranteed to fit 12 chunks

Pure and deterministic. chunk_text cuts windows of about size characters with overlap characters of continuity. It prefers a paragraph break, then a line break, then a sentence end, but only inside the tail third of the window (break_floor), so a chunk stays near its target size and rarely cuts mid-thought. max_chars_for_chunks is the exact worst case of that walk, derived from the same floor, so a caller's character budget for "at most n chunks" cannot drift from the chunker.

content_hash

content_hash(text: str) -> str

Return the full sha256 hex of the text: the change detector (same hash, skip the re-embed).

should_ingest

should_ingest(
    existing_hash: str | None, new_hash: str
) -> bool

Return whether to ingest: the document is new or its content changed.

break_floor

break_floor(size: int) -> int

Return the offset in a window at or after which chunk_text may break: its tail third.

max_chars_for_chunks

max_chars_for_chunks(
    n: int,
    *,
    size: int = CHUNK_CHARS,
    overlap: int = CHUNK_OVERLAP
) -> int

Return the longest header-free text chunk_text is guaranteed to split into at most n chunks.

In the worst case every window breaks at the floor on a one-character separator, so each chunk advances the start by floor + 1 - overlap; the last window then covers size. The bound is tight: an adversarial text one character longer can need n + 1 chunks.

chunk_text

chunk_text(
    text: str,
    *,
    size: int = CHUNK_CHARS,
    overlap: int = CHUNK_OVERLAP,
    header: str = ""
) -> list[str]

Split text into chunks of about size characters with overlap characters of continuity.

A break prefers a paragraph boundary (a blank line), then a line break, then a sentence end, inside the tail third of the window. A text no longer than size is a single chunk; a blank one is no chunks. header is prepended to every chunk so no chunk floats free of its provenance; a chunk may exceed size by the header's length.

llm_ports.rag.metrics

Score a ranked retrieval against what was relevant, and compare two runs.

Example

rank = first_rank({"doc-7", "doc-9"}, ranked) # 1-based; None means never retrieved hit_at_k(relevant, ranked, 8), reciprocal_rank(relevant, ranked) agree_at_k(old_ranked, new_ranked, 10) # Jaccard of the two top-k sets clears(new_hit10, old_hit10, tolerance=0.05) # a parity gate: new >= old - tolerance

Pure. relevant is an any-of set: one relevant id in the top k is a hit, and the reciprocal rank is of the first relevant id. Averages are the caller's (mean), so a per-query row stays inspectable.

first_rank

first_rank(
    relevant: Collection[str], ranked: Sequence[str]
) -> int | None

Return the 1-based rank of the first relevant id in ranked; None when none was retrieved.

hit_at_k

hit_at_k(
    relevant: Collection[str], ranked: Sequence[str], k: int
) -> bool

Return whether any relevant id is in the top k.

reciprocal_rank

reciprocal_rank(
    relevant: Collection[str], ranked: Sequence[str]
) -> float

Return 1 / rank of the first relevant id; 0.0 when none was retrieved (MRR's per-query term).

agree_at_k

agree_at_k(
    a: Sequence[str], b: Sequence[str], k: int
) -> float

Return the Jaccard similarity of two rankings' top-k id sets.

Two empty top-k sets agree on nothing: 0.0.

mean

mean(values: Iterable[float]) -> float

Return the arithmetic mean.

Raises:

  • ValueError –

    On no values; a metric over zero queries is undefined, not 0.

clears

clears(
    new: float,
    baseline: float,
    *,
    tolerance: float,
    places: int = 4
) -> bool

Return whether new clears the parity gate new >= baseline - tolerance.

The floor is rounded to places, so a metric reported at that precision compares as written (0.9 - 0.05 is not 0.85 in float).