llm_ports.rag¶
llm_ports.rag
¶
Retrieval helpers: text chunking (chunking) and retrieval metrics (metrics).
llm_ports.rag.chunking¶
Split a document for embedding, and decide whether it needs re-embedding at all.
Example
chunks = chunk_text(doc, header="Handbook, 'Returns policy': ") if should_ingest(stored_hash, content_hash(header + doc)): ... # else skip: nothing changed budget = max_chars_for_chunks(12) # the longest text guaranteed to fit 12 chunks
Pure and deterministic. chunk_text cuts windows of about size characters with overlap
characters of continuity. It prefers a paragraph break, then a line break, then a sentence end,
but only inside the tail third of the window (break_floor), so a chunk stays near its target
size and rarely cuts mid-thought. max_chars_for_chunks is the exact worst case of that walk,
derived from the same floor, so a caller's character budget for "at most n chunks" cannot drift
from the chunker.
content_hash
¶
content_hash(text: str) -> str
Return the full sha256 hex of the text: the change detector (same hash, skip the re-embed).
should_ingest
¶
should_ingest(
existing_hash: str | None, new_hash: str
) -> bool
Return whether to ingest: the document is new or its content changed.
break_floor
¶
break_floor(size: int) -> int
Return the offset in a window at or after which chunk_text may break: its tail third.
max_chars_for_chunks
¶
max_chars_for_chunks(
n: int,
*,
size: int = CHUNK_CHARS,
overlap: int = CHUNK_OVERLAP
) -> int
Return the longest header-free text chunk_text is guaranteed to split into at most n chunks.
In the worst case every window breaks at the floor on a one-character separator, so each chunk
advances the start by floor + 1 - overlap; the last window then covers size. The bound is
tight: an adversarial text one character longer can need n + 1 chunks.
chunk_text
¶
chunk_text(
text: str,
*,
size: int = CHUNK_CHARS,
overlap: int = CHUNK_OVERLAP,
header: str = ""
) -> list[str]
Split text into chunks of about size characters with overlap characters of continuity.
A break prefers a paragraph boundary (a blank line), then a line break, then a sentence end,
inside the tail third of the window. A text no longer than size is a single chunk; a blank one
is no chunks. header is prepended to every chunk so no chunk floats free of its provenance; a
chunk may exceed size by the header's length.
llm_ports.rag.metrics¶
Score a ranked retrieval against what was relevant, and compare two runs.
Example
rank = first_rank({"doc-7", "doc-9"}, ranked) # 1-based; None means never retrieved hit_at_k(relevant, ranked, 8), reciprocal_rank(relevant, ranked) agree_at_k(old_ranked, new_ranked, 10) # Jaccard of the two top-k sets clears(new_hit10, old_hit10, tolerance=0.05) # a parity gate: new >= old - tolerance
Pure. relevant is an any-of set: one relevant id in the top k is a hit, and the reciprocal rank
is of the first relevant id. Averages are the caller's (mean), so a per-query row stays
inspectable.
first_rank
¶
first_rank(
relevant: Collection[str], ranked: Sequence[str]
) -> int | None
Return the 1-based rank of the first relevant id in ranked; None when none was retrieved.
hit_at_k
¶
hit_at_k(
relevant: Collection[str], ranked: Sequence[str], k: int
) -> bool
Return whether any relevant id is in the top k.
reciprocal_rank
¶
reciprocal_rank(
relevant: Collection[str], ranked: Sequence[str]
) -> float
Return 1 / rank of the first relevant id; 0.0 when none was retrieved (MRR's per-query term).
agree_at_k
¶
agree_at_k(
a: Sequence[str], b: Sequence[str], k: int
) -> float
Return the Jaccard similarity of two rankings' top-k id sets.
Two empty top-k sets agree on nothing: 0.0.
mean
¶
mean(values: Iterable[float]) -> float
Return the arithmetic mean.
Raises:
-
ValueError–On no values; a metric over zero queries is undefined, not 0.
clears
¶
clears(
new: float,
baseline: float,
*,
tolerance: float,
places: int = 4
) -> bool
Return whether new clears the parity gate new >= baseline - tolerance.
The floor is rounded to places, so a metric reported at that precision compares as written
(0.9 - 0.05 is not 0.85 in float).