osrforge.content
Stage 2: the content pass — per-level batched extraction of keyed areas.
Batches are sliding windows over each level's sorted source-page set, overlapping by one page so an area whose text spans a batch boundary is fully visible to its owning batch. Each area belongs to exactly one batch, enforced through the batch schema's key enum — the model cannot invent, misspell, or duplicate a key the code didn't ask for, so the cross-batch merge is pure concatenation and a duplicate key across batches is a code bug, not a model behavior. Every prompt rule is pinned against observed junk in recorded extraction runs; a prompt edit strands the recorded fixtures and re-runs the eval sweep — see the re-record rule before changing one.
CONTENT_SYSTEM
module-attribute
CONTENT_SYSTEM = 'You extract keyed dungeon areas from tabletop adventure module pages. The user message names the areas to extract, then interleaves each page\'s extracted text (each headed by a [page N] marker) with that page\'s image. Return one entry per named area, using exactly the canonical keys given.\n\nRules:\n- "encounters" are the area\'s hostile or monstrous inhabitants with stat blocks. Ordinary townsfolk and named NPCs without a creature type are not encounters — they belong in the description. Use the creature type as printed ("goblin"), putting proper names ("Snagg") in the description. Wandering-monster tables are not keyed encounters.\n- Encounter counts: put a plain number in "count_fixed" and dice notation like "2d4" in "count_dice"; anything the module says about count that fits neither (rates, conditions, ranges) goes in "count_note". Leave unused count fields null.\n- "trap" describes the area\'s trap, null when it has none. "features" lists notable fixtures a referee should know about (altars, pools, levers); "treasure" lists the area\'s treasure as printed. A secret door that conceals a way to another keyed area is a connection with via "secret_door", not a feature; a secret door concealing no keyed connection (a false closet) stays a feature.\n- "connections": derive each connection\'s "direction" from the level map when the text is silent; use "unknown" only when neither the text nor the map says. "via" is the connection\'s stated mechanism — a door, secret door, stairs, trapdoor, or chute — "passage" when the text names none, "other" for anything else; "door_stuck" and "door_locked" record the door conditions the text states.\n- Connection targets: prefer "to_key" — the connected area\'s canonical key when it appears in the area list, otherwise its printed label. When the text states only a level, not a keyed area ("stairs descend to the second level"), leave "to_key" null and put the level number in "to_level".\n- "source_pages" refer to the [page N] markers in this request, never to page numbers printed on the pages.\n- "confidence" is your self-assessment in [0, 1] of how faithfully you extracted that area.\n'
ContentBatch
dataclass
ContentBatch(
dungeon_id: str,
dungeon_name: str,
level_number: int,
number: int,
pages: tuple[int, ...],
map_pages: tuple[int, ...],
areas: tuple[SurveyArea, ...],
)
One planned content-extraction request.
Attributes:
| Name | Type | Description |
|---|---|---|
dungeon_id |
str
|
The canonical dungeon id. |
dungeon_name |
str
|
The dungeon's printed name, for the prompt. |
level_number |
int
|
The level's number. |
number |
int
|
The 1-based batch number within the level. |
pages |
tuple[int, ...]
|
The batch's own pages — one window over the level's sorted source-page set. Empty when the level's areas have no source pages and the batch carries just the map pages. |
map_pages |
tuple[int, ...]
|
The level's map pages (all of them, ascending). They ride on
every batch — the map is how the model answers |
areas |
tuple[SurveyArea, ...]
|
The survey areas this batch must extract. |
part_pages
property
The request's page order: batch pages, then map pages not already in the batch.
LevelPlan
dataclass
LevelPlan(dungeon_id: str, level_number: int, batches: tuple[ContentBatch, ...])
One level's planned batches — empty when the level needs no model call.
batch_schema
Build one batch's JSON Schema, with key constrained to exactly the given canonical keys.
The enum means the model cannot invent, misspell, or answer a key the code
didn't ask for; a level's area count is far below the proven 512-value
enum budget. The count_dice pattern mirrors osrlib's dice grammar
exactly, so a schema-valid dice string is an osrlib-parseable dice string.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
keys
|
Sequence[str]
|
The canonical area keys this request must extract. |
required |
Returns:
| Type | Description |
|---|---|
dict[str, object]
|
The batch schema. |
build_batch_request
build_batch_request(
batch: ContentBatch, parts: Sequence[TextPart | ImagePart], missing: Sequence[SurveyArea] | None = None
) -> ModelRequest
Build one batch's request — or its missing-key follow-up.
Public and pure so the extraction runner and fixture tests build
fingerprint-identical requests without duplicating prompt code. The
follow-up (missing given) sends the same parts with the prompt naming
just the missing areas and the schema's key enum containing exactly the
missing keys — so it cannot re-answer keys the batch already covered.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
batch
|
ContentBatch
|
The planned batch. |
required |
parts
|
Sequence[TextPart | ImagePart]
|
The batch's page parts, built over
|
required |
missing
|
Sequence[SurveyArea] | None
|
The areas the batch response skipped, for the one follow-up. |
None
|
Returns:
| Type | Description |
|---|---|
ModelRequest
|
The request, tagged |
ModelRequest
|
the follow-up). |
content
content(workdir: Workdir, provider: ModelProvider) -> tuple[LevelContent, ...]
Run stage 2: extract every level's areas and write the per-level caches.
Reads stages/survey.json rather than taking the index as a parameter —
the cache is the contract, matching rerun semantics. Stale
stages/areas.*.json and stages/monsters.json are deleted first (unlike
survey's clear-on-success, because the incremental per-level writes could
otherwise leave a stale mixture), and each level's cache is written as the
level completes, so a mid-stage failure keeps finished levels.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
workdir
|
Workdir
|
A workdir whose survey stage is |
required |
provider
|
ModelProvider
|
The model provider. |
required |
Returns:
| Type | Description |
|---|---|
tuple[LevelContent, ...]
|
Every level's content, in survey order. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the survey stage is not |
ProviderError
|
On provider transport, auth, or rate-limit exhaustion. |
SchemaValidationError
|
If the provider exhausts its schema budget. |
plan_content_batches
plan_content_batches(index: SurveyIndex, batch_pages: int) -> tuple[LevelPlan, ...]
Plan every level's batches, per (dungeon, level) in survey order.
Each level's page set is the union of its areas' source_pages (already
clamped by survey normalization), deduplicated and sorted; batches are
sliding windows over that sorted list with stride batch_pages - 1, so
consecutive batches share exactly one page. Contiguity of the page numbers
themselves is irrelevant — a sparse set windows exactly like a dense one.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
index
|
SurveyIndex
|
The normalized survey index. |
required |
batch_pages
|
int
|
The |
required |
Returns:
| Type | Description |
|---|---|
tuple[LevelPlan, ...]
|
One plan per level, in survey order. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |