Skip to content

LLM referees

An LLM-driven referee (a model that reads the game and decides what happens next) is one of the consumers osrlib is built for. The engine's shape is already the agent loop's shape: typed commands in, typed events out, a full-knowledge view to observe, and a deterministic core that makes every run reproducible. Everything such an agent consumes ships with the library: the schemas, the referee surface, the authored-content story, and the determinism guarantee. The complete program at the end runs as written, and every snippet along the way comes from it.

The schemas are the tool definitions

The reference section ships two raw artifacts alongside its pages: commands.json and events.json. The command surface is a discriminated-union JSON Schema keyed on command_type, and the event surface is one keyed on event_type. They are generated from the same registries the engine executes, so they cannot drift from what a session will actually accept and emit. The command schema reference and the event schema reference render the same schemas page by page for human readers.

The same unions are importable as AnyCommand and AnyEvent, so a Python agent can build its tool definitions in-process instead of shipping files around:

# The whole command surface as one discriminated union: a ready-made tool definition.
tools = TypeAdapter(AnyCommand).json_schema()
assert tools["discriminator"]["propertyName"] == "command_type"
assert len(tools["oneOf"]) == len(ALL_COMMAND_CLASSES)

# The event surface is the matching observation schema, keyed on event_type.
observations = TypeAdapter(AnyEvent).json_schema()
assert observations["discriminator"]["propertyName"] == "event_type"
assert len(observations["oneOf"]) == len(ALL_EVENT_CLASSES)
assert json.loads(json.dumps(tools)) == tools  # plain JSON Schema, ready for a tool registry

The whole command surface and the whole event surface have one discriminator field each. An agent framework that accepts JSON Schema tool definitions can load the command union as-is and let the model emit any command in the game, with validation for free. The loop such an agent runs is short (this is a sketch, not a framework):

# Sketch: the agent loop, framework left to the reader.
while not session.mode.terminal:  # the party fell, or the adventure is won
    observation = session.view(Visibility.REFEREE)
    payload = model.decide(observation, tools)  # the model emits one JSON command
    command = parse_command(payload)
    result = session.execute(command)  # rejected? that's feedback — the model reads why and retries

The referee sees everything

The observation side is GameSession.view with Visibility.REFEREE, which returns a RefereeView: the full session state (party internals, monster hit points, session flags, door states, the complete event log) with exactly two things withheld, the RNG internals and the master seed, which live only in the save document. Each group is a field of its own, typed as the session's own model, so an agent reads view.monsters[0].current_hp and view.flags["key"] off it and serializes the whole observation with view.model_dump(mode="json"). The player view is the opposite discipline, an enumerated whitelist. Views and visibility draws the line precisely.

# The referee view is full state — flags, monster internals — minus RNG state and the seed.
view = session.view(Visibility.REFEREE)
assert view.flags == {"ambush_sprung": True}
assert all(monster.current_hp >= 0 for monster in view.monsters)
dumped = view.model_dump()
assert "master_seed" not in dumped and "rng_streams" not in dumped

The event stream is unfiltered for the same reason. GameSession.execute returns its events unfiltered, and each event is stamped with a visibility: referee-visibility events include the hidden rolls (surprise, reaction, secret-door detection) that a player-facing front end must strip at its wire, as the FastAPI pattern does. An in-process referee agent reads them all. They are its perception of what the dice just did.

# The unfiltered event stream is the observation: referee events carry the hidden rolls.
codes = [event.code for event in result.events]
assert "session.monsters.spawned" in codes
assert any(event.visibility is Visibility.REFEREE for event in result.events)

The authorial surface

Player commands let the model drive the party's turn. Referee commands let it run the table. They use the same envelope and the same rejection discipline as everything else: no separate API, only more entries in the union:

# Referee commands are the authorial surface: record a fact, then spring an ambush.
session.execute(SetFlag(key="ambush_sprung", value=True))
result = session.execute(SpawnMonsters(template_id="goblin", count_fixed=2, distance_feet=30))
assert result.accepted

The rejection contract matters as much here as it does for players: a rejected command changes nothing and explains itself with a machine-readable code (see the rejection code reference), so a model that asks for something illegal gets structured feedback to correct against instead of a stack trace.

Narrate from codes, not prose

Events never include engine-baked prose. Each has a stable message code (a compact fact like session.monsters.spawned or encounter.surprise.rolled) plus typed fields. The message code reference lists every shipped code with its emitting event class and default template, and each event's fields are on its schema page. That is exactly what a narrator model wants: ground truth it can render freely without parsing English back into facts. The one kind of English an event does include is authored narrative, a beat the adventure's author wrote in a structured field, which Narrating authored content below treats separately from the model's own words. When a plain default line is enough, format_message renders one for any event, appending any authored beat verbatim:

# Every event also renders to a default English line the model can lean on.
lines = [format_message(event) for event in result.events]
assert all(lines)

A practical narrator prompt sends the structured events (or their codes and fields) as the facts to narrate, and keeps the model's creativity in the telling. The dice already decided what happened.

Narrating authored content

An adventure written for the authored layer (Gates, triggers, and quests) arrives with material aimed squarely at a narrating referee, and an agent serving one should use all of it.

Register the library's Interpreter and do no bookkeeping. One session.register_listener(Interpreter(session)) after the session is built (and again after a load), and the interpreter plays the triggers, the quests, the fired-marks, and the rewards as ordinary logged commands. The agent referees, and the interpreter runs the adventure's wiring.

guidance is steering, never script. NarrativeBlock has a guidance field on any authored object, and LevelSpec.guidance holds whole-level ambience that hangs on no object at all. The TUI barrow's first level reads:

guidance=(
    "Grave goods, not treasure: the barrow king was buried, not hoarded. "
    "Keep the goblins squalid and the shrine quiet."
),

A referee-side narrator reads these straight off the adventure document it is refereeing and never prints them verbatim, the same trust posture as an area's description prose, which already flows into narration. No event includes guidance and no view ships it. It is the author talking to the narrator.

Authored beats are text to weave, not paraphrase. A quest's offer and completion, an objective's progress, a gate's success and refusal arrive as structured fields on player-visible events and rejections, with speaker attribution when the author wrote one ("the temple almoner"). Those are the table's words: deliver them as written, in the speaker's voice, and put the model's creativity around them rather than over them.

Command.source keeps the agent's hands visible. Every command the interpreter issues is stamped trigger:{id} or quest:{id}, so an agent that stamps its own referee commands, or leaves them unstamped, leaves a log where its choices and the adventure's consequences never blur. That attribution is what completes the eval story below: replay a trajectory and the log itself says which grants were the model's ideas and which were the adventure playing out.

Determinism is the eval story

Every random draw in osrlib comes from a named stream forked from the master seed, so the same seed plus the same command sequence produces the same game, bit for bit. For agent work this is the property that makes everything else tractable: a trajectory (the seed and the list of commands the model chose) is a complete, reproducible record of a run. Re-execute it offline and you get the same events to score. Change a prompt and replay the same seeds to regression-test the change. Diff two models on identical dungeons. Determinism, saves, and replay covers the exact guarantee and its boundary (identical replays are promised only under an identical engine version).

# Determinism is the eval story: same seed, same commands, same trajectory.
replay = new_session(seed=7)
replay.execute(EnterDungeon(dungeon_id="crypt"))
replay.execute(SetFlag(key="ambush_sprung", value=True))
rerun = replay.execute(SpawnMonsters(template_id="goblin", count_fixed=2, distance_feet=30))
assert [e.model_dump(mode="json") for e in rerun.events] == [e.model_dump(mode="json") for e in result.events]

The complete program

import json

from pydantic import TypeAdapter

from osrlib.core.alignment import Alignment
from osrlib.core.character import CHARACTER_CREATION_STREAM, create_character
from osrlib.core.events import Visibility
from osrlib.core.rng import RngStreams
from osrlib.core.ruleset import Ruleset
from osrlib.crawl.adventure import Adventure, TownSpec
from osrlib.crawl.commands import (
    ALL_COMMAND_CLASSES,
    AnyCommand,
    EnterDungeon,
    SetFlag,
    SpawnMonsters,
)
from osrlib.crawl.dungeon import DungeonSpec, Edge, EdgeKind, LevelSpec
from osrlib.crawl.events import ALL_EVENT_CLASSES, AnyEvent
from osrlib.crawl.party import Party
from osrlib.crawl.session import GameSession
from osrlib.messages import format_message

# The whole command surface as one discriminated union: a ready-made tool definition.
tools = TypeAdapter(AnyCommand).json_schema()
assert tools["discriminator"]["propertyName"] == "command_type"
assert len(tools["oneOf"]) == len(ALL_COMMAND_CLASSES)

# The event surface is the matching observation schema, keyed on event_type.
observations = TypeAdapter(AnyEvent).json_schema()
assert observations["discriminator"]["propertyName"] == "event_type"
assert len(observations["oneOf"]) == len(ALL_EVENT_CLASSES)
assert json.loads(json.dumps(tools)) == tools  # plain JSON Schema, ready for a tool registry


def new_session(seed: int) -> GameSession:
    """One tiny two-cell dungeon and one fighter: enough engine to referee."""
    rules = Ruleset()
    stream = RngStreams(master_seed=seed).get(CHARACTER_CREATION_STREAM)
    hero = create_character(
        name="Hild",
        class_id="fighter",
        alignment=Alignment.LAWFUL,
        ruleset=rules,
        stream=stream,
    )
    level = LevelSpec(
        number=1,
        width=2,
        height=1,
        entrance=(0, 0),
        edges={"1,0:west": Edge(kind=EdgeKind.OPEN)},
    )
    crypt = DungeonSpec(id="crypt", name="The Old Crypt", levels=(level,))
    town = TownSpec(name="Threshold", travel_turns={"crypt": 1})
    adventure = Adventure(name="A First Delve", town=town, dungeons=(crypt,))
    return GameSession.new(Party(members=[hero.character]), adventure, seed=seed)


session = new_session(seed=7)
session.execute(EnterDungeon(dungeon_id="crypt"))

# Referee commands are the authorial surface: record a fact, then spring an ambush.
session.execute(SetFlag(key="ambush_sprung", value=True))
result = session.execute(SpawnMonsters(template_id="goblin", count_fixed=2, distance_feet=30))
assert result.accepted

# The unfiltered event stream is the observation: referee events carry the hidden rolls.
codes = [event.code for event in result.events]
assert "session.monsters.spawned" in codes
assert any(event.visibility is Visibility.REFEREE for event in result.events)

# Every event also renders to a default English line the model can lean on.
lines = [format_message(event) for event in result.events]
assert all(lines)

# The referee view is full state — flags, monster internals — minus RNG state and the seed.
view = session.view(Visibility.REFEREE)
assert view.flags == {"ambush_sprung": True}
assert all(monster.current_hp >= 0 for monster in view.monsters)
dumped = view.model_dump()
assert "master_seed" not in dumped and "rng_streams" not in dumped

# Determinism is the eval story: same seed, same commands, same trajectory.
replay = new_session(seed=7)
replay.execute(EnterDungeon(dungeon_id="crypt"))
replay.execute(SetFlag(key="ambush_sprung", value=True))
rerun = replay.execute(SpawnMonsters(template_id="goblin", count_fixed=2, distance_feet=30))
assert [e.model_dump(mode="json") for e in rerun.events] == [e.model_dump(mode="json") for e in result.events]

Where next