Train your agent against a Mock Machines world.

A reinforcement-learning agent needs an environment: something it can reset, step, and observe. Writing that environment in Python is the usual approach — and the slow one. Mock Machines lets you declare the environment in a scenario YAML instead, then drive it from Python at engine speed.

The engine runs in-process as a CGo shared library loaded through cffi — no network, no subprocess, no server to babysit. Entity state crosses into Python zero-copy over the Apache Arrow C Data Interface, so the step loop is fast enough for millions of environment steps. The package is intentionally minimal: it is not an RL framework, it is the cleanest bridge between Python and the engine. You bring the observation vector, the reward, the policy, and the training loop.

# Prebuilt wheels ship the engine + the mm CLI — no Go toolchain or checkout needed.
uv pip install "mockmachines[notebook]"

The wheel bundles the engine shared library and the mm CLI binary, so no Go toolchain, C compiler, or repository checkout is required to use the package — just pip / uv. The [notebook] extra pulls in PyTorch, TensorBoard, and Jupyter for the worked example below.

import mockmachines as mm

sim = mm.load("scenarios/CliffWalkingRL/CliffWalkingRL.yaml")  # or a dir, .mock, or gs:// URI
sim.reset()                                    # seed a fresh episode (1 walker)
sim.step(target="Walker", event="move_east")   # one deterministic move = one turn
table = sim.observe("Walker")                  # pyarrow.Table of the walker's state
print(table.to_pandas())

scenarios/CliffWalkingRL/ is a single-agent variant of the classic CliffWalking gridworld, tuned for RL. One Walker lives on a 4×12 graph of GridCell entities; the bottom row is a cliff, the bottom-right cell is the goal.

The Walking state puts all its probability mass on "stay", so a plain advance never moves the Walker — movement comes only from a targeted move_north / south / east / west request. Each move's is_not blocked condition rejects an off-grid step, so the engine is a pure, deterministic transition function. There are no terminal states and no reward in the YAML: termination at the goal and the cliff penalty are computed Python-side from the observed cell. One request = one move = one turn.

N_CELLS = 48
ACTIONS = ["move_north", "move_south", "move_east", "move_west"]

class CliffWalkingEnv:
    """A minimal RL environment over a Mock Machines CliffWalkingRL simulation."""

    def __init__(self, scenario):
        self.sim = mm.load(scenario)
        self.kind = {r["ID"]: r["kind"]
                     for r in self.sim.observe("GridCell").to_pylist()}

    def _cell(self):
        return self.sim.observe("Walker").to_pylist()[0]["current_cell_id"]

    def reset(self):
        self.sim.reset(sim_count=1)
        return self._obs()

    def step(self, action):
        self.sim.step(target="Walker", event=ACTIONS[action])  # the action
        kind = self.kind[self._cell()]                          # observe the result
        # reward shaping, cliff penalty, and goal-termination are pure Python here
        ...

On top of that wrapper the notebook trains a standard deep Q-network: a small PyTorch multilayer perceptron mapping the one-hot cell to a Q-value per action, with an experience-replay buffer, a periodically synced target network, ε-greedy exploration, and a Huber TD loss. Returns and loss stream to TensorBoard.

Over a couple of hundred episodes the agent learns to hug the safe edge above the cliff, and the greedy rollout converges on the textbook optimal return of −13. None of the learning logic touches the engine — change the environment by pointing mm.load at a different scenario, and the same training loop applies. The full notebook is python/examples/notebooks/cliffwalking_dqn.ipynb.

• Change the environment, not the code. Point mm.load at any other scenario — or a .mock bundle from a public bucket — and only your observation / action / reward wrapper changes.
• Richer observations. sim.observe(machine) returns a full pyarrow Table; build whatever feature vector your task needs from its columns.
• Run without a checkout. mm.load accepts a gs:// or https://storage.googleapis.com/… URL, so a notebook can pull a published .mock bundle and run anywhere.
• Scale up. The in-process, zero-copy bridge keeps the step loop fast enough for millions of environment steps.