# ydnyshhh/rlvr-gym architecture

> Architecture diagram of the GitHub repository ydnyshhh/rlvr-gym, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ydnyshhh/rlvr-gym
- Repository: https://github.com/ydnyshhh/rlvr-gym

## Overview

1. **Project shape**
   - RLVR-Gym is a **Python library** organized as a core framework plus several task-family modules.
   - The package lives under `src/rlvr_gym`, with a small CLI, reusable core abstractions, family-specific generators, and documentation/artifact outputs.

2. **Primary architectural boundary: core vs. families**
   - `src/rlvr_gym/core` contains the generic infrastructure that every task family uses.
   - `src/rlvr_gym/families` contains the concrete environment families:
     - `deduction_grid`
     - `graph_planning`
     - `scheduling`
     - `sokoban`
     - `symbolic_transformation`
   - This is the main extension boundary: new task types are added as new family modules, not by modifying the runtime model.

3. **Family abstraction and registry**
   - `core/family.py` defines the family abstraction: seeded sampling, environment creation, and config-driven task generation.
   - `registry.py` is the lookup layer that exposes named families to the CLI and library consumers.
   - The README’s `get_family("...")` usage reflects this indirection: callers work with family names, not concrete modules.

4. **Task generation flow**
   - The generation pipeline is consistent across families:
     - sample parameters from `FamilyConfig` and difficulty
     - generate a latent world model
     - derive the objective
     - define observability/rendering
     - attach verifier and reward settings
     - optionally attach an oracle/solver
     - build a runnable environment and exportable task spec
   - This means the repo is centered on **task synthesis**, not on a single fixed simulator.

5. **Runtime and interaction model**
   - `core/runtime.py` provides the lightweight environment loop with `reset()` / `step()` semantics via `RLVREnv`.
   - The runtime records traces and exposes structured observations and valid actions.
   - This is the main execution boundary between generated task definitions and agent interaction.

6. **Verification and reward**
   - `core/verifier.py` encodes feasibility and correctness checks.
   - `core/reward.py` defines how task outcomes are converted into reward signals.
   - The architecture is “verification-first”: correctness is computed from the formal state/rules, not from heuristics or text matching.

7. **Oracle and solution metadata**
   - `core/oracle.py` provides optional exact oracle support.
   - Oracle output is used for rollout generation, label extraction, and proof-style solution metadata.
   - Families that have tractable exact solutions can expose stronger supervision or benchmark labeling.

8. **Export pipeline**
   - `core/exporters.py` is the conversion layer from a sampled task into downstream data formats.
   - It supports multiple research workflows:
     - task specs
     - oracle rollouts
     - SFT examples
     - offline RL transitions
     - oracle label views
     - benchmark splits
   - This is a key data boundary: generation/runtime on one side, consumable datasets and benchmarks on the other.

9. **CLI as the operational entry point**
   - `src/rlvr_gym/cli.py` exposes commands like listing families, sampling tasks, and building benchmark splits.
   - The CLI is a thin wrapper over the core/family/export APIs, so it does not define a separate execution model.
   - The repository is intended to be run both as a library and as a command-line task factory.

10. **Family implementations**
   - Each family module encodes its own latent problem structure and verification logic:
     - `graph_planning`: weighted graph shortest-path style tasks
     - `scheduling`: single-machine scheduling with deadlines and precedence constraints
     - `sokoban`: solver-backed box-pushing/planning with deadlock diagnostics
     - `symbolic_transformation`: symbolic rewriting and normalization
     - `deduction_grid`: logic-grid deduction with exact table propagation
   - These modules are where domain-specific state, action space, and task difficulty live.

11. **Benchmark generation and frozen artifacts**
   - `scripts/build_benchmark_artifacts.py` likely generates the contents under `docs/benchmark_artifacts`.
   - The benchmark artifact directories include spec files, split manifests, diversity summaries, and baseline results for each family.
   - This shows a second major output path: beyond interactive environments, the repo produces frozen evaluation bundles.

12. **Documentation as an architecture contract**
   - `docs/architecture.md` and the family-specific benchmark cards document intended capability coverage, verification guarantees, oracle semantics, metrics, and shortcut risks.
   - These docs are part of the repo’s architecture because they define how each family should be interpreted and benchmarked.

13. **Testing and validation**
   - `tests/test_rlvr_gym.py` is the main automated check for the package behavior.
   - The test suite verifies that the library, CLI, and family registration/runtime path remain coherent.

14. **Typical data flow to diagram**
   - A caller chooses a family name and seed/config.
   - The registry resolves the family module.
   - The family samples a task instance and builds an `RLVREnv`.
   - The runtime emits observations/actions while verifiers and reward logic evaluate outcomes.
   - Exporters transform the resulting task, trace, or oracle into benchmark or training artifacts.

15. **Technology and operational context**
   - The project is pure Python, packaged with `pyproject.toml`, and managed with `uv`.
   - It is optimized for research workflows around RLVR, offline RL, SFT, rejection sampling, and benchmark construction rather than for a production service stack.

Main entry points: [`src/rlvr_gym/core/family.py`](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/family.py) (Family API), [`src/rlvr_gym/core/runtime.py`](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/runtime.py) (Runtime), [`src/rlvr_gym/core/exporters.py`](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/exporters.py) (Exporters), [`src/rlvr_gym/cli.py`](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/cli.py) (CLI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_package["Python package"]
  node_cli["CLI<br/>entrypoint<br/>[cli.py]"]
  node_registry["Registry<br/>lookup<br/>[registry.py]"]
  node_family_api["Family API<br/>abstraction<br/>[family.py]"]
  node_runtime["Runtime<br/>env loop<br/>[runtime.py]"]
  node_verifier["Verifier<br/>correctness<br/>[verifier.py]"]
  node_reward["Reward<br/>signal<br/>[reward.py]"]
  node_oracle["Oracle<br/>solver<br/>[oracle.py]"]
  node_exporters["Exporters<br/>data export<br/>[exporters.py]"]
  node_types["Types<br/>shared model<br/>[types.py]"]
  node_utils["Utils<br/>helpers<br/>[utils.py]"]
end

subgraph group_families["Task families"]
  node_deduction_grid["Deduction grid<br/>family<br/>[deduction_grid.py]"]
  node_graph_planning["Graph planning<br/>family<br/>[graph_planning.py]"]
  node_scheduling["Scheduling<br/>family<br/>[scheduling.py]"]
  node_sokoban["Sokoban<br/>family<br/>[sokoban.py]"]
  node_symbolic["Symbolic transform<br/>family"]
end

subgraph group_docs["Docs & artifacts"]
  node_bench_cards["Benchmark cards<br/>docs"]
  node_bench_artifacts["Benchmark artifacts<br/>outputs"]
  node_build_script["Build script<br/>artifact builder"]
end

node_cli -->|"resolve family"| node_registry
node_cli -->|"build outputs"| node_exporters
node_registry -->|"binds"| node_family_api
node_family_api -->|"creates env"| node_runtime
node_family_api -->|"attaches checks"| node_verifier
node_family_api -->|"attaches reward"| node_reward
node_family_api -->|"optional solver"| node_oracle
node_runtime -->|"evaluates"| node_verifier
node_runtime -->|"scores"| node_reward
node_oracle -->|"labels"| node_exporters
node_runtime -->|"traces"| node_exporters
node_family_api -->|"task spec"| node_exporters
node_deduction_grid -->|"implements"| node_family_api
node_graph_planning -->|"implements"| node_family_api
node_scheduling -->|"implements"| node_family_api
node_sokoban -->|"implements"| node_family_api
node_symbolic -->|"implements"| node_family_api
node_build_script -->|"generates"| node_bench_artifacts
node_bench_cards -->|"documents"| node_bench_artifacts
node_types -->|"shared models"| node_family_api
node_utils -->|"supports"| node_family_api

click node_cli "https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/cli.py"
click node_registry "https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/registry.py"
click node_family_api "https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/family.py"
click node_runtime "https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/runtime.py"
click node_verifier "https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/verifier.py"
click node_reward "https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/reward.py"
click node_oracle "https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/oracle.py"
click node_exporters "https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/exporters.py"
click node_types "https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/types.py"
click node_utils "https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/utils.py"
click node_deduction_grid "https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/families/deduction_grid.py"
click node_graph_planning "https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/families/graph_planning.py"
click node_scheduling "https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/families/scheduling.py"
click node_sokoban "https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/families/sokoban.py"
click node_symbolic "https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/families/symbolic_transformation.py"
click node_bench_cards "https://github.com/ydnyshhh/rlvr-gym/tree/main/docs/benchmark_cards"
click node_bench_artifacts "https://github.com/ydnyshhh/rlvr-gym/tree/main/docs/benchmark_artifacts"
click node_build_script "https://github.com/ydnyshhh/rlvr-gym/blob/main/scripts/build_benchmark_artifacts.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_registry,node_family_api,node_runtime,node_verifier,node_reward,node_oracle,node_exporters,node_types,node_utils toneBlue
class node_deduction_grid,node_graph_planning,node_scheduling,node_sokoban,node_symbolic toneAmber
class node_bench_cards,node_bench_artifacts,node_build_script toneMint
```

## Components

### Python package

Runtime library, CLI, and family modules under src/rlvr_gym

- [CLI](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/cli.py): entrypoint · Command-line entry point for listing, sampling, and building artifacts · `src/rlvr_gym/cli.py`
- [Registry](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/registry.py): lookup · Resolves family names to concrete implementations · `src/rlvr_gym/registry.py`
- [Family API](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/family.py): abstraction · Core family abstraction for seeded sampling and task construction · `src/rlvr_gym/core/family.py`
- [Runtime](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/runtime.py): env loop · Lightweight reset/step environment that records traces and observations · `src/rlvr_gym/core/runtime.py`
- [Verifier](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/verifier.py): correctness · Feasibility and solution checking against formal task rules · `src/rlvr_gym/core/verifier.py`
- [Reward](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/reward.py): signal · Converts task outcomes into reward values · `src/rlvr_gym/core/reward.py`
- [Oracle](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/oracle.py): solver · Optional exact solver support for labels, rollouts, and metadata · `src/rlvr_gym/core/oracle.py`
- [Exporters](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/exporters.py): data export · Converts sampled tasks and traces into research datasets and benchmark views · `src/rlvr_gym/core/exporters.py`
- [Types](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/types.py): shared model · Shared typed structures used across the core package · `src/rlvr_gym/core/types.py`
- [Utils](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/core/utils.py): helpers · Shared utility functions used by core and families · `src/rlvr_gym/core/utils.py`

### Task families

Concrete benchmark domains implemented as family modules

- [Deduction grid](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/families/deduction_grid.py): family · Logic-grid deduction tasks with exact table propagation · `src/rlvr_gym/families/deduction_grid.py`
- [Graph planning](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/families/graph_planning.py): family · Weighted graph shortest-path style tasks · `src/rlvr_gym/families/graph_planning.py`
- [Scheduling](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/families/scheduling.py): family · Single-machine scheduling with deadlines and precedence constraints · `src/rlvr_gym/families/scheduling.py`
- [Sokoban](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/families/sokoban.py): family · Solver-backed box pushing and planning tasks · `src/rlvr_gym/families/sokoban.py`
- [Symbolic transform](https://github.com/ydnyshhh/rlvr-gym/blob/main/src/rlvr_gym/families/symbolic_transformation.py): family · Symbolic rewriting and normalization tasks · `src/rlvr_gym/families/symbolic_transformation.py`

### Docs & artifacts

Benchmark specs, cards, and frozen outputs

- [Benchmark cards](https://github.com/ydnyshhh/rlvr-gym/tree/main/docs/benchmark_cards): docs · Family-level benchmark guidance and evaluation contract · `docs/benchmark_cards`
- [Benchmark artifacts](https://github.com/ydnyshhh/rlvr-gym/tree/main/docs/benchmark_artifacts): outputs · Frozen specs, splits, diversity summaries, and baseline results · `docs/benchmark_artifacts`
- [Build script](https://github.com/ydnyshhh/rlvr-gym/blob/main/scripts/build_benchmark_artifacts.py): artifact builder · Generates benchmark artifact bundles from family definitions · `scripts/build_benchmark_artifacts.py`

## Connections

- CLI → Registry: resolve family · CLI looks up named families through the registry
- CLI → Exporters: build outputs · CLI can trigger task and benchmark exports
- Registry → Family API: binds · Registry exposes family implementations via the shared abstraction
- Family API → Runtime: creates env · Families construct runnable environments for agent interaction
- Family API → Verifier: attaches checks · Family generation includes formal validation logic
- Family API → Reward: attaches reward · Family generation includes task outcome scoring
- Family API → Oracle: optional solver · Some families expose exact oracle support
- Runtime → Verifier: evaluates · Runtime outcomes are checked against task rules
- Runtime → Reward: scores · Runtime transitions feed reward computation
- Oracle → Exporters: labels · Oracle outputs support rollouts, labels, and benchmark metadata
- Runtime → Exporters: traces · Recorded trajectories are transformed into export formats
- Family API → Exporters: task spec · Sampled tasks are converted into downstream artifacts
- Deduction grid → Family API: implements · Concrete family module follows the shared family contract
- Graph planning → Family API: implements · Concrete family module follows the shared family contract
- Scheduling → Family API: implements · Concrete family module follows the shared family contract
- Sokoban → Family API: implements · Concrete family module follows the shared family contract
- Symbolic transform → Family API: implements · Concrete family module follows the shared family contract
- Build script → Benchmark artifacts: generates · Build script produces frozen benchmark bundles
- Benchmark cards → Benchmark artifacts: documents · Benchmark cards describe the intended artifact and evaluation contract
- Types → Family API: shared models · Core abstractions reuse common typed structures
- Utils → Family API: supports · Utility helpers support core generation and runtime code
