# sahilmohammed-ai/agentic-verifier-research architecture

> Architecture diagram of the GitHub repository sahilmohammed-ai/agentic-verifier-research, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sahilmohammed-ai/agentic-verifier-research
- Repository: https://github.com/sahilmohammed-ai/agentic-verifier-research

## Overview

This repository is a research pipeline for comparing verifiers that assess agents solving long-horizon TextWorldExpress tasks. The principal workflow runs a planner-worker agent in an environment, collects episode trajectories, labels them with Monte Carlo return-to-go targets, trains the temporal value verifier, and evaluates its scores. A parallel preference-verifier branch trains on won/lost episode pairs. The graph also preserves the documented frozen-LLM baseline and evaluation capabilities. Runner, agent, environment, labeling, and training relationships are supported by README or sampled source; unsampled evaluation wiring is shown only at the documented subsystem level.

Main entry points: [`rollout/runner.py`](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/rollout/runner.py) (Episode runner), [`scripts`](https://github.com/sahilmohammed-ai/agentic-verifier-research/tree/main/scripts) (Evaluation scripts), [`agents/llm.py`](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/agents/llm.py) (LLM backends), [`verifier/infer.py`](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/verifier/infer.py) (TVV inference)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_execution["Agent execution"]
  node_collect["Rollout collection<br/>[collect.py]"]
  node_runner["Episode runner<br/>[runner.py]"]
  node_thinker["Episode planner<br/>[thinker.py]"]
  node_worker["Turn action<br/>[worker.py]"]
  node_llm["LLM backends<br/>[llm.py]"]
  node_environment["TextWorldExpress"]
end

subgraph group_data["Rollout data"]
  node_trajectories["Episode trajectories<br/>[schemas.py]"]
  node_labeler["MC reward labeler<br/>[label_twx.py]"]
  node_labeled["Labeled episodes"]
end

subgraph group_verifiers["Verifier models"]
  node_tvv_train["TVV training<br/>[train.py]"]
  node_tvv_infer["TVV inference<br/>[infer.py]"]
  node_tvv_model["TVV model<br/>[model.py]"]
  node_epv_train["EPV training<br/>[train.py]"]
  node_epv_infer["EPV inference<br/>[infer.py]"]
  node_epv_model["EPV model<br/>[model.py]"]
  node_frozen["Frozen LLM judge<br/>[frozen_llm.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_evaluation["Evaluation scripts"]
  node_best_n["Best-of-N selection<br/>[best_of_n_eval.py]"]
  node_failure_eval["Failure detection"]
  node_scoring["Verifier scoring"]
  node_frozen_scoring["Frozen judge scoring"]
end

node_researcher(("Researcher"))
node_twx{{"TextWorldExpress service"}}
node_external_llms{{"Model providers"}}

node_researcher -->|"starts collection"| node_collect
node_collect -->|"runs episodes"| node_runner
node_runner -->|"requests plan"| node_thinker
node_runner -->|"requests action"| node_worker
node_runner -->|"resets and steps"| node_environment
node_environment -->|"executes actions"| node_twx
node_thinker -->|"requests completion"| node_llm
node_worker -->|"requests completion"| node_llm
node_llm -.->|"calls provider"| node_external_llms
node_runner -->|"records turns"| node_trajectories
node_collect -->|"writes episodes"| node_trajectories
node_trajectories -->|"reads episodes"| node_labeler
node_labeler -->|"writes labels"| node_labeled
node_labeled -->|"trains from"| node_tvv_train
node_tvv_train -->|"optimizes"| node_tvv_model
node_tvv_infer -->|"loads model"| node_tvv_model
node_labeled -->|"trains from"| node_epv_train
node_epv_train -->|"optimizes"| node_epv_model
node_epv_infer -->|"loads model"| node_epv_model
node_frozen -->|"requests completion"| node_llm
node_researcher -->|"starts evaluation"| node_evaluation
node_evaluation -->|"dispatches"| node_best_n
node_evaluation -->|"dispatches"| node_failure_eval
node_evaluation -->|"dispatches"| node_scoring
node_evaluation -->|"dispatches"| node_frozen_scoring
node_best_n -->|"samples actions"| node_worker
node_best_n -->|"ranks candidates"| node_tvv_infer
node_failure_eval -->|"scores turns"| node_tvv_infer
node_scoring -->|"scores verifier"| node_tvv_infer
node_scoring -->|"scores verifier"| node_epv_infer
node_frozen_scoring -->|"scores turns"| node_frozen

click node_collect "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/rollout/collect.py"
click node_runner "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/rollout/runner.py"
click node_thinker "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/agents/thinker.py"
click node_worker "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/agents/worker.py"
click node_llm "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/agents/llm.py"
click node_environment "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/envs/textworldexpress_env.py"
click node_trajectories "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/rollout/schemas.py"
click node_labeler "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/rollout/label_twx.py"
click node_tvv_train "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/verifier/train.py"
click node_tvv_infer "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/verifier/infer.py"
click node_tvv_model "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/verifier/model.py"
click node_epv_train "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/verifier_dpo/train.py"
click node_epv_infer "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/verifier_dpo/infer.py"
click node_epv_model "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/verifier_dpo/model.py"
click node_frozen "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/verifier/frozen_llm.py"
click node_evaluation "https://github.com/sahilmohammed-ai/agentic-verifier-research/tree/main/scripts"
click node_best_n "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/scripts/best_of_n_eval.py"
click node_failure_eval "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/scripts/failure_detection_eval.py"
click node_scoring "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/scripts/score_trained_verifier.py"
click node_frozen_scoring "https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/scripts/score_frozen_verifier.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_collect,node_runner,node_thinker,node_worker,node_llm,node_environment toneBlue
class node_trajectories,node_labeler,node_labeled toneAmber
class node_tvv_train,node_tvv_infer,node_tvv_model,node_epv_train,node_epv_infer,node_epv_model,node_frozen toneMint
class node_evaluation,node_best_n,node_failure_eval,node_scoring,node_frozen_scoring toneRose
class node_researcher,node_twx,node_external_llms toneIndigo
```

## Components

### Agent execution

- [Rollout collection](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/rollout/collect.py): `rollout/collect.py`
- [Episode runner](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/rollout/runner.py): `rollout/runner.py`
- [Episode planner](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/agents/thinker.py): `agents/thinker.py`
- [Turn action](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/agents/worker.py): `agents/worker.py`
- [LLM backends](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/agents/llm.py): `agents/llm.py`
- [TextWorldExpress](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/envs/textworldexpress_env.py): `envs/textworldexpress_env.py`

### Rollout data

- [Episode trajectories](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/rollout/schemas.py): `rollout/schemas.py`
- [MC reward labeler](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/rollout/label_twx.py): `rollout/label_twx.py`
- **Labeled episodes**

### Verifier models

- [TVV training](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/verifier/train.py): `verifier/train.py`
- [TVV inference](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/verifier/infer.py): `verifier/infer.py`
- [TVV model](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/verifier/model.py): `verifier/model.py`
- [EPV training](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/verifier_dpo/train.py): `verifier_dpo/train.py`
- [EPV inference](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/verifier_dpo/infer.py): `verifier_dpo/infer.py`
- [EPV model](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/verifier_dpo/model.py): `verifier_dpo/model.py`
- [Frozen LLM judge](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/verifier/frozen_llm.py): `verifier/frozen_llm.py`

### Evaluation

- [Evaluation scripts](https://github.com/sahilmohammed-ai/agentic-verifier-research/tree/main/scripts): `scripts`
- [Best-of-N selection](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/scripts/best_of_n_eval.py): `scripts/best_of_n_eval.py`
- [Failure detection](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/scripts/failure_detection_eval.py): `scripts/failure_detection_eval.py`
- [Verifier scoring](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/scripts/score_trained_verifier.py): `scripts/score_trained_verifier.py`
- [Frozen judge scoring](https://github.com/sahilmohammed-ai/agentic-verifier-research/blob/main/scripts/score_frozen_verifier.py): `scripts/score_frozen_verifier.py`

### Other components

- **Researcher**
- **TextWorldExpress service**
- **Model providers**

## Connections

- Researcher → Rollout collection: starts collection
- Rollout collection → Episode runner: runs episodes
- Episode runner → Episode planner: requests plan
- Episode runner → Turn action: requests action
- Episode runner → TextWorldExpress: resets and steps
- TextWorldExpress → TextWorldExpress service: executes actions
- Episode planner → LLM backends: requests completion
- Turn action → LLM backends: requests completion
- LLM backends → Model providers: calls provider
- Episode runner → Episode trajectories: records turns
- Rollout collection → Episode trajectories: writes episodes
- Episode trajectories → MC reward labeler: reads episodes
- MC reward labeler → Labeled episodes: writes labels
- Labeled episodes → TVV training: trains from
- TVV training → TVV model: optimizes
- TVV inference → TVV model: loads model
- Labeled episodes → EPV training: trains from
- EPV training → EPV model: optimizes
- EPV inference → EPV model: loads model
- Frozen LLM judge → LLM backends: requests completion
- Researcher → Evaluation scripts: starts evaluation
- Evaluation scripts → Best-of-N selection: dispatches
- Evaluation scripts → Failure detection: dispatches
- Evaluation scripts → Verifier scoring: dispatches
- Evaluation scripts → Frozen judge scoring: dispatches
- Best-of-N selection → Turn action: samples actions
- Best-of-N selection → TVV inference: ranks candidates
- Failure detection → TVV inference: scores turns
- Verifier scoring → TVV inference: scores verifier
- Verifier scoring → EPV inference: scores verifier
- Frozen judge scoring → Frozen LLM judge: scores turns