How sahilmohammed-ai/agentic-verifier-research works
This repository is a research pipeline for comparing verifiers that assess agents solving long-horizon TextWorldExpress tasks. The principal workflow runs a planner-worker agent in an environment, collects episode trajectories, labels them with Monte Carlo return-to-go targets, trains the temporal value verifier, and evaluates its scores. A parallel preference-verifier branch trains on won/lost episode pairs. The graph also preserves the documented frozen-LLM baseline and evaluation capabilities. Runner, agent, environment, labeling, and training relationships are supported by README or sampled source; unsampled evaluation wiring is shown only at the documented subsystem level.
Main entry points: runner.py Episode runner · scripts Evaluation scripts · llm.py LLM backends · infer.py TVV inference
Components 24
Agent execution
- Rollout collection
rollout/collect.py - Episode runner
rollout/runner.py - Episode planner
agents/thinker.py - Turn action
agents/worker.py - LLM backends
agents/llm.py - TextWorldExpress
envs/textworldexpress_env.py
Rollout data
- Episode trajectories
rollout/schemas.py - MC reward labeler
rollout/label_twx.py - Labeled episodes
Verifier models
- TVV training
verifier/train.py - TVV inference
verifier/infer.py - TVV model
verifier/model.py - EPV training
verifier_dpo/train.py - EPV inference
verifier_dpo/infer.py - EPV model
verifier_dpo/model.py - Frozen LLM judge
verifier/frozen_llm.py
Evaluation
- Evaluation scripts
scripts - Best-of-N selection
scripts/best_of_n_eval.py - Failure detection
scripts/failure_detection_eval.py - Verifier scoring
scripts/score_trained_verifier.py - Frozen judge scoring
scripts/score_frozen_verifier.py
Other components
- Researcher
- TextWorldExpress service
- Model providers
Connections 31
- Researcher → Rollout collection: starts collection
- Rollout collection → Episode runner: runs episodes
- Episode runner → Episode planner: requests plan
- Episode runner → Turn action: requests action
- Episode runner → TextWorldExpress: resets and steps
- TextWorldExpress → TextWorldExpress service: executes actions
- Episode planner → LLM backends: requests completion
- Turn action → LLM backends: requests completion
- LLM backends → Model providers: calls provider
- Episode runner → Episode trajectories: records turns
- Rollout collection → Episode trajectories: writes episodes
- Episode trajectories → MC reward labeler: reads episodes
- MC reward labeler → Labeled episodes: writes labels
- Labeled episodes → TVV training: trains from
- TVV training → TVV model: optimizes
- TVV inference → TVV model: loads model
- Labeled episodes → EPV training: trains from
- EPV training → EPV model: optimizes
- EPV inference → EPV model: loads model
- Frozen LLM judge → LLM backends: requests completion
- Researcher → Evaluation scripts: starts evaluation
- Evaluation scripts → Best-of-N selection: dispatches
- Evaluation scripts → Failure detection: dispatches
- Evaluation scripts → Verifier scoring: dispatches
- Evaluation scripts → Frozen judge scoring: dispatches
- Best-of-N selection → Turn action: samples actions
- Best-of-N selection → TVV inference: ranks candidates
- Failure detection → TVV inference: scores turns
- Verifier scoring → TVV inference: scores verifier
- Verifier scoring → EPV inference: scores verifier
- Frozen judge scoring → Frozen LLM judge: scores turns