# one-and-done2022/tau-retail-agentic-grpo-longhorizon architecture

> Architecture diagram of the GitHub repository one-and-done2022/tau-retail-agentic-grpo-longhorizon, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon
- Repository: https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon

## Overview

This repository is a long-horizon retail tool-agent post-training project: synthetic τ-bench tasks provide customer requests, agents interact with a simulated retail environment, and the resulting experience supports SFT and GRPO training before pass-rate evaluation. The README also documents Vanilla, State-, Constraint-, and CS-GRPO variants and fixed-protocol confirmatory evaluation. The graph combines those documented capabilities with paths in the tree; implementation-level wiring for several project-specific stages was not sampled, so uncertain connections are omitted.

Main entry points: [`verl/verl/trainer/ppo/ray_trainer.py`](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/verl/verl/trainer/ppo/ray_trainer.py) (GRPO training), [`agentic-grpo-longhorizon/scripts/eval/check_retail_confirmatory_eval.py`](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/scripts/eval/check_retail_confirmatory_eval.py) (Confirmatory evaluation), [`agentic-grpo-longhorizon/src/envs/retail_cs_grpo.py`](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/src/envs/retail_cs_grpo.py) (Agent interaction), [`tau-bench/tau_bench/envs/retail/env.py`](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/tau-bench/tau_bench/envs/retail/env.py) (Retail environment)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_retail["Retail simulation"]
  node_tasks["Retail tasks<br/>[tasks_train.py]"]
  node_retail_env["Retail environment<br/>[env.py]"]
  node_retail_tools["Retail tools"]
  node_agent_env["Agent interaction<br/>[retail_cs_grpo.py]"]
end

subgraph group_data["Training data"]
  node_sft_collect["SFT collection"]
  node_sft_dataset["SFT dataset<br/>[sft_dataset.py]"]
  node_grpo_data["GRPO parquet"]
  node_tool_config["Tool configuration<br/>[gen_tool_config.py]"]
end

subgraph group_training["Agent training"]
  node_sft_train["SFT training<br/>[sft_train.py]"]
  node_policy["Policy inference<br/>[vllm_policy.py]"]
  node_grpo_train["GRPO training<br/>[ray_trainer.py]"]
  node_reward["Reward scoring<br/>[reward_manager.py]"]
  node_checkpoint[("Model checkpoints")]
end

subgraph group_evaluation["Evaluation"]
  node_eval["Pass-k evaluation<br/>[pass_k_eval.py]"]
  node_eval_runner["Confirmatory evaluation"]
  node_results["Evaluation results"]
end

node_researcher(("Experimenter"))

node_researcher -->|"selects tasks"| node_tasks
node_tasks -->|"supplies scenarios"| node_retail_env
node_retail_env -->|"exposes tools"| node_retail_tools
node_agent_env -->|"interacts with"| node_retail_env
node_sft_collect -->|"collects trajectories"| node_agent_env
node_sft_collect -->|"prepares data"| node_sft_dataset
node_sft_dataset -->|"feeds training"| node_sft_train
node_researcher -->|"prepares runs"| node_grpo_data
node_grpo_data -->|"feeds training"| node_grpo_train
node_tool_config -->|"configures tools"| node_grpo_train
node_grpo_train -->|"updates policy"| node_policy
node_grpo_train -->|"runs rollouts"| node_agent_env
node_grpo_train -->|"requests scores"| node_reward
node_grpo_train -->|"saves model"| node_checkpoint
node_policy -->|"provides policy"| node_eval_runner
node_eval_runner -->|"evaluates trajectories"| node_eval
node_eval_runner -->|"runs evaluation"| node_agent_env
node_eval_runner -->|"summarizes results"| node_results
node_researcher -->|"launches evaluation"| node_eval_runner

click node_tasks "https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/tau-bench/tau_bench/envs/retail/tasks_train.py"
click node_retail_env "https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/tau-bench/tau_bench/envs/retail/env.py"
click node_retail_tools "https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/tree/main/tau-bench/tau_bench/envs/retail/tools"
click node_agent_env "https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/src/envs/retail_cs_grpo.py"
click node_sft_collect "https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/scripts/train/sft/collect_sft_data.py"
click node_sft_dataset "https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/src/training/sft_dataset.py"
click node_grpo_data "https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/scripts/train/grpo/build_grpo_parquet.py"
click node_tool_config "https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/scripts/train/grpo/gen_tool_config.py"
click node_sft_train "https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/scripts/train/sft/sft_train.py"
click node_policy "https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/src/models/vllm_policy.py"
click node_grpo_train "https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/verl/verl/trainer/ppo/ray_trainer.py"
click node_reward "https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/verl/verl/experimental/reward/reward_manager.py"
click node_checkpoint "https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/verl/verl/utils/checkpoint/fsdp_checkpoint_manager.py"
click node_eval "https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/src/evaluation/pass_k_eval.py"
click node_eval_runner "https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/scripts/eval/check_retail_confirmatory_eval.py"
click node_results "https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/scripts/eval/summarize_retail_confirmatory.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_tasks,node_retail_env,node_retail_tools,node_agent_env toneBlue
class node_sft_collect,node_sft_dataset,node_grpo_data,node_tool_config toneAmber
class node_sft_train,node_policy,node_grpo_train,node_reward,node_checkpoint toneMint
class node_eval,node_eval_runner,node_results toneRose
class node_researcher toneIndigo
```

## Components

### Retail simulation

- [Retail tasks](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/tau-bench/tau_bench/envs/retail/tasks_train.py): `tau-bench/tau_bench/envs/retail/tasks_train.py`
- [Retail environment](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/tau-bench/tau_bench/envs/retail/env.py): `tau-bench/tau_bench/envs/retail/env.py`
- [Retail tools](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/tree/main/tau-bench/tau_bench/envs/retail/tools): `tau-bench/tau_bench/envs/retail/tools`
- [Agent interaction](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/src/envs/retail_cs_grpo.py): `agentic-grpo-longhorizon/src/envs/retail_cs_grpo.py`

### Training data

- [SFT collection](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/scripts/train/sft/collect_sft_data.py): `agentic-grpo-longhorizon/scripts/train/sft/collect_sft_data.py`
- [SFT dataset](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/src/training/sft_dataset.py): `agentic-grpo-longhorizon/src/training/sft_dataset.py`
- [GRPO parquet](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/scripts/train/grpo/build_grpo_parquet.py): `agentic-grpo-longhorizon/scripts/train/grpo/build_grpo_parquet.py`
- [Tool configuration](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/scripts/train/grpo/gen_tool_config.py): `agentic-grpo-longhorizon/scripts/train/grpo/gen_tool_config.py`

### Agent training

- [SFT training](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/scripts/train/sft/sft_train.py): `agentic-grpo-longhorizon/scripts/train/sft/sft_train.py`
- [Policy inference](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/src/models/vllm_policy.py): `agentic-grpo-longhorizon/src/models/vllm_policy.py`
- [GRPO training](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/verl/verl/trainer/ppo/ray_trainer.py): `verl/verl/trainer/ppo/ray_trainer.py`
- [Reward scoring](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/verl/verl/experimental/reward/reward_manager.py): `verl/verl/experimental/reward/reward_manager.py`
- [Model checkpoints](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/verl/verl/utils/checkpoint/fsdp_checkpoint_manager.py): `verl/verl/utils/checkpoint/fsdp_checkpoint_manager.py`

### Evaluation

- [Pass-k evaluation](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/src/evaluation/pass_k_eval.py): `agentic-grpo-longhorizon/src/evaluation/pass_k_eval.py`
- [Confirmatory evaluation](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/scripts/eval/check_retail_confirmatory_eval.py): `agentic-grpo-longhorizon/scripts/eval/check_retail_confirmatory_eval.py`
- [Evaluation results](https://github.com/one-and-done2022/tau-retail-agentic-grpo-longhorizon/blob/main/agentic-grpo-longhorizon/scripts/eval/summarize_retail_confirmatory.py): `agentic-grpo-longhorizon/scripts/eval/summarize_retail_confirmatory.py`

### Other components

- **Experimenter**

## Connections

- Experimenter → Retail tasks: selects tasks
- Retail tasks → Retail environment: supplies scenarios
- Retail environment → Retail tools: exposes tools
- Agent interaction → Retail environment: interacts with
- SFT collection → Agent interaction: collects trajectories
- SFT collection → SFT dataset: prepares data
- SFT dataset → SFT training: feeds training
- Experimenter → GRPO parquet: prepares runs
- GRPO parquet → GRPO training: feeds training
- Tool configuration → GRPO training: configures tools
- GRPO training → Policy inference: updates policy
- GRPO training → Agent interaction: runs rollouts
- GRPO training → Reward scoring: requests scores
- GRPO training → Model checkpoints: saves model
- Policy inference → Confirmatory evaluation: provides policy
- Confirmatory evaluation → Pass-k evaluation: evaluates trajectories
- Confirmatory evaluation → Agent interaction: runs evaluation
- Confirmatory evaluation → Evaluation results: summarizes results
- Experimenter → Confirmatory evaluation: launches evaluation