# yyhdbl/shopping-grpo-longhorizon architecture

> Architecture diagram of the GitHub repository yyhdbl/shopping-grpo-longhorizon, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yyhdbl/shopping-grpo-longhorizon
- Repository: https://github.com/yyhdbl/shopping-grpo-longhorizon

## Overview

Shopping GRPO is a reproducible post-training and evaluation system for long-horizon shopping agents. Its main workflow collects teacher demonstrations, filters them into action-focused SFT data, trains with LoRA and online GRPO in ShopSimulator, then evaluates models on a shared held-out benchmark. The embedded simulator covers search through purchase and terminal reward; evaluation separately produces rubric, trajectory-judge, deterministic, and reward results. The README documents several capabilities whose implementations were not sampled, so those are represented at supported module or directory paths without assuming internal call wiring.

Main entry points: [`src/shopping_grpo/evaluation/rollout.py`](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/evaluation/rollout.py) (Evaluation rollout), [`src/shopping_grpo/environment/client.py`](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/environment/client.py) (Shop API client), [`src/shopping_grpo/evaluation/trajectory.py`](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/evaluation/trajectory.py) (Trajectory judge), [`src/shopping_grpo/training/grpo`](https://github.com/yyhdbl/shopping-grpo-longhorizon/tree/main/src/shopping_grpo/training/grpo) (Online GRPO)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data collection"]
  node_sftcollect["Trajectory acceptance<br/>[sft.py]"]
  node_sftdata["SFT dataset<br/>[dataset.py]"]
end

subgraph group_training["Model training"]
  node_sfttrain["LoRA SFT"]
  node_grpotrain["Online GRPO"]
  node_agentloop["Agent loop<br/>[agent_loop.py]"]
  node_session["Environment session<br/>[session.py]"]
  node_tooladapter["Tool adapter<br/>[tools.py]"]
end

subgraph group_sim["Shopping environment"]
  node_shopclient["Shop API client<br/>[client.py]"]
  node_shopservice["ShopSimulator service<br/>[shop_agent.py]"]
  node_webapp["Shopping sessions<br/>[app.py]"]
  node_search["Product search<br/>[search.py]"]
  node_simengine["Shopping engine<br/>[engine.py]"]
  node_reward["Terminal reward"]
end

subgraph group_eval["Evaluation"]
  node_bench["Benchmark tasks<br/>[manifest.py]"]
  node_rollout["Evaluation rollout<br/>[rollout.py]"]
  node_rubric["Rubric curation<br/>[rubric.py]"]
  node_judge["Trajectory judge<br/>[trajectory.py]"]
  node_metrics["Hard checks<br/>[metrics.py]"]
  node_results["Results and comparison<br/>[results.py]"]
end

node_researcher(("Experimenter"))
node_teacher(("Teacher model"))
node_actor(("Trained actor"))
node_judgeapi(("Judge model service"))

node_researcher -->|"requests demonstrations"| node_teacher
node_teacher -->|"provides actions"| node_shopclient
node_shopclient -->|"sends actions"| node_shopservice
node_shopservice -->|"returns observations"| node_shopclient
node_shopclient -->|"records rollouts"| node_sftcollect
node_sftcollect -->|"filters and builds"| node_sftdata
node_sftdata -->|"trains on"| node_sfttrain
node_sfttrain -->|"produces model"| node_actor
node_actor -->|"initializes training"| node_grpotrain
node_grpotrain -->|"runs trajectories"| node_agentloop
node_agentloop -->|"opens session"| node_session
node_agentloop -->|"dispatches tools"| node_tooladapter
node_session -->|"leases environment"| node_shopclient
node_tooladapter -->|"executes actions"| node_shopclient
node_grpotrain -->|"updates policy"| node_actor
node_researcher -->|"selects benchmark"| node_bench
node_bench -->|"provides tasks"| node_rollout
node_actor -->|"generates actions"| node_rollout
node_rollout -->|"runs evaluation"| node_shopclient
node_rollout -->|"provides task evidence"| node_rubric
node_rubric -.->|"requests rubric curation"| node_judgeapi
node_rubric -->|"supplies frozen rubric"| node_judge
node_rollout -->|"provides trajectories"| node_judge
node_judge -.->|"requests scoring"| node_judgeapi
node_rollout -->|"checks outcomes"| node_metrics
node_rollout -->|"provides rollout results"| node_results
node_judge -->|"provides judge scores"| node_results
node_metrics -->|"provides hard checks"| node_results
node_shopservice -->|"serves interactions"| node_webapp
node_webapp -->|"searches products"| node_search
node_webapp -->|"renders product actions"| node_simengine
node_simengine -->|"evaluates purchase"| node_reward

click node_sftcollect "https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/collection/sft.py"
click node_sftdata "https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/training/sft/dataset.py"
click node_sfttrain "https://github.com/yyhdbl/shopping-grpo-longhorizon/tree/main/src/shopping_grpo/training/sft"
click node_grpotrain "https://github.com/yyhdbl/shopping-grpo-longhorizon/tree/main/src/shopping_grpo/training/grpo"
click node_agentloop "https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/training/grpo/adapter/agent_loop.py"
click node_session "https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/training/grpo/adapter/session.py"
click node_tooladapter "https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/training/grpo/adapter/tools.py"
click node_shopclient "https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/environment/client.py"
click node_shopservice "https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/environments/ShopSimulator/shop_env/shop_env/shop_agent.py"
click node_webapp "https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/environments/ShopSimulator/shop_env/web_agent_site/app.py"
click node_search "https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/environments/ShopSimulator/shop_env/web_agent_site/engine/search.py"
click node_simengine "https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/environments/ShopSimulator/shop_env/web_agent_site/engine/engine.py"
click node_reward "https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/environments/ShopSimulator/shop_env/web_agent_site/envs/web_agent_text_env.py"
click node_bench "https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/evaluation/manifest.py"
click node_rollout "https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/evaluation/rollout.py"
click node_rubric "https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/evaluation/rubric.py"
click node_judge "https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/evaluation/trajectory.py"
click node_metrics "https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/evaluation/metrics.py"
click node_results "https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/evaluation/results.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_sftcollect,node_sftdata toneBlue
class node_sfttrain,node_grpotrain,node_agentloop,node_session,node_tooladapter toneAmber
class node_shopclient,node_shopservice,node_webapp,node_search,node_simengine,node_reward toneMint
class node_bench,node_rollout,node_rubric,node_judge,node_metrics,node_results toneRose
class node_researcher,node_teacher,node_actor,node_judgeapi toneIndigo
```

## Components

### Data collection

- [Trajectory acceptance](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/collection/sft.py): `src/shopping_grpo/collection/sft.py`
- [SFT dataset](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/training/sft/dataset.py): `src/shopping_grpo/training/sft/dataset.py`

### Model training

- [LoRA SFT](https://github.com/yyhdbl/shopping-grpo-longhorizon/tree/main/src/shopping_grpo/training/sft): `src/shopping_grpo/training/sft`
- [Online GRPO](https://github.com/yyhdbl/shopping-grpo-longhorizon/tree/main/src/shopping_grpo/training/grpo): `src/shopping_grpo/training/grpo`
- [Agent loop](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/training/grpo/adapter/agent_loop.py): `src/shopping_grpo/training/grpo/adapter/agent_loop.py`
- [Environment session](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/training/grpo/adapter/session.py): `src/shopping_grpo/training/grpo/adapter/session.py`
- [Tool adapter](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/training/grpo/adapter/tools.py): `src/shopping_grpo/training/grpo/adapter/tools.py`

### Shopping environment

- [Shop API client](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/environment/client.py): `src/shopping_grpo/environment/client.py`
- [ShopSimulator service](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/environments/ShopSimulator/shop_env/shop_env/shop_agent.py): `environments/ShopSimulator/shop_env/shop_env/shop_agent.py`
- [Shopping sessions](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/environments/ShopSimulator/shop_env/web_agent_site/app.py): `environments/ShopSimulator/shop_env/web_agent_site/app.py`
- [Product search](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/environments/ShopSimulator/shop_env/web_agent_site/engine/search.py): `environments/ShopSimulator/shop_env/web_agent_site/engine/search.py`
- [Shopping engine](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/environments/ShopSimulator/shop_env/web_agent_site/engine/engine.py): `environments/ShopSimulator/shop_env/web_agent_site/engine/engine.py`
- [Terminal reward](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/environments/ShopSimulator/shop_env/web_agent_site/envs/web_agent_text_env.py): `environments/ShopSimulator/shop_env/web_agent_site/envs/web_agent_text_env.py`

### Evaluation

- [Benchmark tasks](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/evaluation/manifest.py): `src/shopping_grpo/evaluation/manifest.py`
- [Evaluation rollout](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/evaluation/rollout.py): `src/shopping_grpo/evaluation/rollout.py`
- [Rubric curation](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/evaluation/rubric.py): `src/shopping_grpo/evaluation/rubric.py`
- [Trajectory judge](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/evaluation/trajectory.py): `src/shopping_grpo/evaluation/trajectory.py`
- [Hard checks](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/evaluation/metrics.py): `src/shopping_grpo/evaluation/metrics.py`
- [Results and comparison](https://github.com/yyhdbl/shopping-grpo-longhorizon/blob/main/src/shopping_grpo/evaluation/results.py): `src/shopping_grpo/evaluation/results.py`

### Other components

- **Experimenter**
- **Teacher model**
- **Trained actor**
- **Judge model service**

## Connections

- Experimenter → Teacher model: requests demonstrations
- Teacher model → Shop API client: provides actions
- Shop API client → ShopSimulator service: sends actions
- ShopSimulator service → Shop API client: returns observations
- Shop API client → Trajectory acceptance: records rollouts
- Trajectory acceptance → SFT dataset: filters and builds
- SFT dataset → LoRA SFT: trains on
- LoRA SFT → Trained actor: produces model
- Trained actor → Online GRPO: initializes training
- Online GRPO → Agent loop: runs trajectories
- Agent loop → Environment session: opens session
- Agent loop → Tool adapter: dispatches tools
- Environment session → Shop API client: leases environment
- Tool adapter → Shop API client: executes actions
- Online GRPO → Trained actor: updates policy
- Experimenter → Benchmark tasks: selects benchmark
- Benchmark tasks → Evaluation rollout: provides tasks
- Trained actor → Evaluation rollout: generates actions
- Evaluation rollout → Shop API client: runs evaluation
- Evaluation rollout → Rubric curation: provides task evidence
- Rubric curation → Judge model service: requests rubric curation
- Rubric curation → Trajectory judge: supplies frozen rubric
- Evaluation rollout → Trajectory judge: provides trajectories
- Trajectory judge → Judge model service: requests scoring
- Evaluation rollout → Hard checks: checks outcomes
- Evaluation rollout → Results and comparison: provides rollout results
- Trajectory judge → Results and comparison: provides judge scores
- Hard checks → Results and comparison: provides hard checks
- ShopSimulator service → Shopping sessions: serves interactions
- Shopping sessions → Product search: searches products
- Shopping sessions → Shopping engine: renders product actions
- Shopping engine → Terminal reward: evaluates purchase