# bytedance/agent-r architecture

> Architecture diagram of the GitHub repository bytedance/agent-r, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bytedance/agent-r
- Repository: https://github.com/bytedance/agent-r

## Overview

##### Project Type and Purpose
- This project is a research-driven framework focused on training and self-refining language model agents using an iterative self-training approach based on Monte Carlo Tree Search (MCTS). It features trajectory generation, path revision, and evaluation components.
- The README emphasizes using MCTS for on-the-fly error correction in agent trajectories and shows integration with different environments (e.g., SciWorld, WebShop, TextCraft) via customized environment servers.

##### File Structure
- Top-Level Files & Scripts:
- “mcts_collection.py”: main script for MCTS-based trajectory generation.
- “path_collection.py”: handles revision trajectory construction.
- “eval.py”: for evaluating agent performance.
- “mcts_utils” folder consolidates various utilities and submodules.
- mcts_utils Directory:
- Contains “llm_server.py” and “mcts_raw.py” as core logic modules.
- Contains subdirectories for environment-specific implementations (“sciworld”, “textcraft”, “webshop”) that include task-specific modules (e.g., “mcts_sci.py”, “mcts_tc.py”, “mcts_ws.py”) and corresponding test/clean data JSON files.

##### Core Components
a. Main Processing Components:
- MCTS Engine: Represented by “mcts_collection.py” and “mcts_utils/mcts_raw.py” which handle the generation of trajectories using MCTS.
- Revision Module: “path_collection.py” which processes and revises trajectories.
- Evaluation Module: “eval.py” script for running performance tests.
b. Environment Interface:
- Environment Servers: Indicated by the usage instructions for launching servers (WebShop, SciWorld, TextCraft) with custom startup scripts and environment configurations. These interact with the core MCTS system.
- Each environment-specific module (in “mcts_utils/sciworld”, “mcts_utils/textcraft”, “mcts_utils/webshop”) should be represented as separate components that feed task-specific data and logic.
c. Training & Model Components:
- Dependencies and API integration: Highlight environmental variables (e.g., OPENAI_API_KEY, MODEL_DIR) that enable connectivity between local processing and external language model services.
- Training Pipeline: Illustrate the training script which uses the revision trajectories and Xtuner configuration for training.
d. External Dependencies:
- AgentGym: Referenced for environment server support.
- Xtuner: For distributed training.
- Open-source libraries as outlined in “requirements.txt”, plus the external open-source projects (AgentGym and xtuner).

##### Final Recommendations for the Principal Software Engineer
Environment Data  →  MCTS Trajectory Generation  →  Revision Module  →  Evaluation & Feedback  →  Training Pipeline.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Environment Adapters Subgraph
    subgraph "Environment Adapters"
        direction TB
        sci["SciWorld Adapter"]:::adapter
        txt["TextCraft Adapter"]:::adapter
        web["WebShop Adapter"]:::adapter
    end

    %% Core Processing Subgraph
    subgraph "Core Processing"
        direction TB
        mc["MCTS Collection Script"]:::processing
        mr["MCTS Raw Module"]:::processing
        rev["Revision Trajectory Module"]:::processing
        eval["Evaluation Engine"]:::processing
        train["Training Module (Xtuner)"]:::processing
        llm["LLM Server"]:::processing
    end

    %% External Dependencies Subgraph
    subgraph "External Dependencies"
        direction TB
        ag["AgentGym"]:::external
        xt["Xtuner"]:::external
    end

    %% Data Flow Connections
    sci -->|"feeds"| mc
    txt -->|"feeds"| mc
    web -->|"feeds"| mc

    mc -->|"generates"| rev
    mc -->|"sends"| eval
    rev -->|"sendsRevisions"| eval
    eval -->|"feedback"| train
    train -->|"iterativeFeedback"| mc

    train -->|"queries"| llm
    llm -->|"supports"| mr

    ag -->|"providesEnvs"| sci
    ag -->|"providesEnvs"| txt
    ag -->|"providesEnvs"| web

    train -->|"uses"| xt
    xt -->|"trains"| train

    %% Click Events
    click mc "https://github.com/bytedance/agent-r/blob/main/mcts_collection.py"
    click mr "https://github.com/bytedance/agent-r/blob/main/mcts_utils/mcts_raw.py"
    click rev "https://github.com/bytedance/agent-r/blob/main/path_collection.py"
    click eval "https://github.com/bytedance/agent-r/blob/main/eval.py"
    click sci "https://github.com/bytedance/agent-r/blob/main/mcts_utils/sciworld/mcts_sci.py"
    click txt "https://github.com/bytedance/agent-r/blob/main/mcts_utils/textcraft/mcts_tc.py"
    click web "https://github.com/bytedance/agent-r/blob/main/mcts_utils/webshop/mcts_ws.py"
    click llm "https://github.com/bytedance/agent-r/blob/main/mcts_utils/llm_server.py"

    %% Style Classes
    classDef processing fill:#ffcccc,stroke:#cc0000,stroke-width:2px;
    classDef adapter fill:#ccffcc,stroke:#006600,stroke-width:2px;
    classDef external fill:#cce5ff,stroke:#003399,stroke-width:2px;
```
