# iculizhi/textarena-selfplay-qwen architecture

> Architecture diagram of the GitHub repository iculizhi/textarena-selfplay-qwen, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/iculizhi/textarena-selfplay-qwen
- Repository: https://github.com/iculizhi/textarena-selfplay-qwen

## Overview

##### Project Type and Purpose
- This is an ML training & evaluation pipeline (open-source tool) for Tic-Tac-Toe Chain-of-Thought (CoT) models built on Qwen.
- Purpose:
- Generate self-play game data at six CoT lengths
- Train separate LoRA-fine-tuned Qwen models per length (multi-GPU)
- Evaluate models on standardized test sets (single/multi-GPU)
- Provide monitoring and analysis tools

##### Main Architectural Components
A. Configuration Manager
- Directory: configs/, llama_factory_configs/, scripts/training/llama_factory_configs/
- Files: .yaml and .json that drive dataset parameters, self-play, training, evaluation

B. Data Generation Module
- Core Code: src/data_generation/
- data_collector.py
- selfplay_runner.py
- Scripts: scripts/data_generation/generate_all_cot_lengths_serial.py
- Output: data/processed/*.json (CoT datasets + test sets)

C. Training Orchestration
- Core CLI: llamafactory-cli (external LLaMA-Factory)
- Scripts:
- scripts/training/train_all_models_parallel.sh
- scripts/training/train_all_models.sh
- monitor_training.sh / stop_all_training.sh
- Models output: models/qwen_*_cot_lora

D. Evaluation Orchestration
- Core Code: evaluation/*.py
- generate_test_set.py, multi_optimal_evaluator.py, analyze_results.py
- Scripts: scripts/evaluation/ultra_fast_eval.sh, ultra_turbo_evaluator.py
- Output: evaluation_results/*.json and analysis_summary.json

E. Monitoring & Utilities
- GPU/Training Monitor: scripts/utils/gpu_monitor.sh, monitor_training.sh
- Quickstart & Demo: examples/quick_start.py, examples/long_cot_demo.py
- Top-level helpers: quickstart.sh, monitor_training.sh, stop_all_training.sh

F. Core Agents & Game Logic
- src/agents/qwen_agent.py, src/agents/smart_agent.py
- src/utils/game_parser.py, mock_env.py (for self-play)
- src/main.py (entry point)

##### Component Interactions and Data Flow
1. CONFIG LOAD →
- Training configs (YAML/JSON) feed into self-play runner & LLaMA-Factory CLI
- Dataset configs feed data generation & evaluation
2. DATA GENERATION →
- Self-play runner uses qwen_agent & mock_env → produces raw game logs
- data_formatter.py cleans/structures logs → saved under data/processed/*.json
3. MODEL TRAINING →
- LLaMA-Factory picks up each config → fine-tunes Qwen models on processed CoT data
- Training scripts dispatch jobs to GPU cluster (parallel/sharded)
- Models checkpointed under models/
4. EVALUATION →
- generate_test_set.py or test_set_avoidance scripts produce test files
- model_evaluator.py / ultra_turbo_evaluator loads models + test data → inference & metrics
- Results aggregated into evaluation_results/ & analysis_summary.json
5. MONITORING & ANALYSIS →
- GPU_MONITOR & monitor_training track resource use and logs
- analysis_summary.json + scripts/analysis tools produce plots and tables

##### Architectural Patterns & Principles
- Pipeline pattern: clear separation between data generation, training, evaluation
- Modular design: each stage has its own folder & scripts
- Config-driven orchestration: YAML/JSON files parameterize every step
- Parallel & distributed training/evaluation via shell scripts + multi-GPU workers
- Extensible: add new CoT lengths by supplying new configs and reusing same code

##### Essential Elements
- Labels for all YAML/JSON configs and their target stages.
- A “Zoom-in” inset for a single CoT-length training flow, then a “Zoom-out” view showing fan-out for six models.
- External dependencies: GitHub (LLaMA-Factory), Qwen model repository.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration Manager
    subgraph "Configuration Manager"
        CMYAML["YAML/JSON Configs"]:::config
    end
    click CMYAML "https://github.com/iculizhi/textarena-selfplay-qwen/tree/main/configs/"

    %% Data Generation Module
    subgraph "Data Generation Module" 
        SP["selfplay_runner.py"]:::compute
        DC["data_collector.py"]:::compute
        DF["data_formatter.py"]:::compute
        GS["generate_all_cot_lengths_serial.py"]:::script
        DP["data/processed/*.json"]:::datastore
    end
    click SP "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/src/data_generation/selfplay_runner.py"
    click DC "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/src/data_generation/data_collector.py"
    click GS "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/scripts/data_generation/generate_all_cot_lengths_serial.py"
    click DP "https://github.com/iculizhi/textarena-selfplay-qwen/tree/main/data/processed/"

    %% Core Agents & Game Logic
    subgraph "Core Agents & Game Logic"
        QA["qwen_agent.py"]:::compute
        SA["smart_agent.py"]:::compute
        GP["game_parser.py"]:::compute
        ME["mock_env.py"]:::compute
        TS["test_set_avoider.py"]:::compute
        MAIN["main.py"]:::compute
    end
    click QA "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/src/agents/qwen_agent.py"
    click SA "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/src/agents/smart_agent.py"
    click GP "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/src/utils/game_parser.py"
    click ME "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/src/utils/mock_env.py"
    click TS "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/src/utils/test_set_avoider.py"
    click MAIN "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/src/main.py"

    %% Training Orchestration
    subgraph "Training Orchestration"
        CLI["LLaMA-Factory CLI"]:::external
        TALL["train_all_models.sh"]:::script
        TPAR["train_all_models_parallel.sh"]:::script
        MT["monitor_training.sh"]:::script
        ST["stop_all_training.sh"]:::script
        TOOLS["train_qwen.sh"]:::script
        MODSTORE["models/qwen_*_cot_lora"]:::datastore
    end
    click TALL "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/scripts/training/train_all_models.sh"
    click TPAR "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/scripts/training/train_all_models_parallel.sh"
    click MT "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/scripts/training/monitor_training.sh"
    click ST "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/scripts/training/stop_all_training.sh"
    click TOOLS "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/tools/train_qwen.sh"
    click MODSTORE "https://github.com/iculizhi/textarena-selfplay-qwen/tree/main/models/"

    %% Evaluation Orchestration
    subgraph "Evaluation Orchestration"
        GTS["generate_test_set.py"]:::compute
        GMTS["generate_multi_optimal_test_set.py"]:::compute
        MOE["multi_optimal_evaluator.py"]:::compute
        AR["analyze_results.py"]:::compute
        RFE["run_full_evaluation.py"]:::compute
        UFE["ultra_fast_eval.sh"]:::script
        UTE["ultra_turbo_evaluator.py"]:::script
        ER["evaluation_results/*.json"]:::datastore
        AS["analysis_summary.json"]:::datastore
    end
    click GTS "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/evaluation/generate_test_set.py"
    click GMTS "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/evaluation/generate_multi_optimal_test_set.py"
    click MOE "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/evaluation/multi_optimal_evaluator.py"
    click AR "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/evaluation/analyze_results.py"
    click RFE "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/evaluation/run_full_evaluation.py"
    click UFE "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/scripts/evaluation/ultra_fast_eval.sh"
    click UTE "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/scripts/evaluation/ultra_turbo_evaluator.py"
    click ER "https://github.com/iculizhi/textarena-selfplay-qwen/tree/main/evaluation_results/"
    click AS "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/evaluation/analysis_summary.json"

    %% Monitoring & Utilities
    subgraph "Monitoring & Utilities"
        GMON["gpu_monitor.sh"]:::script
        QSC["quick_start.py"]:::compute
        LCD["long_cot_demo.py"]:::compute
        QS["quickstart.sh"]:::script
    end
    click GMON "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/scripts/utils/gpu_monitor.sh"
    click QSC "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/examples/quick_start.py"
    click LCD "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/examples/long_cot_demo.py"
    click QS "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/quickstart.sh"

    %% Test Suites
    TSU["tests/test_complete_system.py"]:::script
    click TSU "https://github.com/iculizhi/textarena-selfplay-qwen/blob/main/tests/test_complete_system.py"

    %% Flow connections
    CMYAML -->|provides configs| SP
    CMYAML -->|provides configs| CLI
    CMYAML -->|provides configs| GTS

    SP -->|collects raw logs| DC
    DC --> DF
    DF --> DP

    SP --> ME
    SP --> QA
    SP --> SA
    DF --> DP

    DP --> CLI
    CLI -->|fine-tunes| MODSTORE
    TALL --> CLI
    TPAR --> CLI
    TOOLS --> CLI
    MT -.-> CLI
    ST -.-> CLI

    MODSTORE --> GTS
    GTS --> MOE
    GMTS --> MOE
    MOE --> AR
    AR --> ER
    AR --> AS
    UFE --> MOE
    UTE --> MOE

    GMON -.-> CLI
    GMON -.-> MOE
    QS -.-> MAIN
    QSC -.-> MAIN
    LCD -.-> MAIN

    TSU -.-> MAIN

    %% Legend
    subgraph "Legend" 
        CM["Compute Modules"]:::compute
        DM["Data Stores"]:::datastore
        SC["Scripts/Orchestration"]:::script
        EX["External Dependencies"]:::external
        CF["Configuration"]:::config
    end

    classDef compute fill:#D0E1FF,stroke:#0366d6,color:#0366d6;
    classDef datastore fill:#DFFFE0,stroke:#228B22,color:#228B22;
    classDef script fill:#FFE7C0,stroke:#FFA500,color:#FF8C00;
    classDef external fill:#F0F0F0,stroke:#808080,color:#606060;
    classDef config fill:#E8E8F8,stroke:#6A5ACD,color:#483D8B;
    class CMYAML,CM compute
    class DP,MODSTORE,ER,AS datastore
    class GS,TALL,TPAR,MT,ST,TOOLS,UFE,UTE,GMON,QS script
    class CLI external
    class CM config
```
