How iculizhi/textarena-selfplay-qwen works
Project Type and Purpose
• This is an ML training & evaluation pipeline (open-source tool) for Tic-Tac-Toe Chain-of-Thought (CoT) models built on Qwen.
• Purpose:
• Generate self-play game data at six CoT lengths
• Train separate LoRA-fine-tuned Qwen models per length (multi-GPU)
• Evaluate models on standardized test sets (single/multi-GPU)
• Provide monitoring and analysis tools
Main Architectural Components
A. Configuration Manager
• Directory: configs/, llama_factory_configs/, scripts/training/llama_factory_configs/
• Files: .yaml and .json that drive dataset parameters, self-play, training, evaluation
B. Data Generation Module
• Core Code: src/data_generation/
• data_collector.py
• selfplay_runner.py
• Scripts: scripts/data_generation/generate_all_cot_lengths_serial.py
• Output: data/processed/*.json (CoT datasets + test sets)
C. Training Orchestration
• Core CLI: llamafactory-cli (external LLaMA-Factory)
• Scripts:
• scripts/training/train_all_models_parallel.sh
• scripts/training/train_all_models.sh
• monitor_training.sh / stop_all_training.sh
• Models output: models/qwen_*_cot_lora
D. Evaluation Orchestration
• Core Code: evaluation/*.py
• generate_test_set.py, multi_optimal_evaluator.py, analyze_results.py
• Scripts: scripts/evaluation/ultra_fast_eval.sh, ultra_turbo_evaluator.py
• Output: evaluation_results/*.json and analysis_summary.json
E. Monitoring & Utilities
• GPU/Training Monitor: scripts/utils/gpu_monitor.sh, monitor_training.sh
• Quickstart & Demo: examples/quick_start.py, examples/long_cot_demo.py
• Top-level helpers: quickstart.sh, monitor_training.sh, stop_all_training.sh
F. Core Agents & Game Logic
• src/agents/qwen_agent.py, src/agents/smart_agent.py
• src/utils/game_parser.py, mock_env.py (for self-play)
• src/main.py (entry point)
Component Interactions and Data Flow
1. CONFIG LOAD →
• Training configs (YAML/JSON) feed into self-play runner & LLaMA-Factory CLI
• Dataset configs feed data generation & evaluation
2. DATA GENERATION →
• Self-play runner uses qwen_agent & mock_env → produces raw game logs
• data_formatter.py cleans/structures logs → saved under data/processed/*.json
3. MODEL TRAINING →
• LLaMA-Factory picks up each config → fine-tunes Qwen models on processed CoT data
• Training scripts dispatch jobs to GPU cluster (parallel/sharded)
• Models checkpointed under models/
4. EVALUATION →
• generate_test_set.py or test_set_avoidance scripts produce test files
• model_evaluator.py / ultra_turbo_evaluator loads models + test data → inference & metrics
• Results aggregated into evaluation_results/ & analysis_summary.json
5. MONITORING & ANALYSIS →
• GPU_MONITOR & monitor_training track resource use and logs
• analysis_summary.json + scripts/analysis tools produce plots and tables
Architectural Patterns & Principles
• Pipeline pattern: clear separation between data generation, training, evaluation
• Modular design: each stage has its own folder & scripts
• Config-driven orchestration: YAML/JSON files parameterize every step
• Parallel & distributed training/evaluation via shell scripts + multi-GPU workers
• Extensible: add new CoT lengths by supplying new configs and reusing same code
Essential Elements
• Labels for all YAML/JSON configs and their target stages.
• A “Zoom-in” inset for a single CoT-length training flow, then a “Zoom-out” view showing fan-out for six models.
• External dependencies: GitHub (LLaMA-Factory), Qwen model repository.