How agi-brain/xuance works
Project Type and Purpose
• XuanCe is a full‐featured, open‐source Python library for Deep Reinforcement Learning (DRL) and Multi‐Agent RL (MARL).
• It provides:
• A unified API (xuance.get_runner) for training/testing agents.
• Modular implementations of dozens of single‐agent and multi‐agent algorithms.
• Adapters for three DL backends: PyTorch, TensorFlow2, MindSpore.
• Environment wrappers for Gymnasium, PettingZoo, MuJoCo, etc.
• Hyperparameter tuning, logging (TensorBoard, wandb), parallel & distributed training.
High‐Level File‐Structure Analysis
• Top level:
• setup.py, README, examples/, docs/, tests/
• xuance/ – core library package
• xuance/ subpackages:
• common – shared utilities (memory buffers, tuning tools, stats)
• configs – YAML default configs per algorithm/environment
• environment – single & multi‐agent environment wrappers & vector envs
• {mindspore, tensorflow, torch} – per‐backend Agents, Learners, Policies, Runners, Representations, Utils
• __init__.py – entry point exposing get_runner, etc.
• examples/ – runnable scripts demonstrating usage per algorithm
• tests/ – framework‐specific and tuning tests
README Insights
• Describes supported algorithms, environments, features.
• Emphasizes modularity, extensibility, multi‐backend compatibility.
• Shows example code: calling get_runner(method, env, env_id, is_test) → runner.run()
• Mentions external services: TensorBoard, wandb, mpi4py.
Main Components and Relationships
• User/API Layer:
• User code or example script → calls xuance.get_runner()
• Runner Factory:
• Reads method, env, config → instantiates a Runner object from appropriate backend subpackage.
• Config Loader:
• Merges YAML defaults (xuance/configs and docs/_static/configs) with user overrides.
• Environment Module:
• SingleAgentEnv / MultiAgentEnv (gymnasium, PettingZoo, custom envs)
• VectorEnv orchestrates multiple envs in parallel (dummy or subprocess).
• Agent Abstraction:
• Holds representation (MLP/CNN/RNN), policy (categorical/deterministic/gaussian), memory buffer.
• Learner Abstraction:
• Implements update logic: sample from memory, compute losses, backprop through chosen backend.
• Common Tools:
• Memory buffers (replay, prioritized, on‐policy), tuning tools, statistic tools.
• Hyperparameter Tuner:
• Interfaces with tuning_tools/hyperparameters and external services (WandB, TensorBoard).
• Backend Adapters:
• torch/, tensorflow/, mindspore/ subpackages each implement Agents/Learners/Runners with framework‐specific code.
• Logging & Visualization:
• Logs metrics to local files for TensorBoard, optionally pushes to WandB.
• External Services:
• mpi4py for distributed training, Gym/PettingZoo for environments, DL frameworks.
Architectural Patterns & Design Principles
• Factory Pattern: get_runner selects Runner class based on method and backend.
• Strategy Pattern: Agent+Learner pair encapsulates algorithm strategy.
• Adapter Pattern: Separate per‐backend code under torch/, tensorflow/, mindspore/.
• Layered Architecture: clear separation between:
1) API/Runner
2) Config management
3) Environment layer
4) Algorithm core (Agent & Learner)
5) Backend glue
6) Logging & tuning
• Modular & Pluggable: new algorithms, new envs, new backends can be added by following existing package structure.
Be Detailed
• Algorithm registration point (method → AlgorithmClass).
• Where configs (YAML) are loaded & applied.
• Multi‐agent vs single‐agent env wrapper difference.
• Parallel env (VectorEnv) and distribution (MPI).
• Per‐backend module boundaries.