# liuruoze/mini-alphastar architecture

> Architecture diagram of the GitHub repository liuruoze/mini-alphastar, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/liuruoze/mini-alphastar
- Repository: https://github.com/liuruoze/mini-alphastar

## Overview

##### Major Components
- Core Architecture (alphastarmini/core/arch): Contains modules for the deep neural network architecture. Files such as agent.py, arch_model.py, and various “head” files (e.g., action_type_head.py, delay_head.py) indicate that separate neural network components or layers have been modularized.
- Supervised Learning (alphastarmini/core/sl): Implements the supervised learning pipeline. Modules include dataset.py, transform_replay_data.py, and multiple variants for training (by tensor, by pickle, single GPU vs. multi-GPU).
- Reinforcement Learning (alphastarmini/core/rl): Contains the reinforcement learning modules. Look at actor.py, learner.py, rl_train_with_replay.py, and utilities for RL – these are critical for the RL training and evaluation phases.
- Multi-agent (alphastarmini/core/ma): Contains components for managing multi-agent training (coordinator, league, etc.) that are used to set up the environment and manage training for multiple agents.
- Utility and Support Libraries (alphastarmini/lib, alphastarmini/third): Provide supporting functionalities and external integrations. For instance, transformer components and additional helper functions are encapsulated here.
- Scripts and Documentation: Top-level scripts (run.py, test.py) and files in the “doc” and “res” directories supply instructions, pseudocode, and historical context.

##### Relationships and Interactions
- Data Flow: Begin with “replay” data that is downloaded (instructions in the README and scripts in the “scripts” folder) and then transformed by the transform_replay_data module into training data.
- Supervised Learning Pipeline: Training data is passed through the supervised learning modules in core/sl where an initial model is trained. This model is evaluated in the SC2 environment via the evaluation module in core/rl.
- Reinforcement Learning Pipeline: After validating the SL model via evaluation, the RL modules take over. Reinforcement learning is driven by multi-processing (see files like rl_vs_inner_bot_mp.py) and involves actors, learners, and pseudo-reward modules to further train the agent.
- Multi-agent Coordination: Components from core/ma work in tandem to organize the training across multiple agents or processes.
- Neural Architecture: The deep learning parts (architectural “heads” and encoders) interact with both training pipelines to produce the action decisions in gameplay.

##### Architectural Patterns and Design Principles
- Modular Design: The project adheres to separation of concerns by distinguishing modules for network architecture, supervised learning, reinforcement learning, and multi-agent coordination.
- Pipeline Flow: The setup clearly defines a sequential pipeline starting from data transformation, through initial SL training, then evaluation, and finally to RL correction.
- Extensibility: With packages segmented into core and library functions, the design encourages extensibility and potential replacement of components (for instance, swapping SL training strategies).

##### Relevant Technologies
- PyTorch is the central dependency (as specified in the README and requirements.txt) and is used throughout for network construction and training.
- Python’s built-in modules handle multi-processing and threading (particularly for the RL experiments where multiple processes and threads are spawned).

##### Detailed Elements to Be Included
- Clear Labels: Mark major modules (for instance, “Supervised Learning (core/sl)”, “Reinforcement Learning (core/rl)”, “Multi-agent (core/ma)” and “Deep Neural Architecture (core/arch)”).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Ingestion and Transformation
    subgraph "Data Ingestion & Transformation"
        A["Replay Data"]:::data
        B["Transform Replay Data"]:::data
    end
    A -->|"transforms"| B

    %% Training Pipeline
    subgraph "Training Pipeline"
        F["Deep Neural Architecture (core/arch)"]:::arch
        C["Supervised Learning (core/sl)"]:::sl
        D["Reinforcement Learning (core/rl)"]:::rl
        E["Multi-agent Coordination (core/ma)"]:::ma
    end
    B -->|"feeds"| C
    C -->|"evaluation"| D
    D -->|"coordination"| E
    F -->|"supports"| C
    F -->|"supports"| D

    %% Support Libraries
    subgraph "Support Libraries"
        G["Utility Libraries (alphastarmini/lib/)"]:::lib
        H["Third Party Libraries (alphastarmini/third/)"]:::third
    end
    C -->|"utilizes"| G
    C -->|"utilizes"| H
    D -->|"utilizes"| G
    D -->|"utilizes"| H
    E -->|"utilizes"| G
    E -->|"utilizes"| H

    %% Scripts and Documentation
    subgraph "Scripts and Docs"
        I["Scripts & Documentation (run.py, test.py, doc, scripts)"]:::scripts
    end
    I -->|"orchestrates"| A

    %% Click Events
    click F "https://github.com/liuruoze/mini-alphastar/tree/main/alphastarmini/core/arch/"
    click C "https://github.com/liuruoze/mini-alphastar/tree/main/alphastarmini/core/sl/"
    click D "https://github.com/liuruoze/mini-alphastar/tree/main/alphastarmini/core/rl/"
    click E "https://github.com/liuruoze/mini-alphastar/tree/main/alphastarmini/core/ma/"
    click G "https://github.com/liuruoze/mini-alphastar/tree/main/alphastarmini/lib/"
    click H "https://github.com/liuruoze/mini-alphastar/tree/main/alphastarmini/third/"
    click I "https://github.com/liuruoze/mini-alphastar/blob/main/run.py"

    %% Styles
    classDef data fill:#ffebcd,stroke:#cd853f,stroke-width:2px;
    classDef sl fill:#c9daf8,stroke:#6d9eeb,stroke-width:2px;
    classDef rl fill:#f4cccc,stroke:#d22d2d,stroke-width:2px;
    classDef ma fill:#d9ead3,stroke:#38761d,stroke-width:2px;
    classDef arch fill:#d9d2e9,stroke:#674ea7,stroke-width:2px;
    classDef lib fill:#fff2cc,stroke:#bf9000,stroke-width:2px;
    classDef third fill:#fce5cd,stroke:#e69138,stroke-width:2px;
    classDef scripts fill:#d0e0e3,stroke:#115e67,stroke-width:2px;
```
