# mango915/haliterl architecture

> Architecture diagram of the GitHub repository mango915/haliterl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mango915/haliterl
- Repository: https://github.com/mango915/haliterl

## Overview

##### Project Type and Purpose
- This project is a reinforcement learning (RL) application developed for solving the Halite challenge, where an agent collects resources on a map.
- Its purpose is twofold: to implement a custom Halite environment (compatible with the OpenAI Gym standard) and to train RL agents (using tabular methods) to navigate and collect halite efficiently.
- The README indicates that the project also serves as a demonstration of RL techniques (particularly tabular methods) with both single-ship and multi-agent implementations, along with tutorials to explain these methods.

##### File Structure
- Top-Level Directories:
- Environment – Contains the implementation of the Halite environment (e.g., “halite_env.py”), which likely conforms to OpenAI Gym standards.
- Modules – Houses auxiliary modules such as encoding (state representation), rendering (visualization of the game environment), and ship-specific functionality (ship.py, shipy.py).
- Tabular_methods – Contains Jupyter notebooks and related files (Q-values, weights, etc.) that implement and demonstrate the tabular RL methods on both single-ship and multi-agent scenarios.
- Tutorials – Contains guided notebooks and visual aids (GIFs, support materials) intended to help users understand the state representation, RL challenges, and improvements to the learning process.
- Other files:
- README.md – Provides context, running instructions (both via Conda and Docker), and links to additional resources.
- changelog, requirements.txt, and backup files – Indicate versioning, project dependencies, and backup support.

##### Main Components of the System
a. Halite Environment:
- Core engine (from Environment/halite_env.py) that models the game map, resource distribution, and game dynamics.
b. RL Agent (Policy and Learning Module):
- This is the heart of the learning process, depicted in Tabular_methods and spread across Modules – receiving state, computing Q-values, and deciding actions.
c. State Representation & Encoding Module:
- Module(s) for converting the game state into a format that is usable by the RL algorithms (e.g., Modules/encode.py, and similar functionality in Tabular_methods).
d. Ship & Action Modules:
- Components for handling ship behaviors and actions (Modules/ship.py, Modules/shipy.py, along with corresponding files in Tabular_methods).
e. Visualization/Rendering:
- Modules focused on visual feedback (Modules/rendering.py and the Demo GIF in Tutorials) to display the board’s state, Q-values heatmap, and overall performance.
f. User Interaction/UI Layer:
g. Deployment Components:
- Optional nodes representing the Conda environment setup and Docker container configuration (as shown in the README deployment instructions).

##### Relationships and Interactions
- The Environment drives the simulation and generates the game state.
- The RL Agent queries the Environment, receives state representations (processed by the encoder), and outputs actions.
- The Ship modules interpret these actions to update the game state.
- The Tutorials act as an interface layer that guides users in experimentation and understanding of these interactions.
- Deployment elements (Conda/Docker) encapsulate the entire system, ensuring reproducibility and ease of use.

##### Architectural Patterns & Design Principles
- Modular design: The clear separation between the environment, modules handling specific functionalities (state encoding, rendering), and notebooks for experimentation.
- Layering: Separation between core simulation (Environment), learning/decision-making (RL Agent and Tabular Methods), and visualization/user interaction interfaces.
- Reusability: Multiple folders (Modules and Tabular_methods) share similar functionalities, suggesting design aimed at reusability and extensibility.
- The project leverages interactive computational notebooks (Jupyter) to facilitate rapid prototyping and visualization of results.

##### Final Touches
- Be as detailed as possible: Ensure that any intermediates (such as modules doing encoding or simulation of ship dynamics) are included.

- How the Halite simulation is structured.
- How the RL agent interacts with the environment.
- The role of each module (state encoding, ship dynamics, rendering).
- The integration of user-facing interactive elements (Jupyter notebooks) and deployment methods (Conda/Docker).

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "Simulation Layer"
        HaliteEnv["Halite Environment"]:::sim
        subgraph "Ship Modules"
            ShipMod["Ship Module (Modules) - ship.py"]:::ship
            ShipMod2["Ship Module (Modules) - shipy.py"]:::ship
            ShipTab["Ship Module (Tabular_methods) - ship.py"]:::ship
            ShipTab2["Ship Module (Tabular_methods) - shipy.py"]:::ship
        end
    end

    subgraph "Learning Layer"
        RLAgent["RL Agent (Tabular Methods)"]:::agent
        subgraph "State Representation & Encoding"
            SE_Mod["State Encoder (Modules)"]:::encoder
            SE_Tab["State Encoder (Tabular_methods)"]:::encoder
        end
    end

    subgraph "Interface Layer"
        subgraph "Visualization"
            VizMod["Visualization Module (Modules)"]:::viz
            VizTut["Visualization Notebook (Tutorials)"]:::viz
        end
        subgraph "User Interaction"
            TutEnv["Tutorial: Halite Environment"]:::tutorial
            TutQL["Tutorial: Q Learning Tabular Methods"]:::tutorial
        end
    end

    subgraph "Deployment"
        DeployRead["Deployment Instructions (README)"]:::deploy
        DeployReq["Requirements File"]:::deploy
    end

    %% Data Flow Connections
    HaliteEnv -- "state" --> SE_Mod
    HaliteEnv -- "state" --> SE_Tab
    SE_Mod -- "processed_state" --> RLAgent
    SE_Tab -- "processed_state" --> RLAgent

    RLAgent -- "action" --> ShipMod
    RLAgent -- "action" --> ShipMod2
    RLAgent -- "action" --> ShipTab
    RLAgent -- "action" --> ShipTab2

    ShipMod -- "update" --> HaliteEnv
    ShipMod2 -- "update" --> HaliteEnv
    ShipTab -- "update" --> HaliteEnv
    ShipTab2 -- "update" --> HaliteEnv

    HaliteEnv -- "state" --> VizMod
    RLAgent -- "Q-values" --> VizTut

    TutEnv -- "experiment" --> RLAgent
    TutQL -- "configure" --> RLAgent

    DeployRead -- "deploy" --> HaliteEnv
    DeployReq -- "dependencies" --> RLAgent

    TutEnv -- "learn" --> VizTut
    TutQL -- "learn" --> VizTut

    %% Styles
    classDef sim fill:#f9f,stroke:#333,stroke-width:2px;
    classDef ship fill:#ccf,stroke:#333,stroke-width:2px;
    classDef agent fill:#cfc,stroke:#333,stroke-width:2px;
    classDef encoder fill:#fc9,stroke:#333,stroke-width:2px;
    classDef viz fill:#cff,stroke:#333,stroke-width:2px;
    classDef tutorial fill:#ffc,stroke:#333,stroke-width:2px;
    classDef deploy fill:#f99,stroke:#333,stroke-width:2px;

    %% Click Events
    click HaliteEnv "https://github.com/mango915/haliterl/blob/master/Environment/halite_env.py"
    click RLAgent "https://github.com/mango915/haliterl/tree/master/Tabular_methods"
    click SE_Mod "https://github.com/mango915/haliterl/blob/master/Modules/encode.py"
    click SE_Tab "https://github.com/mango915/haliterl/blob/master/Tabular_methods/encode.py"
    click ShipMod "https://github.com/mango915/haliterl/blob/master/Modules/ship.py"
    click ShipMod2 "https://github.com/mango915/haliterl/blob/master/Modules/shipy.py"
    click ShipTab "https://github.com/mango915/haliterl/blob/master/Tabular_methods/ship.py"
    click ShipTab2 "https://github.com/mango915/haliterl/blob/master/Tabular_methods/shipy.py"
    click VizMod "https://github.com/mango915/haliterl/blob/master/Modules/rendering.py"
    click VizTut "https://github.com/mango915/haliterl/blob/master/Tutorials/Bonus_Track_Rendering.ipynb"
    click TutEnv "https://github.com/mango915/haliterl/blob/master/Tutorials/Tutorial_1_Halite_Environment.ipynb"
    click TutQL "https://github.com/mango915/haliterl/blob/master/Tutorials/Tutorial_4_Q_Learning_Tabular_Methods.ipynb"
    click DeployRead "https://github.com/mango915/haliterl/blob/master/README.md"
    click DeployReq "https://github.com/mango915/haliterl/blob/master/requirements.txt"
```
