# damasioj/ml-agents-fsm architecture

> Architecture diagram of the GitHub repository damasioj/ml-agents-fsm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/damasioj/ml-agents-fsm
- Repository: https://github.com/damasioj/ml-agents-fsm

## Overview

##### Project Type and Purpose
- This is a Unity-based simulation environment integrating manually programmed Finite State Machines (FSMs) and Reinforcement Learning (RL) agents via the Unity ML-Agents SDK.
- Purpose: Compare handcrafted AI (FSM) against ML-trained agents in an ecosystem scenario (paper experiment).

##### Key Components (to represent
A. External Trainer
- Python RL Trainer (mlagents Python API, version 0.16.1)
B. Unity Environment
1. Scene Manager
– Loads the “Ecosystem” scene
– Hosts all GameObjects and coordinates simulation steps
2. Environment Manager (EnvironmentManager.cs)
– Orchestrates environment update loop
– Spawns/despawns resources, structures, targets
3. Agents
a. Reinforcement Learning Agents (AnimalAgent.cs, CollectorAgent.cs, HaulerAgent.cs)
– Inherits from Unity’s Agent base class
– Uses ML-Agents Communicator for obs/actions
b. Manual FSM Agents (CollectorAgentABFSM.cs, CollectorAgentTBFSM.cs, ManualAgent.cs)
– Implements IHasGoal & AgentState FSM states (Idle, Move, Gather, Haul, Build, etc.)
4. Environment Objects
– Sources (AppleSource, WoodSource, StoneSource, FoodSource)
– Resources (AppleResource, WoodResource, StoneResource, BambooResource)
– Structures & Targets (ResourceDepot, HouseStructure, FlatBoxTarget, Rock, Apple Trees)
5. Utility Services
– Pathfinding (BasicPathfinder.cs)
– Helper (ObjectHelper.cs)
6. Asset Database
– Prefabs (Agents, Environments, Goals, Targets, Zones)
– Models (.nn neural-network files for pretrained models)
– Materials, Timers (configuration JSON), Scenes

##### Relationships & Data Flow
- Python Trainer ↔ Unity Environment
- Observations (vector, visual) sent from Unity → Python
- Actions returned from Python → Unity
– Communication via gRPC under ML-Agents SDK
- Within Unity
- EnvironmentManager updates scene → provides sensory data to Agents
- Agents compute next action either by:
– FSM module (AgentState.cs → next state)
– ML-Agents decision request (OnActionReceived)
- Actions applied to environment (move agent, collect resource, build structure)
- Environment changes fed back as observations

##### Architectural Patterns & Design Principles
- Modular separation:
- “Environment” = world logic and objects
- “Agents” = decision logic (FSM vs RL)
- “Utilities” = common services (pathfinding, helpers)
- FSM pattern: explicit state classes for manual agents
- Agent-Environment interface: conforms to RL Gym-style API via ML-Agents

##### Technologies & Frameworks
- Unity 2019.2.0f1+ (C# scripting, Scenes, Prefabs)
- Unity ML-Agents SDK (C# Agent base, Python trainer)
- Python 3.6+ with mlagents v0.16.1
- JSON for configuration (Timers/Ecosystem_timers.json)
- Excel/ pdata files for result aggregation (Results folder)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Service
    PythonTrainer["Python RL Trainer"]:::external

    %% Unity Simulation
    subgraph "Unity Simulation"
        direction TB

        SceneManager["Scene Manager"]:::env
        EnvironmentManager["Environment Manager"]:::env

        subgraph "Agents"
            direction TB
            subgraph "Reinforcement Learning Agents"
                direction TB
                AnimalAgent["AnimalAgent.cs"]:::agents
                CollectorAgentRL["CollectorAgent.cs"]:::agents
                HaulerAgent["HaulerAgent.cs"]:::agents
            end
            subgraph "Manual FSM Agents"
                direction TB
                ManualAgent["ManualAgent.cs"]:::agents
                CollectorABFSM["CollectorAgentABFSM.cs"]:::agents
                CollectorTBFSM["CollectorAgentTBFSM.cs"]:::agents
                subgraph "FSM States"
                    direction TB
                    AgentState["AgentState.cs"]:::agents
                    IdleState["IdleState.cs"]:::agents
                    MoveState["MoveState.cs"]:::agents
                    GatherState["MGatherState.cs"]:::agents
                    HaulState["HaulState.cs"]:::agents
                end
            end
        end

        subgraph "Environment Objects"
            direction TB
            subgraph "Sources"
                direction TB
                AppleSource["AppleSource.cs"]:::env
                WoodSource["WoodSource.cs"]:::env
                StoneSource["StoneSource.cs"]:::env
                FoodSource["FoodSource.cs"]:::env
            end
            subgraph "Resources"
                direction TB
                AppleResource["AppleResource.cs"]:::env
                WoodResource["WoodResource.cs"]:::env
                StoneResource["StoneResource.cs"]:::env
                BambooResource["BambooResource.cs"]:::env
            end
            subgraph "Structures & Targets"
                direction TB
                ResourceDepot["ResourceDepot.cs"]:::env
                HouseStructure["HouseStructure.cs"]:::env
                FlatBoxTarget["FlatBoxTarget.cs"]:::env
                BaseTarget["BaseTarget.cs"]:::env
            end
        end

        subgraph "Utility Services"
            direction TB
            Pathfinding["BasicPathfinder.cs"]:::util
            ObjectHelper["ObjectHelper.cs"]:::util
        end

        subgraph "Asset Database"
            direction TB
            Prefabs[(Prefabs)]:::store
            Models[(.nn Models)]:::store
            Timers[(Ecosystem_timers.json)]:::store
        end
    end

    %% Data Stores
    Results[(Results Folder)]:::store

    %% Connections
    PythonTrainer -->|"Actions"| EnvironmentManager
    EnvironmentManager -->|"Observations"| PythonTrainer

    EnvironmentManager -->|"Perception"| AnimalAgent
    EnvironmentManager -->|"Perception"| CollectorAgentRL
    EnvironmentManager -->|"Perception"| HaulerAgent
    EnvironmentManager -->|"Perception"| ManualAgent

    AnimalAgent -->|"Action"| EnvironmentManager
    CollectorAgentRL -->|"Action"| EnvironmentManager
    HaulerAgent -->|"Action"| EnvironmentManager
    ManualAgent -->|"Action"| EnvironmentManager

    SceneManager --> EnvironmentManager
    EnvironmentManager --> AppleSource
    EnvironmentManager --> AppleResource
    EnvironmentManager --> ResourceDepot

    %% Click Events
    click SceneManager "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scenes/Ecosystem.unity"
    click EnvironmentManager "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Environment/EnvironmentManager.cs"
    click AnimalAgent "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Agent/AnimalAgent.cs"
    click CollectorAgentRL "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Agent/CollectorAgent.cs"
    click HaulerAgent "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Agent/HaulerAgent.cs"
    click ManualAgent "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Agent/ManualAgent.cs"
    click CollectorABFSM "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Agent/CollectorAgentABFSM.cs"
    click CollectorTBFSM "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Agent/CollectorAgentTBFSM.cs"
    click AgentState "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Agent/FSM/AgentState.cs"
    click IdleState "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Agent/FSM/IdleState.cs"
    click MoveState "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Agent/FSM/MoveState.cs"
    click GatherState "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Agent/FSM/MGatherState.cs"
    click AppleSource "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Environment/Source/AppleSource.cs"
    click WoodSource "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Environment/Source/WoodSource.cs"
    click StoneSource "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Environment/Source/StoneSource.cs"
    click FoodSource "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Environment/Source/FoodSource.cs"
    click AppleResource "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Environment/Resource/AppleResource.cs"
    click WoodResource "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Environment/Resource/WoodResource.cs"
    click StoneResource "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Environment/Resource/StoneResource.cs"
    click BambooResource "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Environment/Resource/BambooResource.cs"
    click ResourceDepot "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Environment/Structure/ResourceDepot.cs"
    click HouseStructure "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Environment/Structure/HouseStructure.cs"
    click FlatBoxTarget "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Environment/Structure/FlatBoxTarget.cs"
    click BaseTarget "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Environment/BaseTarget.cs"
    click Pathfinding "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Util/BasicPathfinder.cs"
    click ObjectHelper "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/Scripts/Util/ObjectHelper.cs"
    click Prefabs "https://github.com/damasioj/ml-agents-fsm/tree/main/Assets/Prefabs"
    click Models "https://github.com/damasioj/ml-agents-fsm/tree/main/Assets/Models"
    click Timers "https://github.com/damasioj/ml-agents-fsm/blob/main/Assets/ML-Agents/Timers/Ecosystem_timers.json"

    %% Styles
    classDef external fill:#BBDEFB,stroke:#1E88E5,color:#0D47A1;
    classDef env fill:#C8E6C9,stroke:#388E3C,color:#1B5E20;
    classDef agents fill:#FFE0B2,stroke:#FB8C00,color:#E65100;
    classDef util fill:#ECEFF1,stroke:#90A4AE,color:#37474F;
    classDef store fill:#ECEFF1,stroke:#90A4AE,color:#37474F;
```
