# comeanotherday4/mutil_agent_path_planning architecture

> Architecture diagram of the GitHub repository comeanotherday4/mutil_agent_path_planning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/comeanotherday4/mutil_agent_path_planning
- Repository: https://github.com/comeanotherday4/mutil_agent_path_planning

## Overview

This research project compares reinforcement-learning methods for resource-constrained desert route planning across three scenarios. Users run a scenario script to simulate travel, resource purchases and mining under weather and deadline constraints, then inspect learned or optimized routes and outcomes. Scenario 1 compares dynamic programming with Q-learning for known weather; scenario 2 evaluates Q-learning under four initial resource mixes; scenario 3 compares fixed-policy-plus-PPO, IPPO and MAPPO in a three-agent setting. The source excerpts are partial, so links between unsampled implementations are omitted; the DP capability is documented in the README.

Main entry points: [`code/q2.py`](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q2.py) (Unknown-weather environment), [`code/q3_fix.py`](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_fix.py) (Three-player environment), [`code/q3_mappo.py`](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_mappo.py) (Three-player environment), [`code/q1.py`](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q1.py) (Known-weather environment)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_single["Single-agent scenarios"]
  node_q1env["Known-weather environment<br/>[q1.py]"]
  node_q1methods["DP and Q-learning<br/>[q1.py]"]
  node_q1result["Route comparison<br/>[q1.py]"]
  node_q2env["Unknown-weather environment<br/>[q2.py]"]
  node_q2strategies["Initial-buy strategies<br/>[q2.py]"]
  node_q2result["Strategy outcomes<br/>[q2.py]"]
end

subgraph group_multi["Multi-agent scenario"]
  node_fixenv["Three-player environment<br/>[q3_fix.py]"]
  node_fixedpolicies["Fixed player policies<br/>[q3_fix.py]"]
  node_fixresult["Fixed-plus-PPO results<br/>[q3_fix.py]"]
  node_ippoenv["Three-player environment<br/>[q3_ippo.py]"]
  node_ipporesult["IPPO results<br/>[q3_ippo.py]"]
  node_mappoenv["Three-player environment<br/>[q3_mappo.py]"]
  node_mapporesult["MAPPO results<br/>[q3_mappo.py]"]
end

subgraph group_learning["Learning components"]
  node_q2learner["Q-learning agent<br/>[q2.py]"]
  node_fixppo["Player-three PPO<br/>[q3_fix.py]"]
  node_ippoagent["Independent PPO agents<br/>[q3_ippo.py]"]
  node_mappoactors["Distributed actors<br/>[q3_mappo.py]"]
  node_critic["Centralized critic<br/>[q3_mappo.py]"]
end

node_user(("Researcher"))

node_user -->|"runs scenario"| node_q1env
node_q1env -->|"provides transitions"| node_q1methods
node_q1methods -->|"compares routes"| node_q1result
node_user -->|"runs scenario"| node_q2env
node_q2env -->|"evaluates purchases"| node_q2strategies
node_q2env -->|"provides transitions"| node_q2learner
node_q2strategies -->|"reports outcomes"| node_q2result
node_q2learner -->|"learns policy"| node_q2result
node_user -->|"runs scenario"| node_fixenv
node_fixenv -->|"uses policies"| node_fixedpolicies
node_fixenv -->|"provides observations"| node_fixppo
node_fixppo -->|"trains policy"| node_fixresult
node_user -->|"runs scenario"| node_ippoenv
node_ippoenv -->|"provides observations"| node_ippoagent
node_ippoagent -->|"trains agents"| node_ipporesult
node_user -->|"runs scenario"| node_mappoenv
node_mappoenv -->|"provides observations"| node_mappoactors
node_mappoenv -->|"provides global state"| node_critic
node_mappoactors -->|"trains actors"| node_mapporesult
node_critic -->|"trains critic"| node_mapporesult

click node_q1env "https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q1.py"
click node_q1methods "https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q1.py"
click node_q1result "https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q1.py"
click node_q2env "https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q2.py"
click node_q2strategies "https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q2.py"
click node_q2learner "https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q2.py"
click node_q2result "https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q2.py"
click node_fixenv "https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_fix.py"
click node_fixedpolicies "https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_fix.py"
click node_fixppo "https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_fix.py"
click node_fixresult "https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_fix.py"
click node_ippoenv "https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_ippo.py"
click node_ippoagent "https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_ippo.py"
click node_ipporesult "https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_ippo.py"
click node_mappoenv "https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_mappo.py"
click node_mappoactors "https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_mappo.py"
click node_critic "https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_mappo.py"
click node_mapporesult "https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_mappo.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_q1env,node_q1methods,node_q1result,node_q2env,node_q2strategies,node_q2result toneBlue
class node_fixenv,node_fixedpolicies,node_fixresult,node_ippoenv,node_ipporesult,node_mappoenv,node_mapporesult toneAmber
class node_q2learner,node_fixppo,node_ippoagent,node_mappoactors,node_critic toneMint
class node_user toneIndigo
```

## Components

### Single-agent scenarios

- [Known-weather environment](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q1.py): `code/q1.py`
- [DP and Q-learning](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q1.py): `code/q1.py`
- [Route comparison](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q1.py): `code/q1.py`
- [Unknown-weather environment](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q2.py): `code/q2.py`
- [Initial-buy strategies](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q2.py): `code/q2.py`
- [Strategy outcomes](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q2.py): `code/q2.py`

### Multi-agent scenario

- [Three-player environment](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_fix.py): `code/q3_fix.py`
- [Fixed player policies](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_fix.py): `code/q3_fix.py`
- [Fixed-plus-PPO results](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_fix.py): `code/q3_fix.py`
- [Three-player environment](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_ippo.py): `code/q3_ippo.py`
- [IPPO results](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_ippo.py): `code/q3_ippo.py`
- [Three-player environment](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_mappo.py): `code/q3_mappo.py`
- [MAPPO results](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_mappo.py): `code/q3_mappo.py`

### Learning components

- [Q-learning agent](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q2.py): `code/q2.py`
- [Player-three PPO](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_fix.py): `code/q3_fix.py`
- [Independent PPO agents](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_ippo.py): `code/q3_ippo.py`
- [Distributed actors](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_mappo.py): `code/q3_mappo.py`
- [Centralized critic](https://github.com/comeanotherday4/mutil_agent_path_planning/blob/main/code/q3_mappo.py): `code/q3_mappo.py`

### Other components

- **Researcher**

## Connections

- Researcher → Known-weather environment: runs scenario
- Known-weather environment → DP and Q-learning: provides transitions
- DP and Q-learning → Route comparison: compares routes
- Researcher → Unknown-weather environment: runs scenario
- Unknown-weather environment → Initial-buy strategies: evaluates purchases
- Unknown-weather environment → Q-learning agent: provides transitions
- Initial-buy strategies → Strategy outcomes: reports outcomes
- Q-learning agent → Strategy outcomes: learns policy
- Researcher → Three-player environment: runs scenario
- Three-player environment → Fixed player policies: uses policies
- Three-player environment → Player-three PPO: provides observations
- Player-three PPO → Fixed-plus-PPO results: trains policy
- Researcher → Three-player environment: runs scenario
- Three-player environment → Independent PPO agents: provides observations
- Independent PPO agents → IPPO results: trains agents
- Researcher → Three-player environment: runs scenario
- Three-player environment → Distributed actors: provides observations
- Three-player environment → Centralized critic: provides global state
- Distributed actors → MAPPO results: trains actors
- Centralized critic → MAPPO results: trains critic