# haricharan-25/q-learning architecture

> Architecture diagram of the GitHub repository haricharan-25/q-learning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/haricharan-25/q-learning
- Repository: https://github.com/haricharan-25/q-learning

## Overview

This is an educational Q-learning grid-world visualizer: users configure a grid, run or train an agent, then inspect movement, Bellman updates, the Q-table, and learning metrics. The browser’s shared context drives the simulator and training through backend APIs; the backend engine owns environment and agent behavior. A separate Python workflow trains local-observation agents on generated grids. Edges follow shown calls, imports, or documented API behavior; several UI/backend details and unsampled capabilities are represented at subsystem level because their implementation was not excerpted.

Main entry points: [`frontend/src/App.jsx`](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/App.jsx) (Dashboard shell), [`frontend/src/pages/GridSimulatorPage.jsx`](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/pages/GridSimulatorPage.jsx) (Grid simulator), [`backend/q_engine.py`](https://github.com/haricharan-25/q-learning/blob/main/backend/q_engine.py) (Simulation and training engine), [`frontend/src/pages/QTableExplorerPage.jsx`](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/pages/QTableExplorerPage.jsx) (Q-table explorer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ui["Interactive dashboard"]
  node_app["Dashboard shell<br/>[App.jsx]"]
  node_context["Shared state and API"]
  node_simpage["Grid simulator"]
  node_gridboard["Grid editor<br/>[GridBoard.jsx]"]
  node_controls["Simulation controls<br/>[GridControls.jsx]"]
  node_telemetry["Step telemetry<br/>[StepDetails.jsx]"]
  node_thinking["Agent reasoning<br/>[AgentThinking.jsx]"]
  node_alerts["Collision and loop alerts<br/>[LoopAlert.jsx]"]
  node_tablepage["Q-table explorer"]
  node_qmatrix["Q-table matrix<br/>[QTableGrid.jsx]"]
  node_bellman["Bellman inspector"]
  node_actionchart["Action comparison"]
  node_analyticspage["Learning analytics"]
  node_charts["Training charts<br/>[TrainingCharts.jsx]"]
  node_metrics["Metric cards<br/>[MetricCards.jsx]"]
  node_settings["Learning settings<br/>[SettingsModal.jsx]"]
end

subgraph group_service["Simulation service"]
  node_api["HTTP and WebSocket API<br/>[main.py]"]
  node_engine["Simulation and training engine<br/>[q_engine.py]"]
end

subgraph group_learning["Learning runtime"]
  node_environment["Grid environment<br/>[grid_env.py]"]
  node_coordinateagent["Coordinate Q-agent<br/>[q_learning.py]"]
  node_localagent["Local-state Q-agent"]
  node_generator["Obstacle generator<br/>[grid_generator.py]"]
end

subgraph group_offline["Offline experiments"]
  node_localtraining["Local-state trainer<br/>[train_local.py]"]
  node_training["Coordinate trainer<br/>[train.py]"]
  node_demo["Generalization demo<br/>[main.py]"]
end

node_user(("Learner"))

node_user -->|"uses"| node_app
node_app -->|"provides state"| node_context
node_app -->|"renders"| node_simpage
node_app -->|"renders"| node_tablepage
node_app -->|"renders"| node_analyticspage
node_app -->|"renders"| node_settings
node_simpage -->|"renders"| node_gridboard
node_simpage -->|"renders"| node_controls
node_simpage -->|"renders"| node_telemetry
node_simpage -->|"renders"| node_thinking
node_simpage -->|"renders"| node_alerts
node_tablepage -->|"renders"| node_qmatrix
node_tablepage -->|"renders"| node_bellman
node_tablepage -->|"renders"| node_actionchart
node_analyticspage -->|"renders"| node_charts
node_analyticspage -->|"renders"| node_metrics
node_context -->|"requests API"| node_api
node_api -->|"delegates to"| node_engine
node_engine -->|"steps grid"| node_environment
node_engine -->|"uses agent"| node_coordinateagent
node_engine -->|"uses agent"| node_localagent
node_engine -->|"generates grids"| node_generator
node_training -->|"trains against"| node_environment
node_training -->|"updates agent"| node_coordinateagent
node_training -->|"creates layouts"| node_generator
node_localtraining -->|"trains against"| node_environment
node_localtraining -->|"updates agent"| node_localagent
node_localtraining -->|"creates layouts"| node_generator
node_demo -->|"trains policy"| node_localtraining
node_demo -->|"tests on grids"| node_environment
node_demo -->|"generates layouts"| node_generator

click node_app "https://github.com/haricharan-25/q-learning/blob/main/frontend/src/App.jsx"
click node_context "https://github.com/haricharan-25/q-learning/blob/main/frontend/src/context/QLearningContext.jsx"
click node_simpage "https://github.com/haricharan-25/q-learning/blob/main/frontend/src/pages/GridSimulatorPage.jsx"
click node_gridboard "https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/GridSimulator/GridBoard.jsx"
click node_controls "https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/GridSimulator/GridControls.jsx"
click node_telemetry "https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/GridSimulator/StepDetails.jsx"
click node_thinking "https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/GridSimulator/AgentThinking.jsx"
click node_alerts "https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/GridSimulator/LoopAlert.jsx"
click node_tablepage "https://github.com/haricharan-25/q-learning/blob/main/frontend/src/pages/QTableExplorerPage.jsx"
click node_qmatrix "https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/QTableExplorer/QTableGrid.jsx"
click node_bellman "https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/QTableExplorer/BellmanCalculator.jsx"
click node_actionchart "https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/QTableExplorer/StateActionChart.jsx"
click node_analyticspage "https://github.com/haricharan-25/q-learning/blob/main/frontend/src/pages/LearningAnalyticsPage.jsx"
click node_charts "https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/Analytics/TrainingCharts.jsx"
click node_metrics "https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/Analytics/MetricCards.jsx"
click node_settings "https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/SettingsModal.jsx"
click node_api "https://github.com/haricharan-25/q-learning/blob/main/backend/main.py"
click node_engine "https://github.com/haricharan-25/q-learning/blob/main/backend/q_engine.py"
click node_environment "https://github.com/haricharan-25/q-learning/blob/main/Q-learning/environment/grid_env.py"
click node_coordinateagent "https://github.com/haricharan-25/q-learning/blob/main/Q-learning/agent/q_learning.py"
click node_localagent "https://github.com/haricharan-25/q-learning/blob/main/Q-learning/agent/local_q_learning.py"
click node_generator "https://github.com/haricharan-25/q-learning/blob/main/Q-learning/utils/grid_generator.py"
click node_localtraining "https://github.com/haricharan-25/q-learning/blob/main/Q-learning/training/train_local.py"
click node_training "https://github.com/haricharan-25/q-learning/blob/main/Q-learning/training/train.py"
click node_demo "https://github.com/haricharan-25/q-learning/blob/main/Q-learning/main.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_context,node_simpage,node_gridboard,node_controls,node_telemetry,node_thinking,node_alerts,node_tablepage,node_qmatrix,node_bellman,node_actionchart,node_analyticspage,node_charts,node_metrics,node_settings toneBlue
class node_api,node_engine toneAmber
class node_environment,node_coordinateagent,node_localagent,node_generator toneMint
class node_localtraining,node_training,node_demo toneRose
class node_user toneIndigo
```

## Components

### Interactive dashboard

- [Dashboard shell](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/App.jsx): `frontend/src/App.jsx`
- [Shared state and API](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/context/QLearningContext.jsx): `frontend/src/context/QLearningContext.jsx`
- [Grid simulator](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/pages/GridSimulatorPage.jsx): `frontend/src/pages/GridSimulatorPage.jsx`
- [Grid editor](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/GridSimulator/GridBoard.jsx): `frontend/src/components/GridSimulator/GridBoard.jsx`
- [Simulation controls](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/GridSimulator/GridControls.jsx): `frontend/src/components/GridSimulator/GridControls.jsx`
- [Step telemetry](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/GridSimulator/StepDetails.jsx): `frontend/src/components/GridSimulator/StepDetails.jsx`
- [Agent reasoning](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/GridSimulator/AgentThinking.jsx): `frontend/src/components/GridSimulator/AgentThinking.jsx`
- [Collision and loop alerts](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/GridSimulator/LoopAlert.jsx): `frontend/src/components/GridSimulator/LoopAlert.jsx`
- [Q-table explorer](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/pages/QTableExplorerPage.jsx): `frontend/src/pages/QTableExplorerPage.jsx`
- [Q-table matrix](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/QTableExplorer/QTableGrid.jsx): `frontend/src/components/QTableExplorer/QTableGrid.jsx`
- [Bellman inspector](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/QTableExplorer/BellmanCalculator.jsx): `frontend/src/components/QTableExplorer/BellmanCalculator.jsx`
- [Action comparison](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/QTableExplorer/StateActionChart.jsx): `frontend/src/components/QTableExplorer/StateActionChart.jsx`
- [Learning analytics](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/pages/LearningAnalyticsPage.jsx): `frontend/src/pages/LearningAnalyticsPage.jsx`
- [Training charts](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/Analytics/TrainingCharts.jsx): `frontend/src/components/Analytics/TrainingCharts.jsx`
- [Metric cards](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/Analytics/MetricCards.jsx): `frontend/src/components/Analytics/MetricCards.jsx`
- [Learning settings](https://github.com/haricharan-25/q-learning/blob/main/frontend/src/components/SettingsModal.jsx): `frontend/src/components/SettingsModal.jsx`

### Simulation service

- [HTTP and WebSocket API](https://github.com/haricharan-25/q-learning/blob/main/backend/main.py): `backend/main.py`
- [Simulation and training engine](https://github.com/haricharan-25/q-learning/blob/main/backend/q_engine.py): `backend/q_engine.py`

### Learning runtime

- [Grid environment](https://github.com/haricharan-25/q-learning/blob/main/Q-learning/environment/grid_env.py): `Q-learning/environment/grid_env.py`
- [Coordinate Q-agent](https://github.com/haricharan-25/q-learning/blob/main/Q-learning/agent/q_learning.py): `Q-learning/agent/q_learning.py`
- [Local-state Q-agent](https://github.com/haricharan-25/q-learning/blob/main/Q-learning/agent/local_q_learning.py): `Q-learning/agent/local_q_learning.py`
- [Obstacle generator](https://github.com/haricharan-25/q-learning/blob/main/Q-learning/utils/grid_generator.py): `Q-learning/utils/grid_generator.py`

### Offline experiments

- [Local-state trainer](https://github.com/haricharan-25/q-learning/blob/main/Q-learning/training/train_local.py): `Q-learning/training/train_local.py`
- [Coordinate trainer](https://github.com/haricharan-25/q-learning/blob/main/Q-learning/training/train.py): `Q-learning/training/train.py`
- [Generalization demo](https://github.com/haricharan-25/q-learning/blob/main/Q-learning/main.py): `Q-learning/main.py`

### Other components

- **Learner**

## Connections

- Learner → Dashboard shell: uses (evidence: `README.md`)
- Dashboard shell → Shared state and API: provides state (evidence: `frontend/src/App.jsx`)
- Dashboard shell → Grid simulator: renders (evidence: `frontend/src/App.jsx`)
- Dashboard shell → Q-table explorer: renders (evidence: `frontend/src/App.jsx`)
- Dashboard shell → Learning analytics: renders (evidence: `frontend/src/App.jsx`)
- Dashboard shell → Learning settings: renders (evidence: `frontend/src/App.jsx`)
- Grid simulator → Grid editor: renders (evidence: `frontend/src/pages/GridSimulatorPage.jsx`)
- Grid simulator → Simulation controls: renders (evidence: `frontend/src/pages/GridSimulatorPage.jsx`)
- Grid simulator → Step telemetry: renders (evidence: `frontend/src/pages/GridSimulatorPage.jsx`)
- Grid simulator → Agent reasoning: renders (evidence: `frontend/src/pages/GridSimulatorPage.jsx`)
- Grid simulator → Collision and loop alerts: renders (evidence: `frontend/src/pages/GridSimulatorPage.jsx`)
- Q-table explorer → Q-table matrix: renders (evidence: `frontend/src/pages/QTableExplorerPage.jsx`)
- Q-table explorer → Bellman inspector: renders (evidence: `frontend/src/pages/QTableExplorerPage.jsx`)
- Q-table explorer → Action comparison: renders (evidence: `frontend/src/pages/QTableExplorerPage.jsx`)
- Learning analytics → Training charts: renders (evidence: `frontend/src/pages/LearningAnalyticsPage.jsx`)
- Learning analytics → Metric cards: renders (evidence: `frontend/src/pages/LearningAnalyticsPage.jsx`)
- Shared state and API → HTTP and WebSocket API: requests API (evidence: `frontend/src/context/QLearningContext.jsx`)
- HTTP and WebSocket API → Simulation and training engine: delegates to (evidence: `backend/main.py`)
- Simulation and training engine → Grid environment: steps grid (evidence: `backend/q_engine.py`)
- Simulation and training engine → Coordinate Q-agent: uses agent (evidence: `backend/q_engine.py`)
- Simulation and training engine → Local-state Q-agent: uses agent (evidence: `backend/q_engine.py`)
- Simulation and training engine → Obstacle generator: generates grids (evidence: `backend/q_engine.py`)
- Coordinate trainer → Grid environment: trains against (evidence: `Q-learning/training/train.py`)
- Coordinate trainer → Coordinate Q-agent: updates agent (evidence: `Q-learning/training/train.py`)
- Coordinate trainer → Obstacle generator: creates layouts (evidence: `Q-learning/training/train.py`)
- Local-state trainer → Grid environment: trains against (evidence: `Q-learning/training/train_local.py`)
- Local-state trainer → Local-state Q-agent: updates agent (evidence: `Q-learning/training/train_local.py`)
- Local-state trainer → Obstacle generator: creates layouts (evidence: `Q-learning/training/train_local.py`)
- Generalization demo → Local-state trainer: trains policy (evidence: `Q-learning/main.py`)
- Generalization demo → Grid environment: tests on grids (evidence: `Q-learning/main.py`)
- Generalization demo → Obstacle generator: generates layouts (evidence: `Q-learning/main.py`)