How haricharan-25/q-learning works
This is an educational Q-learning grid-world visualizer: users configure a grid, run or train an agent, then inspect movement, Bellman updates, the Q-table, and learning metrics. The browser’s shared context drives the simulator and training through backend APIs; the backend engine owns environment and agent behavior. A separate Python workflow trains local-observation agents on generated grids. Edges follow shown calls, imports, or documented API behavior; several UI/backend details and unsampled capabilities are represented at subsystem level because their implementation was not excerpted.
Main entry points: App.jsx Dashboard shell · GridSimulatorPage.jsx Grid simulator · q_engine.py Simulation and training engine · QTableExplorerPage.jsx Q-table explorer
Components 26
Interactive dashboard
- Dashboard shell
frontend/src/App.jsx - Shared state and API
frontend/src/context/QLearningContext.jsx - Grid simulator
frontend/src/pages/GridSimulatorPage.jsx - Grid editor
frontend/src/components/GridSimulator/GridBoard.jsx - Simulation controls
frontend/src/components/GridSimulator/GridControls.jsx - Step telemetry
frontend/src/components/GridSimulator/StepDetails.jsx - Agent reasoning
frontend/src/components/GridSimulator/AgentThinking.jsx - Collision and loop alerts
frontend/src/components/GridSimulator/LoopAlert.jsx - Q-table explorer
frontend/src/pages/QTableExplorerPage.jsx - Q-table matrix
frontend/src/components/QTableExplorer/QTableGrid.jsx - Bellman inspector
frontend/src/components/QTableExplorer/BellmanCalculator.jsx - Action comparison
frontend/src/components/QTableExplorer/StateActionChart.jsx - Learning analytics
frontend/src/pages/LearningAnalyticsPage.jsx - Training charts
frontend/src/components/Analytics/TrainingCharts.jsx - Metric cards
frontend/src/components/Analytics/MetricCards.jsx - Learning settings
frontend/src/components/SettingsModal.jsx
Simulation service
- HTTP and WebSocket API
backend/main.py - Simulation and training engine
backend/q_engine.py
Learning runtime
- Grid environment
Q-learning/environment/grid_env.py - Coordinate Q-agent
Q-learning/agent/q_learning.py - Local-state Q-agent
Q-learning/agent/local_q_learning.py - Obstacle generator
Q-learning/utils/grid_generator.py
Offline experiments
- Local-state trainer
Q-learning/training/train_local.py - Coordinate trainer
Q-learning/training/train.py - Generalization demo
Q-learning/main.py
Other components
- Learner
Connections 31
- Learner → Dashboard shell: uses
README.md - Dashboard shell → Shared state and API: provides state
frontend/src/App.jsx - Dashboard shell → Grid simulator: renders
frontend/src/App.jsx - Dashboard shell → Q-table explorer: renders
frontend/src/App.jsx - Dashboard shell → Learning analytics: renders
frontend/src/App.jsx - Dashboard shell → Learning settings: renders
frontend/src/App.jsx - Grid simulator → Grid editor: renders
frontend/src/pages/GridSimulatorPage.jsx - Grid simulator → Simulation controls: renders
frontend/src/pages/GridSimulatorPage.jsx - Grid simulator → Step telemetry: renders
frontend/src/pages/GridSimulatorPage.jsx - Grid simulator → Agent reasoning: renders
frontend/src/pages/GridSimulatorPage.jsx - Grid simulator → Collision and loop alerts: renders
frontend/src/pages/GridSimulatorPage.jsx - Q-table explorer → Q-table matrix: renders
frontend/src/pages/QTableExplorerPage.jsx - Q-table explorer → Bellman inspector: renders
frontend/src/pages/QTableExplorerPage.jsx - Q-table explorer → Action comparison: renders
frontend/src/pages/QTableExplorerPage.jsx - Learning analytics → Training charts: renders
frontend/src/pages/LearningAnalyticsPage.jsx - Learning analytics → Metric cards: renders
frontend/src/pages/LearningAnalyticsPage.jsx - Shared state and API → HTTP and WebSocket API: requests API
frontend/src/context/QLearningContext.jsx - HTTP and WebSocket API → Simulation and training engine: delegates to
backend/main.py - Simulation and training engine → Grid environment: steps grid
backend/q_engine.py - Simulation and training engine → Coordinate Q-agent: uses agent
backend/q_engine.py - Simulation and training engine → Local-state Q-agent: uses agent
backend/q_engine.py - Simulation and training engine → Obstacle generator: generates grids
backend/q_engine.py - Coordinate trainer → Grid environment: trains against
Q-learning/training/train.py - Coordinate trainer → Coordinate Q-agent: updates agent
Q-learning/training/train.py - Coordinate trainer → Obstacle generator: creates layouts
Q-learning/training/train.py - Local-state trainer → Grid environment: trains against
Q-learning/training/train_local.py - Local-state trainer → Local-state Q-agent: updates agent
Q-learning/training/train_local.py - Local-state trainer → Obstacle generator: creates layouts
Q-learning/training/train_local.py - Generalization demo → Local-state trainer: trains policy
Q-learning/main.py - Generalization demo → Grid environment: tests on grids
Q-learning/main.py - Generalization demo → Obstacle generator: generates layouts
Q-learning/main.py