How neelpatel31/self-driving-car works
Project type and purpose
• This is a standalone desktop simulation tool that demonstrates Q-learning in a self-driving-car environment with interactive sand drawing.
• It is not a client-server web app but an open-source desktop GUI application built with Python, Kivy, PyTorch, NumPy, and Matplotlib.
File structure
• Top level:
• ai.py – encapsulates the Q-learning agent logic (state representation, Q-table/policy, update rules, saving/loading)
• .py – main application entry point; sets up the Kivy App, simulation loop, environment grid, car entity
• car.kv – Kivy layout file defining the UI widgets (buttons, canvas for sand drawing, car sprite)
• requirements.txt – lists dependencies
• .devcontainer/ – VS Code dev-container configuration (development environment setup)
• There are no separate “frontend” or “backend” folders; instead, the code follows a Model-View-Controller pattern:
• Model = Q-learning agent (ai.py) + environment state
• View = Kivy UI (map.py + car.kv)
• Controller = event handlers in map.py that mediate between UI and agent
README for additional insights
• The README describes:
• Q-learning with saving/loading of Q-values
• Interactive drawing of sand (slow zones)
• Real-time visualization
a. Main components
1. User Interface (Kivy)
– sand-drawing canvas
– control buttons (save, load)
– visualization area (car on map, matplotlib plots if used)
2. Simulation Controller (map.py)
– simulation loop (tick: sense state, choose action, apply action, update UI)
– event listeners for drawing and UI buttons
3. Q-Learning Agent (ai.py)
– state encoder (position, sand grid)
– action selector (ϵ-greedy)
– Q-table or neural network (PyTorch)
– update rule (reward, next state)
– persistence (save/load Q-values)
4. Environment Model
– grid representation of terrain (sand vs. normal)
– reward function (e.g., penalty on sand, goal reward)
5. Data Storage
– filesystem files for Q-values serialization
6. External Libraries
– Kivy (GUI)
– NumPy (arrays)
– PyTorch (if function approximator used)
– Matplotlib (optional plots)
b. Relationships and interactions
• User → UI: drawing sand, clicking save/load
• UI → Controller: events produce calls to the simulation loop or persistence functions
• Controller → Environment Model: queries current terrain under car; updates sand grid
• Controller → Agent: passes state, obtains action; sends reward feedback
• Agent → Controller: returns next action; updates internal Q-table
• Controller → UI: updates car position and visualization each tick
• Controller ↔ Data Storage: save() and load() calls serialize/deserialize Q-values
c. Architectural patterns and principles
• MVC-style separation: Model (agent + environment), View (Kivy UI), Controller (map.py)
• Event-driven UI: Kivy dispatches touch and button events
• Loop-based simulation tick
• Persistence layer abstracted behind agent methods
d. Key technologies and frameworks
• Python 3
• Kivy + .kv file for declarative UI
• NumPy for grid and vector math
• PyTorch (optional) for Q-function approximation
• Matplotlib for any real-time plotting
• Filesystem for Q-value storage
Detailed element list
• From UI events → Controller
• From Controller → Agent (state in) and back (action out)
• From Controller → Environment (load terrain, update sand)
• From Controller → UI (render updates)
• From Controller → Storage (save/load)
• Optional: annotate reward signals and state transitions.