How ayin1412/forzahorizon6-visionai works
This repository is a camera-only closed-loop Forza Horizon 6 driving agent, with a training path for producing geometry-mediated driving models. At runtime, the driver combines game imagery with telemetry, invokes the network, and sends control through a virtual Xbox gamepad; optional trajectory overlay and state broadcasting support inspection. The README also describes line, waypoint, speed-profile, slip, and control outputs. Training derives targets from recordings and optimizes the same network. Connections for unsampled capture, actuation, and dashboard wiring are omitted where excerpts do not establish the implementation details.
Main entry points: drive.py Realtime driver · train.py Training pipeline · model.py DrivingNet · model_30.pt Released weights
Components 17
Driving runtime
- Game telemetry
scripts/drive.py - Camera frames
scripts/drive.py - Realtime driver
scripts/drive.py - Virtual gamepad
scripts/drive.py
Geometry model
- DrivingNet
scripts/model.py - Visual backbone
scripts/model.py - Geometry heads
scripts/model.py - Control heads
scripts/model.py - Released weights
data/models/model_30.pt - Model metadata
data/models/meta.json
Training
- Driving recordings
- Training pipeline
scripts/train.py - Derived dataset
scripts/train.py
Inspection
- Trajectory overlay
scripts/drive.py - State broadcaster
scripts/drive.py - Dashboard UI
scripts/ui/dashboard.html
Other components
- Forza game
Connections 17
- Forza game → Camera frames: provides view
README.md - Forza game → Game telemetry: provides telemetry
README.md - Camera frames → Realtime driver: feeds images
README.md - Game telemetry → Realtime driver: provides state
README.md - Realtime driver → DrivingNet: invokes
scripts/drive.py - DrivingNet → Visual backbone: extracts features
scripts/model.py - Visual backbone → Geometry heads: predicts geometry
scripts/model.py - Geometry heads → Control heads: feeds detached inputs
scripts/model.py - Realtime driver → Virtual gamepad: sends controls
scripts/drive.py - Realtime driver → Released weights: loads checkpoint
- Realtime driver → Model metadata: reads configuration
scripts/drive.py - Driving recordings → Training pipeline: supplies examples
scripts/train.py - Training pipeline → Derived dataset: derives targets
scripts/train.py - Training pipeline → DrivingNet: trains
scripts/train.py - Realtime driver → Trajectory overlay: updates waypoints
scripts/drive.py - Realtime driver → State broadcaster: publishes state
scripts/drive.py - State broadcaster → Dashboard UI: serves dashboard data