How mortem1504/ai-guitar-tutor works
AI Guitar Chord Tutor is a local real-time coaching app: the user selects a lesson, starts camera and microphone processing, forms a chord, and strums; a success requires matching visual and acoustic evidence. The vision path detects guitar and fretboard, maps hand shape, and publishes observations; the audio path analyzes strums and chord tones. Shared state feeds timestamped fusion and evidence-based feedback displayed in the dashboard. README documents calibration, diagnostics, and training metrics; some subsystem links rely on README or SOURCE INDEX references because their source was not excerpted.
Main entry points: app.py Streamlit dashboard · video_processor.py Frame processing · state.py Shared observations · evaluator.py Fusion evaluator
Components 24
Tutor experience
- Streamlit dashboard
app.py - Tutor UI components
guitar_tutor/ui/components.py - Measured feedback
guitar_tutor/tutor/feedback.py - Developer diagnostics
guitar_tutor/tutor/diagnostics.py
EYE vision
- Camera capture
guitar_tutor/vision/camera_manager.py - Vision worker
guitar_tutor/vision/vision_worker.py - Frame processing
guitar_tutor/vision/video_processor.py - Guitar detection
guitar_tutor/vision/guitar_detector.py - Fretboard detection
guitar_tutor/vision/fretboard_detector.py - Fretboard geometry
guitar_tutor/vision/fretboard_geometry.py - Fretboard mapping
guitar_tutor/vision/fretboard_mapper.py - Calibration settings
guitar_tutor/vision/fretboard_calibration.py - Hand landmarks
guitar_tutor/vision/mediapipe_adapter.py - Hand geometry
guitar_tutor/vision/hand_geometry.py - Chord shape matching
guitar_tutor/vision/chord_shape.py
EAR audio
- Microphone capture
guitar_tutor/audio/microphone.py - Audio chord analysis
guitar_tutor/audio/chord_detector.py - DSP and onset tools
guitar_tutor/audio
BRAIN fusion
- Fusion evaluator
guitar_tutor/fusion/evaluator.py - Strum synchronization
guitar_tutor/fusion/event_sync.py
Lesson and state
- Shared observations
guitar_tutor/state.py - Chord definitions
guitar_tutor/chord_library.py - Tutor configuration
guitar_tutor/config.py
Other components
- Guitar learner
Connections 34
- Guitar learner → Streamlit dashboard: chooses lesson
README.md - Streamlit dashboard → Vision worker: starts vision
app.py - Streamlit dashboard → Microphone capture: starts audio
app.py - Streamlit dashboard → Fusion evaluator: starts fusion
app.py - Streamlit dashboard → Tutor configuration: sets options
app.py - Streamlit dashboard → Tutor UI components: renders dashboard
README.md - Streamlit dashboard → Measured feedback: shows guidance
README.md - Streamlit dashboard → Developer diagnostics: shows diagnostics
README.md - Camera capture → Vision worker: provides frames
guitar_tutor/vision/vision_worker.py - Vision worker → Frame processing: processes frames
guitar_tutor/vision/vision_worker.py - Vision worker → Shared observations: publishes results
guitar_tutor/vision/vision_worker.py - Frame processing → Guitar detection: detects guitar
guitar_tutor/vision/video_processor.py - Frame processing → Fretboard detection: detects fretboard
guitar_tutor/vision/video_processor.py - Fretboard detection → Fretboard geometry: uses geometry
guitar_tutor/vision/fretboard_detector.py - Frame processing → Hand landmarks: tracks hands
guitar_tutor/vision/video_processor.py - Frame processing → Hand geometry: measures joints
guitar_tutor/vision/video_processor.py - Frame processing → Fretboard mapping: maps fingertips
guitar_tutor/vision/video_processor.py - Fretboard mapping → Calibration settings: uses calibration
guitar_tutor/vision/fretboard_mapper.py - Frame processing → Chord shape matching: classifies shape
guitar_tutor/vision/video_processor.py - Chord shape matching → Chord definitions: matches fingerings
guitar_tutor/vision/chord_shape.py - Chord shape matching → Shared observations: reads target state
guitar_tutor/vision/chord_shape.py - Microphone capture → Audio chord analysis: analyzes audio
guitar_tutor/audio/microphone.py - Microphone capture → Shared observations: publishes audio
guitar_tutor/audio/microphone.py - Audio chord analysis → DSP and onset tools: uses DSP tools
guitar_tutor/audio/chord_detector.py - Audio chord analysis → Chord definitions: compares templates
guitar_tutor/audio/chord_detector.py - Audio chord analysis → Tutor configuration: uses thresholds
guitar_tutor/audio/chord_detector.py - Fusion evaluator → Shared observations: reads observations
guitar_tutor/fusion/evaluator.py - Fusion evaluator → Strum synchronization: synchronizes strums
guitar_tutor/fusion/evaluator.py - Fusion evaluator → Tutor configuration: checks freshness
guitar_tutor/fusion/evaluator.py - Tutor UI components → Shared observations: renders snapshot
guitar_tutor/ui/components.py - Tutor UI components → Chord definitions: shows target tones
guitar_tutor/ui/components.py - Measured feedback → Shared observations: reads measurements
guitar_tutor/tutor/feedback.py - Measured feedback → Chord definitions: checks chord tones
guitar_tutor/tutor/feedback.py - Shared observations → Fusion evaluator: receives evaluation
guitar_tutor/fusion/evaluator.py