How mmusil25/raynos-ai works
Project type and purpose
• This is a standalone, on-device AI pipeline with a local web UI.
• It ingests audio (mic or file), runs real-time transcription (Whisper), then structured intent/entity extraction (Gemma 3-N optimized via Unsloth), and displays JSON output—all locally.
File structure
• Top-level “src/” holds the two main Python modules:
• gradio_app_integrated.py → defines the Gradio web interface and orchestrates the pipeline
• gemma_3n_json_extractor.py → encapsulates the ExtractionManager for Gemma 3-N
• “unsloth_compiled_cache/” (both under src/ and root) contains the drop-in optimized model implementations generated by Unsloth
• “example_audio/” offers sample inputs; “docs/architecture.png” shows a reference sketch
README for insights
• Confirms the three-stage pipeline:
Audio Source → Whisper transcription → Gemma 3-N JSON extraction
• Highlights dependencies and environment (Python 3.10+, torch, whisper, unsloth, gradio)
• Clear, concise labels for each component
• UI vs. compute vs. storage
• static assets vs. dynamically generated artifacts
• Annotations for key dependencies (Whisper, Gemma, Unsloth, Gradio)