# bhanuprakashvangala/visionai architecture

> Architecture diagram of the GitHub repository bhanuprakashvangala/visionai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bhanuprakashvangala/visionai
- Repository: https://github.com/bhanuprakashvangala/visionai

## Overview

##### Project Type and Purpose
- VisionAI is an AI-powered assistive system (a monolithic Python application) that processes image or voice inputs to provide real-time hazard alerts and natural speech responses for visually impaired users.
- It integrates computer vision, NLP, speech recognition, text-to-speech, and external APIs (Google Maps, Google News) into a single pipeline.

##### File Structure
- Top-level:
- README.md – overall project description
- src/ – application source code
- src/:
- README.md – module-level notes
- VisionAI.py – main entry point, orchestrates end-to-end pipeline
- visualizations.py – functions for audio analysis (SNR, spectrograms)
- No separate frontend/backend folders: all logic lives in Python modules under src/.

##### README Insights
- Key components described: scene understanding (BLIP-2, LLaMA-2), real-time hazard detection (Maps & News APIs), speech I/O (speech recognition & TTS), audio clarity evaluation.
- Technologies: PyTorch, OpenAI Transformers, OpenCV, skimage, Matplotlib/NumPy/SciPy, Google Speech API.

##### Main Components
a. User Interfaces
- Image/Video Input Module
- Voice Command Input (Speech Recognition)
b. Processing & AI Core
- Scene Understanding: BLIP-2 → LLaMA-2
- Hazard Detection: Google Maps API + Google News API
- Question Answering / Contextual Reasoning (LLaMA-2)
c. Output Modules
- Text-to-Speech (TTS) Engine
- Audio Clarity Analyzer (visualizations.py: SNR, spectrogram)
d. Data & Storage (optional)
- Temporary caches for API results or model outputs
e. External Services
- Google Maps Service
- Google News Service
- Google Speech Recognition Service

##### Relationships and Data Flow
1. User → [Image Input] → Scene Understanding → Raw Description
2. Raw Description → LLaMA-2 → Enhanced Description
3. User Location → Google Maps/News → Hazard Data
4. Enhanced Description + Hazard Data → LLaMA-2 QA → Text Response
5. Text Response → TTS → Audio Output
6. Audio Output → Audio Analyzer → SNR & Spectrogram Metrics → Logging/Display
- Asynchronous calls to external APIs and model inference steps.

##### Architectural Patterns & Design Principles
- Pipeline pattern: a linear sequence of processing stages
- Modular design: separate modules for vision, language, speech, API integration, and analysis
- Single responsibility: each component (module) has one clear function

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User and Inputs
    subgraph "User Interaction"
        UI_User["User"] 
        UI_Image["Image/Video Input Module"]:::io
        UI_Voice["Voice Command Input<br/>(Speech Recognition)"]:::io
    end

    %% Main Orchestrator
    MainOrch["Main Orchestrator<br/>(VisionAI.py)"]:::ai

    %% AI Core
    subgraph "AI Core"
        SceneUnderstanding["Scene Understanding<br/>(BLIP-2 + PyTorch)"]:::ai
        QA["Question Answering / Contextual Reasoning<br/>(LLaMA-2)"]:::ai
    end

    %% Hazard Detection Module
    HazardDetection["Hazard Detection<br/>(Maps & News APIs)"]:::ai

    %% Output Modules
    TTS["Text-to-Speech Engine<br/>(Tacotron2)"]:::ai
    AudioAnalyzer["Audio Clarity Analyzer<br/>(SNR & Spectrogram)"]:::analysis

    %% Data Stores / Caches
    Cache["API/Model Cache"]:::cache
    RawDesc["Raw Description"]:::data
    EnhancedDesc["Enhanced Description"]:::data
    MapData["Map Data"]:::data
    NewsData["News Data"]:::data
    HazardData["Hazard Data"]:::data
    SpeechText["Speech Text"]:::data
    TextResponse["Text Response"]:::data
    AudioOutput["Audio Output"]:::data
    Metrics["SNR & Spectrogram Metrics"]:::data
    Logging["Logging/Display"]:::data

    %% External Services
    subgraph "External Services"
        GoogleMaps["Google Maps API"]:::external
        GoogleNews["Google News API"]:::external
        GoogleSpeech["Google Speech Recognition API"]:::external
    end

    %% Top-level Documentation
    README["Project Documentation<br/>(README.md)"]:::doc
    ModuleNotes["Module Notes<br/>(src/README.md)"]:::doc

    %% Connections: Orchestration
    UI_User --> UI_Image
    UI_User --> UI_Voice
    MainOrch --> UI_Image
    MainOrch --> UI_Voice
    MainOrch --> SceneUnderstanding
    MainOrch --> HazardDetection
    MainOrch --> QA
    MainOrch --> TTS
    MainOrch --> AudioAnalyzer

    %% Image pipeline
    UI_Image --> SceneUnderstanding
    SceneUnderstanding --> RawDesc
    RawDesc --> QA
    RawDesc --> Cache

    %% Voice pipeline
    UI_Voice -->|ASR request| GoogleSpeech
    GoogleSpeech --> SpeechText
    SpeechText --> QA
    SpeechText --> Cache

    %% Hazard data flow
    UI_User -->|Location data| GoogleMaps
    UI_User -->|Location data| GoogleNews
    GoogleMaps --> MapData
    GoogleNews --> NewsData
    MapData --> HazardDetection
    NewsData --> HazardDetection
    HazardDetection --> HazardData
    HazardData --> QA
    HazardData --> Cache

    %% QA to Response
    QA --> EnhancedDesc
    EnhancedDesc --> TextResponse
    TextResponse --> TTS
    TextResponse --> Cache

    %% Audio output and analysis
    TTS --> AudioOutput
    AudioOutput --> AudioAnalyzer
    AudioAnalyzer --> Metrics
    Metrics --> Logging
    Metrics --> Cache

    %% Documentation links
    README
    ModuleNotes

    %% Click Events
    click MainOrch "https://github.com/bhanuprakashvangala/visionai/blob/main/src/VisionAI.py"
    click SceneUnderstanding "https://github.com/bhanuprakashvangala/visionai/blob/main/src/VisionAI.py"
    click QA "https://github.com/bhanuprakashvangala/visionai/blob/main/src/VisionAI.py"
    click HazardDetection "https://github.com/bhanuprakashvangala/visionai/blob/main/src/VisionAI.py"
    click TTS "https://github.com/bhanuprakashvangala/visionai/blob/main/src/VisionAI.py"
    click UI_Voice "https://github.com/bhanuprakashvangala/visionai/blob/main/src/VisionAI.py"
    click AudioAnalyzer "https://github.com/bhanuprakashvangala/visionai/blob/main/src/visualizations.py"
    click README "https://github.com/bhanuprakashvangala/visionai/blob/main/README.md"
    click ModuleNotes "https://github.com/bhanuprakashvangala/visionai/blob/main/src/README.md"

    %% Styles
    classDef io fill:#ffa500,stroke:#333,stroke-width:1px;
    classDef ai fill:#98fb98,stroke:#333,stroke-width:1px;
    classDef external fill:#87ceeb,stroke:#333,stroke-width:1px;
    classDef data fill:#dda0dd,stroke:#333,stroke-width:1px;
    classDef cache fill:#cccccc,stroke:#333,stroke-width:1px;
    classDef analysis fill:#ffcccb,stroke:#333,stroke-width:1px;
    classDef doc fill:#f0e68c,stroke:#333,stroke-width:1px;
```
