# bharathsr16/ai_smartclassroom architecture

> Architecture diagram of the GitHub repository bharathsr16/ai_smartclassroom, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bharathsr16/ai_smartclassroom
- Repository: https://github.com/bharathsr16/ai_smartclassroom

## Overview

##### Project type and purpose
- This is a standalone, multimodal AI assistant (not a full‐stack web app) that runs locally, orchestrated by a central Python script. Its purpose is to interact with students via text, voice, and vision, answer questions, transcribe lectures, summarize, gauge engagement, generate visual aids, and log feedback.

##### File structure
Top-level grouping:
- Main orchestrator: `main_assistant.py`, `app_ui.py`, `summarize_lecture_cli.py`
- Text modules: `text_qa_module.py`, `text_summarizer.py`, `text_summarizer_ov.py`, `convert_summarizer_to_ov.py`
- Voice modules: `voice_processing_module.py`, `lecture_transcriber.py`
- Visual modules: `visual_module.py`, `visual_aid_generator.py`
- Engagement & feedback: `engagement_module.py`, `feedback_reporter.py`, `feedback_log.csv`
- Model conversion & IR storage: `convert_bert.py`, `bert_ir/`, `ov_model_placeholder/`
- Utilities & data: `feedback_log.csv`, placeholder model files, XML IR files

##### Extract key insights from README
- Central orchestrator integrates all modules but currently fails in sandbox.
- Each module depends on specific external libraries (PyTorch/Transformers, OpenCV, SpeechRecognition/PyAudio, pyttsx3, fer/TensorFlow, Matplotlib).
- OpenVINO conversion path exists for optimization.
- Hardware dependencies: mic, speaker, webcam, TTS engine, Haar cascade XML.

1. User Interface
- CLI (`summarize_lecture_cli.py`)
- Simple GUI (`app_ui.py`)
2. Orchestrator
- `main_assistant.py`
- Text QA & Summarization
- Voice (STT & TTS)
- Visual Processing & Captioning
- Engagement Monitoring
- Visual Aid Generation
4. Model Layer
- Hugging Face Transformers Models
- OpenVINO IR Models (`bert_ir/`, `ov_model_placeholder/`)
5. Utilities & Data Stores
- Feedback Reporter & CSV Log
6. External Dependencies & Hardware
- Microphone, Speaker, Webcam
- OS-level engines (eSpeak-NG, PortAudio, Tesseract OCR)
- Haarcascade XML

- User interacts with UI → UI calls Orchestrator
- Orchestrator routes requests to the appropriate module → module loads model (HF or OpenVINO) → processes input → returns result to Orchestrator → Orchestrator outputs via UI
- Engagement module continuously pulls camera feed → analyzes → streams engagement metrics back to Orchestrator
- Feedback Reporter listens for specific events/responses → appends to CSV store

c. Architectural patterns & design principles
- Modular “plugin” pattern: each modality is a self-contained module with a well-defined interface
- Mediator pattern: `main_assistant.py` acts as the central coordinator
- Separation of concerns: UI, orchestration, AI logic, model storage, and reporting are distinct layers
- Extensibility via conversion scripts for OpenVINO optimization

d. Key technologies & frameworks
- Python 3.9+, Hugging Face Transformers + PyTorch
- OpenVINO (IR model directory)
- OpenCV for vision & face detection
- speech_recognition + PyAudio, pyttsx3 for voice
- fer (TensorFlow) for emotion recognition
- Matplotlib for chart generation
- Tesseract OCR (optional advanced visual queries)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interface Layer
    subgraph "User Interface"
        CLI["CLI Interface"]:::python
        GUI["GUI Interface"]:::python
    end

    %% Orchestrator Layer
    subgraph "Orchestrator Layer"
        ORCH["main_assistant.py"]:::python
    end

    %% Modules Layer
    subgraph "Modules Layer"
        subgraph "Text Modules"
            QA["Text QA Module"]:::python
            SUM["Text Summarization Module"]:::python
            SUMOV["Text Summarization (OpenVINO) Module"]:::python
        end
        subgraph "Voice Modules"
            VP["Voice Processing Module"]:::python
            LT["Lecture Transcriber"]:::python
        end
        subgraph "Visual Modules"
            VM["Visual Processing & Captioning Module"]:::python
            VAG["Visual Aid Generation Module"]:::python
        end
        ENG["Engagement Monitoring Module"]:::python
        FR["Feedback Reporter"]:::python
    end

    %% Model Layer
    subgraph "Model Layer"
        HF["Hugging Face Models"]:::model
        IR["OpenVINO IR Models"]:::model
        OVP["Placeholder OpenVINO Models"]:::model
    end

    %% Utilities & Data Stores
    subgraph "Utilities & Data Stores"
        FL["feedback_log.csv"]:::data
    end

    %% External & Hardware
    subgraph "External & Hardware"
        MIC["Microphone"]:::hardware
        SPK["Speaker"]:::hardware
        CAM["Webcam"]:::hardware
        TTS["TTS Engine"]:::external
        OCR["Tesseract OCR"]:::external
        XML["Haarcascade XML"]:::external
    end

    %% Conversion Utilities
    CS["Summarizer Conversion Utility"]:::conversion
    CB["BERT Conversion Script"]:::conversion

    %% Flows
    CLI -->|interacts| ORCH
    GUI -->|interacts| ORCH

    ORCH -->|routes to| QA
    ORCH -->|routes to| SUM
    ORCH -->|routes to| SUMOV
    ORCH -->|routes to| VP
    ORCH -->|routes to| LT
    ORCH -->|routes to| VM
    ORCH -->|routes to| VAG
    ORCH -->|receives metrics from| ENG
    ORCH -->|triggers reports on| FR

    QA -->|loads model| HF
    SUM -->|loads model| HF
    SUMOV -->|loads model| IR
    VP -->|uses STT/TTS| LT
    LT -->|outputs text| ORCH
    VM -->|processes feed from| CAM
    VM -->|uses OCR| OCR
    VM -->|uses Haarcascade| XML
    VAG -->|generates visuals| ORCH

    ENG -->|streams video from| CAM
    ENG -->|returns engagement| ORCH

    FR -->|writes to| FL

    MIC -->|audio stream| LT
    VP -->|audio out| TTS
    TTS -->|to| SPK

    HF -->|optional conversion| CS
    CS -->|produces| SUMOV

    HF -->|optional conversion| CB
    CB -->|produces| IR
    IR -->|stored with| OVP

    %% Click Events
    click CLI "https://github.com/bharathsr16/ai_smartclassroom/blob/master/summarize_lecture_cli.py"
    click GUI "https://github.com/bharathsr16/ai_smartclassroom/blob/master/app_ui.py"
    click ORCH "https://github.com/bharathsr16/ai_smartclassroom/blob/master/main_assistant.py"
    click QA "https://github.com/bharathsr16/ai_smartclassroom/blob/master/text_qa_module.py"
    click SUM "https://github.com/bharathsr16/ai_smartclassroom/blob/master/text_summarizer.py"
    click SUMOV "https://github.com/bharathsr16/ai_smartclassroom/blob/master/text_summarizer_ov.py"
    click CS "https://github.com/bharathsr16/ai_smartclassroom/blob/master/convert_summarizer_to_ov.py"
    click VP "https://github.com/bharathsr16/ai_smartclassroom/blob/master/voice_processing_module.py"
    click LT "https://github.com/bharathsr16/ai_smartclassroom/blob/master/lecture_transcriber.py"
    click VM "https://github.com/bharathsr16/ai_smartclassroom/blob/master/visual_module.py"
    click VAG "https://github.com/bharathsr16/ai_smartclassroom/blob/master/visual_aid_generator.py"
    click ENG "https://github.com/bharathsr16/ai_smartclassroom/blob/master/engagement_module.py"
    click FR "https://github.com/bharathsr16/ai_smartclassroom/blob/master/feedback_reporter.py"
    click FL "https://github.com/bharathsr16/ai_smartclassroom/blob/master/feedback_log.csv"
    click CB "https://github.com/bharathsr16/ai_smartclassroom/blob/master/convert_bert.py"
    click IR "https://github.com/bharathsr16/ai_smartclassroom/tree/master/bert_ir/"
    click OVP "https://github.com/bharathsr16/ai_smartclassroom/tree/master/ov_model_placeholder/"

    %% Styles
    classDef python fill:#CEE5F2,stroke:#0366D6,stroke-width:1px;
    classDef model fill:#FFE5B4,stroke:#E69500,stroke-width:1px;
    classDef hardware fill:#D5E8D4,stroke:#82B366,stroke-width:1px;
    classDef external fill:#D5E8D4,stroke:#82B366,stroke-width:1px,stroke-dasharray: 5 5;
    classDef data fill:#F2F2F2,stroke:#999999,stroke-width:1px;
    classDef conversion fill:#FFE5B4,stroke:#E69500,stroke-width:2px,stroke-dasharray: 5 5;
```
