# rishabhshah13/real-time-slr architecture

> Architecture diagram of the GitHub repository rishabhshah13/real-time-slr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rishabhshah13/real-time-slr
- Repository: https://github.com/rishabhshah13/real-time-slr

## Overview

##### Main Components

A. Input Layer:
- Webcam Interface
- Video Input Processing (MediaPipe)
- Landmark Extraction System

B. Processing Layer:
- Two Main Recognition Modules:
1. Fingerspelling Recognition System
- Letters Model
- Numbers Model
2. Gloss Recognition System

C. Application Interfaces:
- OpenCV Local Interface
- Streamlit Web Interface

D. Model Management:
- Model Storage
- Configuration Management

**[Input Sources] → [Preprocessing] → [Recognition Systems] → [User Interfaces]**

**[Webcam/Video Input] → [MediaPipe Processing] → [Landmark Extraction]**
- ↓
- [Feature Processing]

- ┌─ [Fingerspelling Models] ─┐
**[Processed Features]─┤                          ├─→ [Prediction Output]**
- [Gloss Models] ───────┘

##### Key Relationships to Highlight

- Data flow from video input through landmark extraction to feature processing
- Parallel processing paths for fingerspelling and gloss recognition
- Demonstrate how the configuration system interfaces with all components
- Two separate user interface paths (OpenCV and Streamlit)

##### Technical Details

- Use of MediaPipe for hand landmark extraction
- 1D CNN and Transformer architecture in the model components
- Configuration management system (config.yaml)
- Inference pipeline flow

##### Implementation-Specific Elements

- Scripts/inference directory components as the core processing unit
- Models directory as the model storage system
- Relationship between st_app.py and cv2main.py as alternative interfaces
- Utility functions layer that supports all components

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Input
        WC[Webcam Interface]:::input
        VI[Video Input]:::input
    end

    subgraph Preprocessing
        MP[MediaPipe Processing]:::processing
        LE["Landmark Extraction"]:::processing
        FP[Feature Processing]:::processing
    end

    subgraph "Recognition Systems"
        subgraph "Fingerspelling Recognition"
            FR["Fingerspelling Recognition"]:::model
            LM["Letters Model"]:::model
            NM["Numbers Model"]:::model
        end

        subgraph "Gloss Recognition"
            GR["Gloss Recognition"]:::model
            BB["Model Backbone"]:::model
            GU["Gloss Utils"]:::util
        end
    end

    subgraph "User Interfaces"
        CV["OpenCV Interface"]:::interface
        ST["Streamlit Interface"]:::interface
    end

    subgraph "Configuration & Storage"
        CF["Config Management"]:::config
        MS["Model Storage"]:::storage
        PM["Prediction Mapping"]:::util
    end

    WC --> MP
    VI --> MP
    MP --> LE
    LE --> FP
    FP --> FR
    FP --> GR
    
    FR --> LM
    FR --> NM
    LM --> CV
    NM --> CV
    
    GR --> BB
    BB --> GU
    GU --> CV
    
    LM --> ST
    NM --> ST
    GU --> ST
    
    CF --> FR
    CF --> GR
    CF --> MP
    
    MS --> FR
    MS --> GR
    
    PM --> FR
    PM --> GR

    %% Click Events
    click LE "https://github.com/rishabhshah13/Real-Time-SLR/blob/main/scripts/gloss/landmarks_extraction.py"
    click FR "https://github.com/rishabhshah13/Real-Time-SLR/blob/main/scripts/inference/fingerspellinginference.py"
    click GR "https://github.com/rishabhshah13/Real-Time-SLR/blob/main/scripts/inference/glossinference.py"
    click CV "https://github.com/rishabhshah13/Real-Time-SLR/blob/main/cv2main.py"
    click ST "https://github.com/rishabhshah13/Real-Time-SLR/blob/main/st_app.py"
    click MS "https://github.com/rishabhshah13/Real-Time-SLR/tree/main/models"
    click CF "https://github.com/rishabhshah13/Real-Time-SLR/blob/main/config/config.yaml"
    click PM "https://github.com/rishabhshah13/Real-Time-SLR/blob/main/data/sign_to_prediction_index_map.json"
    click GU "https://github.com/rishabhshah13/Real-Time-SLR/blob/main/scripts/gloss/gloss_utils.py"
    click BB "https://github.com/rishabhshah13/Real-Time-SLR/blob/main/scripts/gloss/backbone.py"

    %% Styling
    classDef input fill:#3498db,stroke:#2980b9,stroke-width:2px
    classDef processing fill:#2ecc71,stroke:#27ae60,stroke-width:2px
    classDef model fill:#e67e22,stroke:#d35400,stroke-width:2px
    classDef interface fill:#9b59b6,stroke:#8e44ad,stroke-width:2px
    classDef config fill:#95a5a6,stroke:#7f8c8d,stroke-width:2px
    classDef storage fill:#34495e,stroke:#2c3e50,stroke-width:2px
    classDef util fill:#f1c40f,stroke:#f39c12,stroke-width:2px
```
