# agarwaldevesh374/gsoc-25-humanai-issr2-screening-test--nlp-tool architecture

> Architecture diagram of the GitHub repository agarwaldevesh374/gsoc-25-humanai-issr2-screening-test--nlp-tool, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/agarwaldevesh374/gsoc-25-humanai-issr2-screening-test--nlp-tool
- Repository: https://github.com/agarwaldevesh374/gsoc-25-humanai-issr2-screening-test--nlp-tool

## Overview

##### Project Overview and Purpose
- This is a Python-based machine learning and NLP project aimed at communication analysis during driving simulations.
- It processes video files to extract audio, cleans the audio, transcribes it, performs speaker diarization and sentiment analysis, and then visualizes the results.
- Advanced techniques (e.g., speaker embedding via pretrained models) along with dynamic reporting and visualization are key features.

##### Main Components
a. Data Ingestion:
- Input Videos: Represented by the "files_audio_video/input_videos" directory where users add the raw video files.
b. Audio/Video Processing Pipeline:
- Audio Extraction: Use the "audio_extraction.py" script to extract audio from video.
- Noise Removal: Display "audio_noiseremoval.py" for cleaning the extracted audio.
- Speaker Diarization: Show the "speaker_diarization.py" step that leverages the pretrained model in "pretrained_models/spkrec-ecapa-voxceleb".
- Transcription: Include "transcription.py" for converting audio to text.
- Sentiment Analysis: Include "sentiment_analysis.py" to classify sentiment (Positive, Negative, Neutral).
- Process Orchestration: "process_all_videos.py" ties all these steps together.
c. Data Storage and Outputs:
- Processed Data Storage: CSV files and cleaned audio are stored in "files_audio_video/output_audios_csv" and "cleaned_audio".
- Visualization: The "visualization_plots.py" script produces output plots (targeting "Output Plots" directory).
- Additional Data: Also include the "transcript_with_speakers.csv" output.
d. External Dependencies:
- Pretrained Models: Represent the pretrained speaker recognition model files (classifier, embedding_model, hyperparams, etc.) that the diarization component relies on.
- Python Environment: Note that the setup is based on a virtual environment with Python 3.11.

##### Relationships and Data Flow
- Sequential processing:
- Audio Extraction → Noise Removal → Speaker Diarization → Transcription → Sentiment Analysis.
- Each of these steps feeds into a CSV generation step in the output directory.
- CSV output to the Visualization component.
- Involvement of pretrained models that are ‘called’ or ‘referenced’ by the speaker diarization functionality.
- A side note for the file storing “cleaned_audio” as an output of the noise removal step.

##### Architectural Patterns and Design Principles
- Pipeline Design: Represent the project as a clear sequential processing pipeline where each module is a distinct component.
- Modularity: Each processing script (audio extraction, noise removal, etc.) is a modular component that can be executed separately.
- Separation of Concerns: Emphasize that ingestion, processing, external module integration (pretrained models), and visualization are clearly separated.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Ingestion
    subgraph "Data Ingestion"
        A["Video Input"]:::data_ingestion
    end

    %% Processing Pipeline
    subgraph "Processing Pipeline"
        B["Audio Extraction"]:::module
        C["Noise Removal"]:::module
        D["Speaker Diarization"]:::module
        E["Transcription"]:::module
        F["Sentiment Analysis"]:::module
    end

    %% Process Orchestration
    subgraph "Process Orchestration"
        O["Process Orchestration"]:::orchestration
    end

    %% Data Storage & Outputs
    subgraph "Data Storage & Outputs"
        G["CSV Output"]:::storage
        H["Cleaned Audio Output"]:::storage
        I["Transcript with Speakers"]:::storage
    end

    %% Visualization
    subgraph "Visualization"
        J["Visualization"]:::visualization
    end

    %% External Dependencies
    subgraph "External Dependencies"
        K["Pretrained Model"]:::external
    end

    %% Data Flow
    A -->|"to"| B
    B -->|"to"| C
    C -->|"to"| D
    D -->|"to"| E
    E -->|"to"| F
    F -->|"outputCSV"| G
    F -->|"outputTranscript"| I
    C -->|"outputCleaned"| H
    G -->|"to"| J
    D ---|"calls"| K

    %% Process Orchestration Dependencies
    O -->|"triggers"| B
    O -->|"triggers"| C
    O -->|"triggers"| D
    O -->|"triggers"| E
    O -->|"triggers"| F

    %% Click Events
    click A "https://github.com/agarwaldevesh374/gsoc-25-humanai-issr2-screening-test--nlp-tool/tree/master/files_audio_video/input_videos"
    click B "https://github.com/agarwaldevesh374/gsoc-25-humanai-issr2-screening-test--nlp-tool/blob/master/python_files/audio_extraction.py"
    click C "https://github.com/agarwaldevesh374/gsoc-25-humanai-issr2-screening-test--nlp-tool/blob/master/python_files/audio_noiseremoval.py"
    click D "https://github.com/agarwaldevesh374/gsoc-25-humanai-issr2-screening-test--nlp-tool/blob/master/python_files/speaker_diarization.py"
    click E "https://github.com/agarwaldevesh374/gsoc-25-humanai-issr2-screening-test--nlp-tool/blob/master/python_files/transcription.py"
    click F "https://github.com/agarwaldevesh374/gsoc-25-humanai-issr2-screening-test--nlp-tool/blob/master/python_files/sentiment_analysis.py"
    click O "https://github.com/agarwaldevesh374/gsoc-25-humanai-issr2-screening-test--nlp-tool/blob/master/python_files/process_all_videos.py"
    click G "https://github.com/agarwaldevesh374/gsoc-25-humanai-issr2-screening-test--nlp-tool/tree/master/files_audio_video/output_audios_csv"
    click H "https://github.com/agarwaldevesh374/gsoc-25-humanai-issr2-screening-test--nlp-tool/tree/master/cleaned_audio"
    click J "https://github.com/agarwaldevesh374/gsoc-25-humanai-issr2-screening-test--nlp-tool/blob/master/python_files/visualization_plots.py"
    click K "https://github.com/agarwaldevesh374/gsoc-25-humanai-issr2-screening-test--nlp-tool/tree/master/pretrained_models/spkrec-ecapa-voxceleb"
    click I "https://github.com/agarwaldevesh374/gsoc-25-humanai-issr2-screening-test--nlp-tool/blob/master/transcript_with_speakers.csv"

    %% Styles
    classDef data_ingestion fill:#cfe2f3,stroke:#2E86C1,stroke-width:2px;
    classDef module fill:#f9e79f,stroke:#f39c12,stroke-width:2px;
    classDef orchestration fill:#abebc6,stroke:#27ae60,stroke-width:2px;
    classDef storage fill:#d6eaf8,stroke:#3498db,stroke-width:2px;
    classDef visualization fill:#fadbd8,stroke:#e74c3c,stroke-width:2px;
    classDef external fill:#d5f5e3,stroke:#1e8449,stroke-width:2px;
```
