# mujtaba-a-khan/call-analytics-system architecture

> Architecture diagram of the GitHub repository mujtaba-a-khan/call-analytics-system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mujtaba-a-khan/call-analytics-system
- Repository: https://github.com/mujtaba-a-khan/call-analytics-system

## Overview

##### Project type and purpose
- This is a full-stack, locally hosted call analytics system.
- Purpose: ingest call audio and CSV transcripts, transcribe speech, label calls, index and semantically search them, and present dashboards and Q&A via a Streamlit UI.

##### File structure
- Top-level directories:
- config/ (TOML config for audio, models, rules, vectorstore)
- src/
– core/ (audio & CSV processing, labeling, storage)
– ml/ (Whisper STT, embeddings, LLM interface)
– vectordb/ (ChromaDB client, indexer, retriever)
– analysis/ (filters, aggregations, semantic search, query interpreter)
– ui/ (Streamlit app, pages: upload, dashboard, analysis, Q&A; UI components)
– utils/ (logging, file I/O, text utils, validators)
– cli.py (command-line entry points)
- data/ (uploads, processed, cache, vectorstore storage)
- models/ (downloaded Whisper models)
- scripts/ (environment setup, model download, index rebuild)
- tests/
- Configuration: pyproject.toml, config/*.toml
- Deployment/CI: Jenkinsfile, GitHub workflows (implied), Docker not present but can be added.

##### README for insights
- Tech stack: Python 3.13, Streamlit, ChromaDB, OpenAI Whisper, sentence-transformers, FFmpeg
- Features: audio transcription, CSV ingest, intelligent labeling, semantic search, NL Q&A, dashboards.
- Data flow: upload → process (Whisper STT / CSV) → label → embed → store in ChromaDB → query & retrieve → render in UI.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Layer
    Client["Web Browser"]:::frontend

    %% UI Layer
    subgraph "Streamlit UI"
        direction TB
        UIApp["Streamlit App"]:::frontend
        UploadPage["Upload Page"]:::frontend
        DashboardPage["Dashboard Page"]:::frontend
        AnalysisPage["Analysis Page"]:::frontend
        QAPage["Q&A Page"]:::frontend
        Charts["Charts Component"]:::frontend
        FiltersUI["Filters Component"]:::frontend
        Metrics["Metrics Component"]:::frontend
        Tables["Tables Component"]:::frontend
    end

    %% API / CLI Layer
    CLI["CLI Entry Point"]:::api

    %% Core Services Layer
    subgraph "Core Services"
        direction TB
        AudioProcessor["AudioProcessor"]:::core
        CSVProcessor["CSVProcessor"]:::core
        LabelingEngine["LabelingEngine"]:::core
        StorageManager["StorageManager"]:::core
        DataSchema["Data Schema Definitions"]:::core
    end

    %% ML Services Layer
    subgraph "ML Services"
        direction TB
        WhisperSTT["Whisper STT Service"]:::ml
        EmbeddingsGen["Embeddings Generator"]:::ml
        LLMInterface["LLM Interface"]:::ml
    end

    %% Vector DB Layer
    subgraph "VectorDB (ChromaDB)"
        direction TB
        ChromadbClient["ChromaDB Client"]:::vectordb
        Indexer["Indexer"]:::vectordb
        Retriever["Retriever"]:::vectordb
    end

    %% Analysis Services Layer
    subgraph "Analysis Services"
        direction TB
        FiltersModule["Filters Module"]:::analysis
        Aggregations["Aggregations Module"]:::analysis
        SemanticSearch["Semantic Search Module"]:::analysis
        QueryInterpreter["Query Interpreter"]:::analysis
    end

    %% Utility Modules
    subgraph "Utilities"
        direction TB
        FileHandlers["File Handlers"]:::utils
        TextProcessing["Text Processing Utilities"]:::utils
        Formatters["Formatters"]:::utils
        Validators["Validators"]:::utils
        Logger["Logger"]:::utils
    end

    %% Data Stores
    subgraph "Data Stores"
        direction TB
        LocalFiles["Local File Storage"]:::storage
        VectorStore["Vector Store"]:::storage
        ConfigFiles["Config Files"]:::storage
    end

    %% External Dependencies
    subgraph "External"
        direction TB
        FFmpeg["FFmpeg"]:::external
        GPU["GPU/CUDA"]:::external
    end

    %% Scripts / Batch Jobs
    subgraph "Scripts"
        direction TB
        SetupEnv["Setup Environment"]:::script
        DownloadModels["Download Models"]:::script
        LaunchUI["Launch UI"]:::script
        RebuildIndex["Rebuild Index"]:::script
    end

    %% Connections: Ingestion Pipeline
    Client --> UIApp
    UIApp --> AudioProcessor
    UIApp --> CSVProcessor
    CLI --> AudioProcessor
    CLI --> CSVProcessor

    AudioProcessor --> WhisperSTT
    CSVProcessor --> Formatters

    WhisperSTT --> LabelingEngine
    Formatters --> LabelingEngine
    LabelingEngine --> StorageManager
    StorageManager --> FileHandlers
    StorageManager --> EmbeddingsGen
    EmbeddingsGen --> Indexer
    Indexer --> VectorStore

    %% Query Pipeline
    UIApp --> QueryInterpreter
    QueryInterpreter --> Retriever
    Retriever --> SemanticSearch
    Retriever --> Aggregations
    SemanticSearch --> UIApp
    Aggregations --> UIApp

    %% Config usage
    ConfigFiles --> UIApp
    ConfigFiles --> AudioProcessor
    ConfigFiles --> CSVProcessor
    ConfigFiles --> WhisperSTT
    ConfigFiles --> EmbeddingsGen
    ConfigFiles --> LLMInterface
    ConfigFiles --> ChromadbClient
    ConfigFiles --> Indexer

    %% Scripts interactions
    SetupEnv --> ConfigFiles
    SetupEnv --> GPU
    DownloadModels --> WhisperSTT
    DownloadModels --> models["models/"]:::storage
    RebuildIndex --> VectorStore
    LaunchUI --> UIApp

    %% External tools
    AudioProcessor --> FFmpeg
    WhisperSTT --> GPU
    EmbeddingsGen --> GPU

    %% UI Components integration
    UIApp --> UploadPage
    UIApp --> DashboardPage
    UIApp --> AnalysisPage
    UIApp --> QAPage
    UIApp --> Charts
    UIApp --> FiltersUI
    UIApp --> Metrics
    UIApp --> Tables

    %% Click Events
    click ConfigFiles "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/config/app.toml"
    click ConfigFiles "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/config/fields.toml"
    click ConfigFiles "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/config/models.toml"
    click ConfigFiles "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/config/rules.toml"
    click ConfigFiles "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/config/vectorstore.toml"
    click CLI "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/cli.py"
    click UIApp "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/ui/app.py"
    click UploadPage "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/ui/pages/upload.py"
    click DashboardPage "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/ui/pages/dashboard.py"
    click AnalysisPage "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/ui/pages/analysis.py"
    click QAPage "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/ui/pages/qa_interface.py"
    click Charts "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/ui/components/charts.py"
    click FiltersUI "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/ui/components/filters.py"
    click Metrics "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/ui/components/metrics.py"
    click Tables "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/ui/components/tables.py"
    click AudioProcessor "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/core/audio_processor.py"
    click CSVProcessor "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/core/csv_processor.py"
    click LabelingEngine "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/core/labeling_engine.py"
    click StorageManager "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/core/storage_manager.py"
    click DataSchema "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/core/data_schema.py"
    click WhisperSTT "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/ml/whisper_stt.py"
    click EmbeddingsGen "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/ml/embeddings.py"
    click LLMInterface "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/ml/llm_interface.py"
    click ChromadbClient "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/vectordb/chroma_client.py"
    click Indexer "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/vectordb/indexer.py"
    click Retriever "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/vectordb/retriever.py"
    click FiltersModule "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/analysis/filters.py"
    click Aggregations "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/analysis/aggregations.py"
    click SemanticSearch "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/analysis/semantic_search.py"
    click QueryInterpreter "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/analysis/query_interpreter.py"
    click FileHandlers "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/utils/file_handlers.py"
    click TextProcessing "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/utils/text_processing.py"
    click Formatters "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/utils/formatters.py"
    click Validators "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/utils/validators.py"
    click Logger "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/src/utils/logger.py"
    click SetupEnv "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/scripts/setup_environment.py"
    click DownloadModels "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/scripts/download_models.py"
    click LaunchUI "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/scripts/launch_ui.py"
    click RebuildIndex "https://github.com/mujtaba-a-khan/call-analytics-system/blob/main/scripts/rebuild_index.py"

    %% Styles
    classDef frontend fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef api fill:#87CEFA,stroke:#333,stroke-width:1px
    classDef core fill:#90EE90,stroke:#333,stroke-width:1px
    classDef ml fill:#FFA500,stroke:#333,stroke-width:1px
    classDef vectordb fill:#DDA0DD,stroke:#333,stroke-width:1px
    classDef analysis fill:#9370DB,stroke:#333,stroke-width:1px
    classDef utils fill:#C0C0C0,stroke:#333,stroke-width:1px
    classDef storage fill:#A9A9A9,stroke:#333,stroke-width:1px
    classDef external fill:#F0E68C,stroke:#333,stroke-width:1px
    classDef script fill:#FFE4B5,stroke:#333,stroke-width:1px

    %% Optional Models folder
    models["models/"]:::storage
```
