# muhammederbay10/text-sentiment-analysis architecture

> Architecture diagram of the GitHub repository muhammederbay10/text-sentiment-analysis, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/muhammederbay10/text-sentiment-analysis
- Repository: https://github.com/muhammederbay10/text-sentiment-analysis

## Overview

##### Project Type & Purpose
- This is a Python‐based full‐stack ML application combining a model training pipeline (using Hugging Face’s BERT fine-tuning) with a lightweight Streamlit web interface for live inference.
- Key use case: train on IMDB reviews, persist the fine-tuned model, and expose it via Streamlit for real-time sentiment prediction.

##### High-Level Components
a. Data Layer
- CSV data source (data/Movie_Dataset.csv)
- Local filesystem storage for checkpoints and any cached tokenizers
b. Preprocessing & Data Loading
- src/data_loader.py: reads, cleans, tokenizes (via Hugging Face AutoTokenizer), and prepares PyTorch or TensorFlow datasets
c. Training Pipeline
- src/train.py: configures the Hugging Face Trainer API, sets hyperparameters, kicks off fine-tuning, writes model checkpoints
d. Evaluation Module
- src/evaluate.py: loads a saved checkpoint, runs validation on held-out data, computes metrics (accuracy, F1, precision, recall)
e. Model Persistence
- Local folder (implicitly created by Trainer) storing model checkpoints, tokenizer config, training metrics
f. Inference / Serving Layer
- app.py (Streamlit): loads the final checkpoint and tokenizer, provides a web form for users to submit text, displays predictions in real time
g. External Dependencies
- Python >=3.8, streamlit, transformers, pandas, scikit-learn, torch or tensorflow

##### Component Interactions & Data Flow
1. Developer invokes src/train.py
– Data Loader fetches CSV → Tokenizer → Datasets → Trainer API → Model Checkpoint
2. Post-training, src/evaluate.py loads checkpoint → runs validation → outputs metrics
3. app.py (Streamlit) on startup loads the same checkpoint & tokenizer → waits for user input → runs inference → displays sentiment label
4. requirements.txt lists all dependencies; app.py & src modules import these libraries

##### Architectural Patterns & Design Principles
- Pipeline Pattern: clear separation of data loading, training, evaluation, and serving.
- Modularity: each major function is in its own script/module (single responsibility).
- Reproducibility: Trainer API ensures consistent checkpointing; requirements.txt pins dependencies.
- Lightweight Serving: uses Streamlit rather than a full web framework to minimize overhead.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data & Storage Layer
    subgraph "Data & Storage Layer"
        CSV["Movie_Dataset.csv"]:::storage
    end

    %% Codebase
    subgraph "Codebase"
        DL["Data Loader\n(src/data_loader.py)"]:::process
        Trainer["Trainer\n(src/train.py)"]:::process
        Evaluator["Evaluator\n(src/evaluate.py)"]:::process
        UI["Streamlit App\n(app.py)"]:::ui
    end

    %% Model Store
    ModelStore["Model Store\n(checkpoints & tokenizer)"]:::storage

    %% User Interaction
    UserInput["User Text Input"]:::input
    UIOutput["Sentiment Prediction"]:::input

    %% Documentation & Dependencies
    Deps["requirements.txt"]:::docs
    Docs["README.md"]:::docs

    %% Data Flow
    CSV -->|"raw CSV"| DL
    DL -->|"tokenized datasets"| Trainer
    Trainer -->|"model checkpoint"| ModelStore
    ModelStore -->|"saved model"| Evaluator
    ModelStore -->|"checkpoint + tokenizer"| UI
    UserInput -->|"text"| UI
    UI -->|"sentiment prediction"| UIOutput

    %% Dotted arrows for dependencies
    Deps -.-> DL
    Deps -.-> Trainer
    Deps -.-> Evaluator
    Deps -.-> UI
    Docs -.-> DL
    Docs -.-> Trainer
    Docs -.-> Evaluator
    Docs -.-> UI

    %% Click Events
    click CSV "https://github.com/muhammederbay10/text-sentiment-analysis/blob/main/data/Movie_Dataset.csv"
    click DL "https://github.com/muhammederbay10/text-sentiment-analysis/blob/main/src/data_loader.py"
    click Trainer "https://github.com/muhammederbay10/text-sentiment-analysis/blob/main/src/train.py"
    click Evaluator "https://github.com/muhammederbay10/text-sentiment-analysis/blob/main/src/evaluate.py"
    click UI "https://github.com/muhammederbay10/text-sentiment-analysis/blob/main/app.py"
    click Deps "https://github.com/muhammederbay10/text-sentiment-analysis/blob/main/requirements.txt"
    click Docs "https://github.com/muhammederbay10/text-sentiment-analysis/blob/main/README.md"

    %% Styles
    classDef storage fill:#ADD8E6,stroke:#000,stroke-width:1px;
    classDef process fill:#98FB98,stroke:#000,stroke-width:1px;
    classDef ui fill:#FFA500,stroke:#000,stroke-width:1px;
    classDef input fill:#E6E6FA,stroke:#000,stroke-width:1px;
    classDef docs fill:#D3D3D3,stroke:#000,stroke-width:1px;
```
