# yashspr/sentiment_analysis_ml_part architecture

> Architecture diagram of the GitHub repository yashspr/sentiment_analysis_ml_part, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yashspr/sentiment_analysis_ml_part
- Repository: https://github.com/yashspr/sentiment_analysis_ml_part

## Overview

##### Project type and purpose
- This is a full-stack sentiment-analysis application composed of two subprojects:
a) “sentiment_analysis_ml_part” – a Python-based ML pipeline and Flask microservice that trains and serves a sentiment-analysis model.
b) “web_sentiment_analysis” – a Node.js frontend/back-end web application that provides a UI, forwards user input to the ML service, and persists results in MongoDB.
- The Python part handles data ingestion (CSV), preprocessing, feature extraction, model training, and model serving via a REST API.
- The Node.js part hosts the client interface, calls the Flask API for sentiment predictions, and stores or displays results.

##### File structure (Python ML part)
- Top-level scripts:
- preprocess.py – cleans and tokenizes raw text.
- feature_extraction.py & ft.py – extract features (e.g., FastText embeddings).
- train.py & classification.py – train and evaluate the model.
- main.py – likely a convenience script combining preprocess/feature/train steps.
- server.py – Flask app exposing /predict endpoints.
- Data directory “csv_files/” – raw CSVs and training data.
- environment.yml / requirements.txt – dependencies (spaCy, FastText, Flask).
- No database here; persistence is in flat files and saved model artifacts.

##### README for additional insights
- The README confirms the two-service architecture.
- Python service uses Flask, spaCy, FastText.
- Node.js service requires MongoDB and serves the frontend.
- Interaction: Web UI → Node.js API → Python Flask → model → back to Node.js → UI/DB.

##### Components and relationships
a. Main components:
1. Web Browser (Client UI)
2. Node.js Web Server (Express or similar)
3. MongoDB Database
4. Python ML Service (Flask)
5. CSV Data Store & Model Artifacts
6. External Libraries (spaCy, FastText)
b. Interactions:
- Client sends sentiment query (text) → Node.js server
- Node.js server logs request to MongoDB (optional)
- Node.js forwards text via HTTP REST to Python ML service
- Python ML service:
- Runs preprocessing → feature extraction → loads trained model → classification → returns sentiment score
- Node.js receives result, stores it in MongoDB (optional), and returns response to client.
c. Architectural patterns:
- Microservices (separate Python service and Node.js service)
- Data pipeline (preprocess → feature extraction → training → model serving)
- REST API for inter-service communication
- “Train once, serve many” – training pipeline is decoupled from serving pipeline.
d. Key technologies:
- Python: Flask, spaCy (en_core_web_sm), FastText, pandas
- Node.js: Express (or similar), MongoDB driver
- MongoDB: storage for user requests and/or analytics
- Conda (environment.yml) for Python deps

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Client Layer
    subgraph "Client"
        UI["Web Browser"]:::frontend
    end

    %% Web Application Layer
    subgraph "Web Application"
        subgraph "Node.js Web Server"
            API["Express API"]:::nodejs
            DB["MongoDB"]:::db
        end
    end

    %% ML Service Layer
    subgraph "Python ML Service"
        Flask["Flask Sentiment API"]:::python
    end

    %% Data & Artifacts Layer
    subgraph "Data & Artifacts"
        CSV["CSV Data Store"]:::data
        Model["Model Artifacts & FastText Data"]:::data
        Env["Environment & Dependencies"]:::data
        Config["Constants & Config"]:::data
    end

    %% Batch Training Pipeline
    subgraph "Batch Training Pipeline"
        Preprocess["preprocess.py"]:::python
        FeatureExt["feature_extraction.py"]:::python
        FT["ft.py"]:::python
        TrainScript["train.py"]:::python
        Classify["classification.py"]:::python
        Main["main.py"]:::python
    end

    %% Interactions
    UI -->|"Submit Text"| API
    API -->|"Store Request"| DB
    API -->|"HTTP POST /predict"| Flask
    Flask -->|"Sentiment Score"| API
    API -->|"HTTP Response"| UI

    Flask -->|"Load CSV Data"| CSV
    Flask -->|"Load Model"| Model

    %% Batch pipeline flows
    Preprocess --> FeatureExt
    Preprocess --> FT
    FeatureExt --> TrainScript
    FT --> TrainScript
    TrainScript --> Classify
    Classify --> Model
    Main --> Preprocess
    Main --> FeatureExt
    Main --> FT
    Main --> TrainScript

    %% Click Events
    click Flask "https://github.com/yashspr/sentiment_analysis_ml_part/blob/master/server.py"
    click Preprocess "https://github.com/yashspr/sentiment_analysis_ml_part/blob/master/preprocess.py"
    click FeatureExt "https://github.com/yashspr/sentiment_analysis_ml_part/blob/master/feature_extraction.py"
    click FT "https://github.com/yashspr/sentiment_analysis_ml_part/blob/master/ft.py"
    click TrainScript "https://github.com/yashspr/sentiment_analysis_ml_part/blob/master/train.py"
    click Classify "https://github.com/yashspr/sentiment_analysis_ml_part/blob/master/classification.py"
    click Main "https://github.com/yashspr/sentiment_analysis_ml_part/blob/master/main.py"
    click CSV "https://github.com/yashspr/sentiment_analysis_ml_part/tree/master/csv_files/"
    click Model "https://github.com/yashspr/sentiment_analysis_ml_part/blob/master/csv_files/fasttext_data.txt"
    click Env "https://github.com/yashspr/sentiment_analysis_ml_part/blob/master/environment.yml"
    click Env "https://github.com/yashspr/sentiment_analysis_ml_part/blob/master/environment2.yml"
    click Env "https://github.com/yashspr/sentiment_analysis_ml_part/blob/master/requirements.txt"
    click Config "https://github.com/yashspr/sentiment_analysis_ml_part/blob/master/constants.py"

    %% Styles
    classDef frontend fill:#ccffcc,stroke:#333,stroke-width:1px
    classDef nodejs fill:#66cc66,stroke:#333,stroke-width:1px
    classDef python fill:#6699ff,stroke:#333,stroke-width:1px
    classDef db fill:#ffff99,stroke:#333,stroke-width:1px
    classDef data fill:#dddddd,stroke:#333,stroke-width:1px
```
