# swagath18/genai-ecom-review-analyzer architecture

> Architecture diagram of the GitHub repository swagath18/genai-ecom-review-analyzer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/swagath18/genai-ecom-review-analyzer
- Repository: https://github.com/swagath18/genai-ecom-review-analyzer

## Overview

##### Main Components
- User Interface / Client: Although not a web UI per se, the “user” represents the data scientist or e-commerce professional triggering the pipeline (via CLI, script execution, or API call).
- Data Ingestion & Preprocessing: Includes files such as data_loader.py and Datacleaning.ipynb that load and clean the Walmart product review dataset (with the cleaned data saved into cleaneddata.csv).
- Core Analysis Pipeline:
- Review Analysis Module: review_analyzer.py which processes customer reviews using NLP techniques.
- NLP Modules: Leveraging advanced libraries like Transformers and Langchain, which are used for sentiment analysis and review summarization.
- ML Models: Embedded within the analysis pipeline where sentiment analysis, text summarization, and key metrics calculations take place.
- Deployment Component: deploy.py and main.py serve as the orchestration layer, setting up the runtime environment and starting the process.
- MLOps & Monitoring Module:
- MLflow Integration: Evidenced by mlops_tracker.py and the mlruns directory, this component logs performance metrics (e.g., total reviews, sentiment distribution, and average rating) and hyperparameters.
- Continuous Monitoring: Through MLflow’s tracking dashboard, where metrics (accuracy, loss, etc.) are recorded to ensure continuous oversight of model performance.
- External Dependencies: Libraries like MLflow, Langchain, and Transformers are core to the system’s operations.

##### Relationships & Data Flow
- Deployment → Data Ingestion & Preprocessing: Once initiated, the system ingests raw data (data_loader.py) and cleans it (Datacleaning.ipynb becomes the source for the cleaneddata.csv).
- Data Preprocessing → Core Analysis Pipeline: The cleaned data is then processed through the review analysis module (review_analyzer.py) where NLP tasks are applied.
- Analysis Pipeline → Metrics & Logging: The derived insights (review summaries and key metrics) are passed to the MLflow tracking system (integrated via mlops_tracker.py), which saves logs into the mlruns folder.
- Final Output → User: The final analysis results (structured JSON containing summaries and metrics) are then output and can be consumed by the user.

##### Architectural Patterns & Design Principles
- MLOps Integration: Clearly illustrate how MLflow is interwoven throughout the process for performance tracking, ensuring the reproducibility and continuous monitoring of experiments.
- Modular Design: Emphasize modularity with clearly separated modules for loading, processing, analysis, and logging so that each component can be improved independently.

## Diagram (Mermaid)

```mermaid
flowchart TD
    A["User Initiation"]:::client

    subgraph "Deployment Orchestration"
        B["Deploy Script (deploy.py)"]:::deploy
        C["Main Script (main.py)"]:::deploy
        B --> D1["Deployment Orchestration Completed"]:::deploy
        C --> D1
    end

    D1 -->|"Ingest_raw_data"| D["Data Loader (data_loader.py)"]:::ingest
    D -->|"Clean_data"| E["Data Cleaning (Datacleaning.ipynb & cleaneddata.csv)"]:::ingest
    E -->|"Provides_cleanedData"| F["Review Analysis (review_analyzer.py)"]:::analysis
    F -->|"PerformanceMetrics_&_ExperimentLogging"| G["MLflow Tracker (mlops_tracker.py)"]:::monitor
    G -->|"Stores_logs"| H["MLflow Runs (mlruns)"]:::monitor
    H -->|"Outputs_metrics"| I["Final Results Output (JSON)"]:::client

    %% Feedback loops
    H -.->|"feedback"| F
    H -.->|"feedback"| B

    %% Click Events
    click B "https://github.com/swagath18/genai-ecom-review-analyzer/blob/main/deploy.py"
    click C "https://github.com/swagath18/genai-ecom-review-analyzer/blob/main/main.py"
    click D "https://github.com/swagath18/genai-ecom-review-analyzer/blob/main/data_loader.py"
    click E "https://github.com/swagath18/genai-ecom-review-analyzer/blob/main/Datacleaning.ipynb"
    click F "https://github.com/swagath18/genai-ecom-review-analyzer/blob/main/review_analyzer.py"
    click G "https://github.com/swagath18/genai-ecom-review-analyzer/blob/main/mlops_tracker.py"
    click H "https://github.com/swagath18/genai-ecom-review-analyzer/tree/main/mlruns"

    %% Styles
    classDef client fill:#FFD1DC,stroke:#333,stroke-width:2px;
    classDef deploy fill:#B3E5FC,stroke:#333,stroke-width:2px;
    classDef ingest fill:#C8E6C9,stroke:#333,stroke-width:2px;
    classDef analysis fill:#F8BBD0,stroke:#333,stroke-width:2px;
    classDef monitor fill:#FFF9C4,stroke:#333,stroke-width:2px;
```
