# shxntanu/email-classifier architecture

> Architecture diagram of the GitHub repository shxntanu/email-classifier, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shxntanu/email-classifier
- Repository: https://github.com/shxntanu/email-classifier

## Overview

##### Project Type & Purpose
- This is a full-stack solution designed for auto-classifying and routing emails. It uses a combination of email inbox monitoring, distributed task queues (using Celery and RabbitMQ), encryption/decryption, and large language model (LLM) analysis (via Ollama with models like Qwen 4b and Mixtral 8x7b) to determine optimal email routing.
- The system is built as a multi-component service that involves email ingestion, ML-driven classification/summarization, attachments handling, and routing via SMTP.

##### Analyzing the File Structure
- Multiple directories are present indicating separation of responsibilities:
- The “hackathon/ML” folder contains Python scripts for ML processing (e.g., app.py, tree.py, llm.py) and data files (CSV and JSON) used for training or routing.
- The “hackathon/client” folder is a Next.js-based frontend, suggesting a web interface for monitoring or manual interactions.
- The “hackathon/server” directory houses the backend logic (e.g., main.py, lib with modules such as forward.py and summarize.py, and data JSON files defining the routing hierarchy).
- There are “testing” directories for spam detection, dataset testing, and script-testing which can be considered supplementary validation components.
- The “src” top-level directory replicates core functionality (app.py, data, library modules, and tests).
- The presence of configuration files (like .env.example, tailwind.config.js, package.json) and build-related assets means the system is designed for containerized and production deployment.

To accurately represent the architecture, include the following elements and guidelines:

a. Main Components
- Email Receiver Service: A continuously running Python service that uses IMAP protocol to monitor a central email inbox.
- Parsing & Encryption Module: Handles email parsing, attachment extraction, and encryption before forwarding.
- Server/Decryption & LLM Processing: This service decrypts incoming emails, runs them through the LLM (Mixtral 8x7b or Qwen 4b via Ollama) for context extraction and summarization.
- Summarization Module: A lighter model (BART or Qwen 4b) that outputs a summary.
- Routing Engine: Uses JSON-based hierarchy (from data files like rag.json and heirarchy.json) to determine the correct destination team or department.
- Mail Forwarding Service: Uses SMTP for sending emails, with added features like watermarking for product confirmation.
- Frontend Dashboard (Client): A Next.js-based UI for feedback, monitoring, and possibly retraining the model.

b. Relationships and Interactions
- From Email Receiver → to the Task Queue.
- From Task Queue → to the Parsing & Encryption Module.
- From Parsing → to the Server/Decryption & LLM Processing.
- From LLM Processing → to the Summarization Module.
- From both LLM and Summarization outputs → to the Routing Engine.
- From Routing Engine → to Mail Forwarding.
- Interactions with external systems: IMAP and SMTP servers, the RabbitMQ broker, and the Ollama model service.

c. Architectural Patterns & Design Principles
- Decoupling of components using message brokers (RabbitMQ) and distributed task processing (Celery), which provides horizontal scalability.
- Modular design: separate directories indicate that the ML logic, API services, and web front-end are loosely coupled.

d. Relevant Technologies, Frameworks & Libraries
- Python (core language)
- Anaconda/Poetry for dependency management.
- Celery (distributed task queue) and RabbitMQ (message broker)
- Ollama for on-device LLM inference (Qwen 4b and Mixtral 8x7b)
- Next.js (frontend UI)
- Additional libraries for encryption, summarization, and attachment handling

##### Instructing the Principal Software Engineer
- Finally, consider adding a “Feedback Loop” section to denote the retraining and fine-tuning process based on user or system feedback.

## Diagram (Mermaid)

```mermaid
flowchart TD

    %% Data Ingestion Layer
    subgraph "Data Ingestion Layer"
        A["Email Receiver (IMAP Monitor)"]:::service
        A_IMAP["IMAP Server (External)"]:::external
    end

    %% Processing Layer
    subgraph "Processing Layer"
        B["Distributed Task Queue (Celery & RabbitMQ)"]:::infrastructure
        C["Parsing & Encryption Module"]:::logic
        D["Server/Decryption & LLM Processing"]:::logic
        E["Summarization Module"]:::logic
        F["Ollama LLM Service (Qwen 4b / Mixtral 8x7b)"]:::external
    end

    %% Routing & Delivery Layer
    subgraph "Routing & Delivery Layer"
        G["Routing Engine (JSON-based)"]:::logic
        H["Mail Forwarding Service (SMTP)"]:::service
        H_SMTP["SMTP Server (External)"]:::external
    end

    %% Client/User Interface Layer
    subgraph "Client/User Interface Layer"
        I["Frontend Dashboard (Next.js)"]:::frontend
    end

    %% Connections between layers
    A_IMAP -->|"feeds"| A
    A -->|"EmailReceived"| B
    B -->|"TaskDispatch"| C
    C -->|"ParsedEmail"| D
    D -->|"LLMRequest"| F
    F -->|"LLMResponse"| D
    D -->|"AnalysisData"| E
    D -->|"RoutingData"| G
    E -->|"Summary"| G
    G -->|"RoutingDecision"| H
    H -->|"SendEmail"| H_SMTP
    I -->|"Feedback"| D

    %% Click Events for Component Mapping
    click A "https://github.com/shxntanu/email-classifier/blob/main/hackathon/server/main.py"
    click B "https://github.com/shxntanu/email-classifier/blob/main/src/lib/celery.py"
    click B "https://github.com/shxntanu/email-classifier/blob/main/hackathon/server/src/tasks.py"
    click C "https://github.com/shxntanu/email-classifier/blob/main/hackathon/ML/decrypt.py"
    click C "https://github.com/shxntanu/email-classifier/blob/main/hackathon/server/lib/attachment.py"
    click D "https://github.com/shxntanu/email-classifier/blob/main/hackathon/ML/llm.py"
    click D "https://github.com/shxntanu/email-classifier/blob/main/hackathon/ML/bigllm.py"
    click D "https://github.com/shxntanu/email-classifier/blob/main/src/llm.py"
    click E "https://github.com/shxntanu/email-classifier/blob/main/hackathon/server/lib/summarize.py"
    click E "https://github.com/shxntanu/email-classifier/blob/main/src/lib/summarize.py"
    click G "https://github.com/shxntanu/email-classifier/blob/main/hackathon/server/data/rag.json"
    click G "https://github.com/shxntanu/email-classifier/blob/main/src/data/rag.json"
    click H "https://github.com/shxntanu/email-classifier/blob/main/hackathon/server/lib/forward.py"
    click H "https://github.com/shxntanu/email-classifier/blob/main/src/lib/forward.py"
    click I "https://github.com/shxntanu/email-classifier/tree/main/hackathon/client"

    %% Styling Classes
    classDef service fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef infrastructure fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef logic fill:#FFFACD,stroke:#000,stroke-width:2px;
    classDef external fill:#FFDEAD,stroke:#000,stroke-width:2px;
    classDef frontend fill:#DDA0DD,stroke:#000,stroke-width:2px;
```
