# amitvaranasi/rag-email-sorting architecture

> Architecture diagram of the GitHub repository amitvaranasi/rag-email-sorting, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/amitvaranasi/rag-email-sorting
- Repository: https://github.com/amitvaranasi/rag-email-sorting

## Overview

##### Project type and purpose
- This is a four-phase RAG (Retrieval-Augmented Generation) pipeline, implemented as a sequence of Python scripts, whose goal is to ingest your daily Gmail messages, index them with vector embeddings, generate a concise Markdown report via the Gemini API, and then email that report back to you each morning.

##### Relationships and data flow
- cron_job.sh invokes all four scripts sequentially, scheduled nightly by launchd

##### Architectural patterns and design principles
- Pipeline pattern: clear sequential phases (Ingest → Index → Generate → Deliver)
- Separation of concerns: each script handles one job
- Externalized configuration: credentials.json, token.json, environment variables for API keys
- Local persistence for durability: SQLite for raw data, ChromaDB for embeddings, file system for outputs

##### Technologies, frameworks, libraries
- Python 3.10+
- sqlite3 (standard library)
- ChromaDB (vector database)
- Google’s Gmail API (via google-auth, google-api-python-client)
- Google Gemini API (via HTTP/web client)
- macOS launchd (scheduler)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Orchestration & Scheduling
    subgraph "Orchestration & Scheduling"
        direction TB
        Launchd["launchd Scheduler"]:::scheduling
        Cron["cron_job.sh"]:::scheduling
    end
    Launchd -->|"runs daily at 00:00"| Cron

    %% Phase 1: Ingestion
    subgraph "Phase 1: Ingestion"    
        direction TB
        GmailAPI["Gmail API"]:::external
        Fetcher["gmail_fetcher.py"]:::ingestion
        SQLiteDB["my_emails.db"]:::database
    end
    Cron -->|"invoke"| Fetcher
    GmailAPI -->|"fetch raw emails"| Fetcher
    Fetcher -->|"store raw emails"| SQLiteDB

    %% Phase 2: Indexing
    subgraph "Phase 2: Indexing"
        direction TB
        SQLiteDB2["my_emails.db"]:::database
        Indexer["indexing.py"]:::indexing
        ChromaDB["email_vector_db/"]:::database
        GeminiEmbeddings["Gemini API"]:::external
    end
    Cron -->|"invoke"| Indexer
    SQLiteDB2 -->|"read unprocessed emails"| Indexer
    Indexer -->|"request embeddings"| GeminiEmbeddings
    GeminiEmbeddings -->|"return embeddings"| Indexer
    Indexer -->|"store embeddings"| ChromaDB

    %% Phase 3: Generation
    subgraph "Phase 3: Generation"
        direction TB
        ChromaDB2["email_vector_db/"]:::database
        Reporter["report_generation.py"]:::generation
        GeminiQA["Gemini API"]:::external
        ReportFile["daily_report_YYYY-MM-DD.md"]:::database
    end
    Cron -->|"invoke"| Reporter
    ChromaDB2 -->|"read embeddings"| Reporter
    Reporter -->|"generate questions & prompts"| GeminiQA
    GeminiQA -->|"return answers"| Reporter
    Reporter -->|"write report"| ReportFile

    %% Phase 4: Delivery
    subgraph "Phase 4: Delivery"
        direction TB
        ReportFile2["daily_report_YYYY-MM-DD.md"]:::database
        Sender["send_report.py"]:::delivery
        GmailAPI2["Gmail API"]:::external
    end
    Cron -->|"invoke"| Sender
    ReportFile2 -->|"load report"| Sender
    Sender -->|"send via email"| GmailAPI2

    %% Legend
    subgraph Legend
        direction TB
        A["Rectangle: Script/Component"]:::legend
        B["Cylinder: Data Store/Artifact"]:::legend
        C["Cloud: External Service"]:::legend
    end

    %% Click Events
    click Fetcher "https://github.com/amitvaranasi/rag-email-sorting/blob/main/gmail_fetcher.py"
    click SQLiteDB "https://github.com/amitvaranasi/rag-email-sorting/blob/main/my_emails.db"
    click Indexer "https://github.com/amitvaranasi/rag-email-sorting/blob/main/indexing.py"
    click ChromaDB "https://github.com/amitvaranasi/rag-email-sorting/tree/main/email_vector_db/"
    click Reporter "https://github.com/amitvaranasi/rag-email-sorting/blob/main/report_generation.py"
    click Sender "https://github.com/amitvaranasi/rag-email-sorting/blob/main/send_report.py"
    click Cron "https://github.com/amitvaranasi/rag-email-sorting/blob/main/cron_job.sh"

    %% Styles
    classDef ingestion fill:#D0E8FF,stroke:#3399FF,stroke-width:2px
    classDef indexing fill:#DFF2D8,stroke:#33AA33,stroke-width:2px
    classDef generation fill:#FFE4B2,stroke:#FFAA33,stroke-width:2px
    classDef delivery fill:#E8D0FF,stroke:#AA33FF,stroke-width:2px
    classDef external fill:#F0F0F0,stroke:#888888,stroke-dasharray: 5 5
    classDef database fill:#FFF0D0,stroke:#CCAA33,stroke-width:2px,stroke-dasharray: 2 2
    classDef scheduling fill:#F0EFEF,stroke:#555555,stroke-width:2px,stroke-dasharray: 3 3
    classDef legend fill:#FFFFFF,stroke:#000000,stroke-width:1px,stroke-dasharray: 2 2
```
