# nichuspn/gmail-rule-processor architecture

> Architecture diagram of the GitHub repository nichuspn/gmail-rule-processor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nichuspn/gmail-rule-processor
- Repository: https://github.com/nichuspn/gmail-rule-processor

## Overview

- It authenticates and interacts with the Gmail API (via OAuth credentials) to fetch emails.
- It then stores these emails in a PostgreSQL database and uses configurable, JSON-defined rules to process and take actions on these emails (like marking them as read or moving them).

##### Main Components
- Environment & Configuration:
- .env and .env.example files for database and other runtime configurations.
- credentials.json which holds OAuth credentials.
- config/rule.json which defines customizable rules.

- Data Acquisition (Gmail Fetch):
- Main script: 1_fetch_emails.py, which handles Gmail API authentication and email fetching.
- Gmail API integration components in entities/google (gmail.py for API operations and google.py for authentication).
- Helper functions in utils/gmail.py for additional operations.

- Data Storage & Persistence:
- PostgreSQL database.
- Database setup scripts in the db_setup directory (createdb.sh and init.sql).
- Database operations implemented in entities/db/googledb.py (handles operations for storing and querying Gmail data).

- Rule Processing & Actions:
- Main script: 2_update_emails.py, which loads and applies rules.
- Rule processing logic encapsulated in utils/rules_and_actions.py.
- The rules JSON structure (in config/rule.json) drives condition matching and actions to be executed.

- Testing:
- Test suite in test_rules_and_actions.py for validating rules, SQL generation, and correct action processing.

##### Interactions and Data Flow
- 1_fetch_emails.py:
- Starts with Gmail API authentication using entities/google/google.py.
- Uses entities/google/gmail.py (and utils/gmail.py) to interact with Gmail.
- Fetched emails are then passed to entities/db/googledb.py to be stored in the PostgreSQL database.

- 2_update_emails.py:
- Loads rule definitions from config/rule.json.
- Queries the database (via googledb.py) for emails.
- Passes the emails and the rules to utils/rules_and_actions.py, which processes the conditions and directs actions (e.g., marking emails read/unread) by possibly calling Gmail API operations again (through entities/google or utils/gmail).

##### Architectural Patterns and Design Principles
- Layered Architecture:
- Clear separation between data acquisition (Gmail API integration), storage (database layer), and business logic (rule processing).
- Configuration and Extensibility:
- Use of external JSON configuration for rules supports flexibility.
- Modular file organization (entities, utils, db_setup) ensures that each component is independently manageable and testable.

##### Relevant Technologies and Libraries
- Python (version 3.8+).
- Gmail API via Google OAuth (Google Cloud Platform).
- PostgreSQL for data storage.
- Shell scripting for setting up the database (in db_setup).
- JSON for rule configuration.

## Diagram (Mermaid)

```mermaid
flowchart TD

    %% Configuration & Environment Layer
    subgraph "Configuration & Environment"
        env1[".env"]:::config
        env2[".env.example"]:::config
        ruleConfig["config/rule.json"]:::config
    end

    %% Gmail Integration Layer
    subgraph "Gmail Integration"
        fetchScript["1_fetch_emails.py"]:::integration
        googleAuth["Gmail API Authentication"]:::integration
        gmailOps["Gmail API Operations"]:::integration
        gmailHelper["Gmail Helper Functions"]:::integration
    end

    %% Database Layer
    subgraph "Database Layer"
        dbCreatedb["db_setup/createdb.sh"]:::db
        dbInit["db_setup/init.sql"]:::db
        googledb["Database Operations"]:::db
        postgres["PostgreSQL"]:::external
    end

    %% Rule Processing Layer
    subgraph "Rule Processing"
        updateScript["2_update_emails.py"]:::rules
        ruleLogic["Rule Processing Logic"]:::rules
    end

    %% Testing Layer
    subgraph "Testing"
        testSuite["Test Suite"]:::test
    end

    %% Connections from Configuration to other layers
    ruleConfig -->|"configures"| googleAuth
    ruleConfig -->|"configures"| ruleLogic
    env1 ---|"env vars"| googleAuth
    env2 ---|"env vars"| googledb

    %% Gmail Integration Flow
    fetchScript -->|"calls"| googleAuth
    googleAuth -->|"provides auth to"| gmailOps
    fetchScript -->|"utilizes"| gmailHelper
    gmailOps -->|"stores emails in"| googledb

    %% Database Setup Flow
    dbCreatedb -->|"sets up"| postgres
    dbInit -->|"initializes"| postgres
    postgres -->|"accessible by"| googledb

    %% Rule Processing Flow
    updateScript -->|"triggers"| ruleLogic
    ruleLogic -->|"queries emails from"| googledb
    ruleLogic -->|"triggers actions via"| gmailOps

    %% Testing Flow
    testSuite -->|"tests"| ruleLogic

    %% Click Events for Configuration & Environment
    click env1 "https://github.com/nichuspn/gmail-rule-processor/blob/main/.env"
    click env2 "https://github.com/nichuspn/gmail-rule-processor/blob/main/.env.example"
    click ruleConfig "https://github.com/nichuspn/gmail-rule-processor/blob/main/config/rule.json"

    %% Click Events for Gmail Integration
    click fetchScript "https://github.com/nichuspn/gmail-rule-processor/blob/main/1_fetch_emails.py"
    click googleAuth "https://github.com/nichuspn/gmail-rule-processor/blob/main/entities/google/google.py"
    click gmailOps "https://github.com/nichuspn/gmail-rule-processor/blob/main/entities/google/gmail.py"
    click gmailHelper "https://github.com/nichuspn/gmail-rule-processor/blob/main/utils/gmail.py"

    %% Click Events for Database Layer
    click dbCreatedb "https://github.com/nichuspn/gmail-rule-processor/blob/main/db_setup/createdb.sh"
    click dbInit "https://github.com/nichuspn/gmail-rule-processor/blob/main/db_setup/init.sql"
    click googledb "https://github.com/nichuspn/gmail-rule-processor/blob/main/entities/db/googledb.py"

    %% Click Events for Rule Processing
    click updateScript "https://github.com/nichuspn/gmail-rule-processor/blob/main/2_update_emails.py"
    click ruleLogic "https://github.com/nichuspn/gmail-rule-processor/blob/main/utils/rules_and_actions.py"

    %% Click Event for Testing
    click testSuite "https://github.com/nichuspn/gmail-rule-processor/blob/main/test_rules_and_actions.py"

    %% Styles
    classDef config fill:#FFDDC1,stroke:#333,stroke-width:1px;
    classDef integration fill:#C1FFD7,stroke:#333,stroke-width:1px;
    classDef db fill:#C1D4FF,stroke:#333,stroke-width:1px;
    classDef rules fill:#FFC1FF,stroke:#333,stroke-width:1px;
    classDef test fill:#FFFFC1,stroke:#333,stroke-width:1px;
    classDef external fill:#D1D1D1,stroke:#333,stroke-width:1px;
```
