How nichuspn/gmail-rule-processor works
• It authenticates and interacts with the Gmail API (via OAuth credentials) to fetch emails.
• It then stores these emails in a PostgreSQL database and uses configurable, JSON-defined rules to process and take actions on these emails (like marking them as read or moving them).
Main Components
• Environment & Configuration:
• .env and .env.example files for database and other runtime configurations.
• credentials.json which holds OAuth credentials.
• config/rule.json which defines customizable rules.
• Data Acquisition (Gmail Fetch):
• Main script: 1_fetch_emails.py, which handles Gmail API authentication and email fetching.
• Gmail API integration components in entities/google (gmail.py for API operations and google.py for authentication).
• Helper functions in utils/gmail.py for additional operations.
• Data Storage & Persistence:
• PostgreSQL database.
• Database setup scripts in the db_setup directory (createdb.sh and init.sql).
• Database operations implemented in entities/db/googledb.py (handles operations for storing and querying Gmail data).
• Rule Processing & Actions:
• Main script: 2_update_emails.py, which loads and applies rules.
• Rule processing logic encapsulated in utils/rules_and_actions.py.
• The rules JSON structure (in config/rule.json) drives condition matching and actions to be executed.
• Testing:
• Test suite in test_rules_and_actions.py for validating rules, SQL generation, and correct action processing.
Interactions and Data Flow
• 1_fetch_emails.py:
• Starts with Gmail API authentication using entities/google/google.py.
• Uses entities/google/gmail.py (and utils/gmail.py) to interact with Gmail.
• Fetched emails are then passed to entities/db/googledb.py to be stored in the PostgreSQL database.
• 2_update_emails.py:
• Loads rule definitions from config/rule.json.
• Queries the database (via googledb.py) for emails.
• Passes the emails and the rules to utils/rules_and_actions.py, which processes the conditions and directs actions (e.g., marking emails read/unread) by possibly calling Gmail API operations again (through entities/google or utils/gmail).
Architectural Patterns and Design Principles
• Layered Architecture:
• Clear separation between data acquisition (Gmail API integration), storage (database layer), and business logic (rule processing).
• Configuration and Extensibility:
• Use of external JSON configuration for rules supports flexibility.
• Modular file organization (entities, utils, db_setup) ensures that each component is independently manageable and testable.
Relevant Technologies and Libraries
• Python (version 3.8+).
• Gmail API via Google OAuth (Google Cloud Platform).
• PostgreSQL for data storage.
• Shell scripting for setting up the database (in db_setup).
• JSON for rule configuration.