# fuzzy-memory/caffeine-print architecture

> Architecture diagram of the GitHub repository fuzzy-memory/caffeine-print, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fuzzy-memory/caffeine-print
- Repository: https://github.com/fuzzy-memory/caffeine-print

## Overview

##### Main Components
- News Retrieval Module (src/get_news.py): This component pulls news from worldnewsapi.com.
- Data Processing Modules:
- Clustering (src/similarity_clustering.py): Uses S-BERT sentence embeddings and DBSCAN to cluster similar news stories.
- Ranking (src/ranking.py): Produces weighted scores based on text, source, sentiment, and ChatGPT-generated feedback.
- HTML Generation Module (src/htmls.py): Arranges the picked articles into an HTML email body.
- Email Delivery (likely integrated into src/main.py, together with recipient-related logic in src/db/recipients.py): Handles the actual sending process.
- Data Models and Database Handling (src/data_models.py, src/db/data_models.py, and src/db/__init__.py): Abstract the data schema and possibly manage persistent or temporary storage.
- Configuration and Other Utilities:
- Properties (src/properties.py): Likely holds configuration parameters (e.g., API keys, thresholds).
- Utilities (src/utils.py): Contains helper functions used across components.
- External Data (assets/news_sources.json): Provides information on news sources and their ranking bases.

##### Relationships and Interactions
- The News Retrieval Module fetches data from an external API.
- The raw news data is sent to the Clustering Module where similar stories are grouped.
- The Ranking Module then takes clustered stories (and possibly additional data from the configuration or database modules) to rank articles.
- The HTML Generator receives the selected and ranked articles, formatting them into an email-ready HTML body.
- The Email Delivery system (within the main execution path in src/main.py) draws on the recipients (from src/db/recipients.py) and sends the final email.
- Cross-cutting concerns like data models, utility functions, and configuration support multiple components, ensuring data consistency and proper processing flow.

##### Architectural Patterns and Technologies
- Design follows a modular separation of concerns ensuring each module (retrieval, clustering, ranking, formatting, and mailing) is encapsulated.
- Although not explicitly an MVC, the separation helps distinguish data (models), processing logic (controllers, clustering/ranking), and presentation (HTML generation).
- Use of a configuration (pyproject.toml and src/properties.py) allows flexibility in deployment and possibly dependency injection.
- GitHub Actions (in .github/workflows/daily_run.yml) indicates an automation pipeline (likely for scheduled runs).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Sources & Triggers
    subgraph "External Sources & Triggers"
        API["External News API (worldnewsapi.com)"]:::external
        GitCI["CI/CD Pipeline (Scheduled Run)"]:::cdpipeline
        ExData["News Sources JSON"]:::external
    end

    %% Core Processing Modules
    subgraph "Core Processing Modules"
        NewsRec["News Retrieval Module"]:::processing
        Clust["Clustering Module (S-BERT & DBSCAN)"]:::processing
        Rank["Ranking Module (Weighted Score, Sentiment & ChatGPT)"]:::processing
        HTMLGen["HTML Generation Module"]:::output
        EmailDel["Email Delivery Module"]:::output
    end

    %% Supporting Modules
    subgraph "Supporting Modules"
        Config["Configuration & Properties"]:::config
        Utils["Utility Functions"]:::utility
        DB["Data Models & Database Handling"]:::db
        Recipients["Recipients Data"]:::db
    end

    %% Connections: External Sources & Triggers to Core Processing
    GitCI -->|"TriggersJob"| NewsRec
    API   -->|"FetchArticles"| NewsRec
    ExData -->|"SourceInfo"| Rank

    %% Core Processing Flow
    NewsRec -->|"RawNews"| Clust
    Clust   -->|"ClusteredArticles"| Rank
    Rank    -->|"RankedArticles"| HTMLGen
    HTMLGen -->|"HTMLContent"| EmailDel

    %% Supporting Modules providing cross-cutting inputs
    Config -->|"Config"| NewsRec
    Config -->|"Config"| Clust
    Config -->|"Config"| Rank
    Utils  -->|"Utils"| NewsRec
    Utils  -->|"Utils"| Clust
    Utils  -->|"Utils"| Rank
    DB     -->|"DataModels"| EmailDel
    Recipients -->|"RecipientInfo"| EmailDel

    %% Click Events
    click NewsRec "https://github.com/fuzzy-memory/caffeine-print/blob/master/src/get_news.py"
    click Clust "https://github.com/fuzzy-memory/caffeine-print/blob/master/src/similarity_clustering.py"
    click Rank "https://github.com/fuzzy-memory/caffeine-print/blob/master/src/ranking.py"
    click HTMLGen "https://github.com/fuzzy-memory/caffeine-print/blob/master/src/htmls.py"
    click EmailDel "https://github.com/fuzzy-memory/caffeine-print/blob/master/src/main.py"
    click DB "https://github.com/fuzzy-memory/caffeine-print/blob/master/src/data_models.py"
    click Recipients "https://github.com/fuzzy-memory/caffeine-print/blob/master/src/db/recipients.py"
    click Config "https://github.com/fuzzy-memory/caffeine-print/blob/master/src/properties.py"
    click Utils "https://github.com/fuzzy-memory/caffeine-print/blob/master/src/utils.py"
    click ExData "https://github.com/fuzzy-memory/caffeine-print/blob/master/assets/news_sources.json"
    click GitCI "https://github.com/fuzzy-memory/caffeine-print/blob/master/.github/workflows/daily_run.yml"

    %% Styles
    classDef external fill:#f9c74f,stroke:#333,stroke-width:2px;
    classDef processing fill:#90be6d,stroke:#333,stroke-width:2px;
    classDef output fill:#f9844a,stroke:#333,stroke-width:2px;
    classDef config fill:#577590,stroke:#333,stroke-width:2px,color:#fff;
    classDef utility fill:#277da1,stroke:#333,stroke-width:2px,color:#fff;
    classDef db fill:#ef476f,stroke:#333,stroke-width:2px,color:#fff;
    classDef cdpipeline fill:#b773ef,stroke:#333,stroke-width:2px,color:#fff;
```
