# theashutoshn/daily-five architecture

> Architecture diagram of the GitHub repository theashutoshn/daily-five, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/theashutoshn/daily-five
- Repository: https://github.com/theashutoshn/daily-five

## Overview

##### Project type and purpose
- From the file tree and README title (“daily-five”), this is a full-stack Node.js web application for aggregating, summarizing, and clustering news across multiple sources.
- Key features:
- Fetch RSS feeds (tech & finance)
- Clean and summarize articles (TextRank, Xenova Summarizer)
- Cluster related briefs across sources
- Serve processed data as JSON to a front-end

##### File structure
- Top-level files and folders:
- index.html, about.html, land.html, style.css → Static front-end pages & styling
- index.js → Root Node.js server entry point
- /news-triage → Module for fetching, cleaning, summarizing news
– app.js (server routes or orchestration)
– htmlcleaner.js, summarize.js, summ-textrank.js → processing logic
– /public/*.json → output summaries & RSS snapshots
- /multi-source-clustering → Module for clustering briefs from multiple sources
– ingest.js, embed.js, cluster.js, pick.js, build.js → pipeline scripts
– xenova-summarizer.js → optional summarization
– /public/*.json → clustered output

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External RSS Feeds
    subgraph "External RSS Feeds"
        techrss["Tech RSS"]:::external
        financrss["Finance RSS"]:::external
    end

    %% News-Triage Pipeline
    subgraph "News-Triage Pipeline"
        NTApp["app.js"]:::pipeline
        Cleaner["htmlcleaner.js"]:::pipeline
        TRank["summ-textrank.js"]:::pipeline
        XSumm["summarize.js"]:::pipeline
        NTStoreTechRSS["(JSON) tech-rss.json"]:::datastore
        NTStoreFinRSS["(JSON) fin-rss.json"]:::datastore
        NTStoreTechSum["(JSON) tech-news-summary.json"]:::datastore
        NTStoreFinSum["(JSON) fin-news-summary.json"]:::datastore
        NTStoreAll["(JSON) allItems.json"]:::datastore
    end

    %% Multi-Source Clustering Pipeline
    subgraph "Multi-Source Clustering Pipeline"
        Ingest["ingest.js"]:::pipeline
        Embed["embed.js"]:::pipeline
        Cluster["cluster.js"]:::pipeline
        Pick["pick.js"]:::pipeline
        Build["build.js"]:::pipeline
        Xeno2["xenova-summarizer.js"]:::pipeline
        CSFin["(JSON) finance-briefs.json"]:::datastore
        CSTech["(JSON) tech-briefs.json"]:::datastore
    end

    %% Node.js Server / API Layer
    subgraph "Node.js Server / API Layer"
        Index["index.js"]:::api
    end

    %% Front-End UI
    subgraph "Front-End UI"
        Home["index.html"]:::frontend
        Landing["land.html"]:::frontend
        About["about.html"]:::frontend
        Style["style.css"]:::frontend
    end

    %% Data Flows News-Triage
    techrss --> NTApp
    financrss --> NTApp
    NTApp --> Cleaner
    Cleaner --> TRank
    TRank --> XSumm
    XSumm --> NTStoreTechRSS
    XSumm --> NTStoreFinRSS
    XSumm --> NTStoreTechSum
    XSumm --> NTStoreFinSum
    XSumm --> NTStoreAll

    %% Data Flows Clustering
    NTStoreTechSum --> Ingest
    NTStoreFinSum --> Ingest
    Ingest --> Embed
    Embed --> Cluster
    Cluster --> Pick
    Pick --> Build
    Build --> CSFin
    Build --> CSTech

    %% Orchestration by API
    Index --> NTApp
    Index --> Build
    Index --> Home
    Index --> Landing
    Index --> About
    Index --> Style

    %% Front-End calls API for JSON
    Home -->|"GET tech-news-summary.json"| Index
    Home -->|"GET finance-briefs.json"| Index
    Landing -->|"GET tech-briefs.json"| Index
    About -->|"GET allItems.json"| Index

    %% Click Events
    click Home "https://github.com/theashutoshn/daily-five/blob/main/index.html"
    click Landing "https://github.com/theashutoshn/daily-five/blob/main/land.html"
    click About "https://github.com/theashutoshn/daily-five/blob/main/about.html"
    click Style "https://github.com/theashutoshn/daily-five/blob/main/style.css"
    click Index "https://github.com/theashutoshn/daily-five/blob/main/index.js"
    click NTApp "https://github.com/theashutoshn/daily-five/blob/main/news-triage/app.js"
    click Cleaner "https://github.com/theashutoshn/daily-five/blob/main/news-triage/htmlcleaner.js"
    click TRank "https://github.com/theashutoshn/daily-five/blob/main/news-triage/summ-textrank.js"
    click XSumm "https://github.com/theashutoshn/daily-five/blob/main/news-triage/summarize.js"
    click NTStoreTechRSS "https://github.com/theashutoshn/daily-five/blob/main/news-triage/public/tech-rss.json"
    click NTStoreFinRSS "https://github.com/theashutoshn/daily-five/blob/main/news-triage/public/fin-rss.json"
    click NTStoreTechSum "https://github.com/theashutoshn/daily-five/blob/main/news-triage/public/tech-news-summary.json"
    click NTStoreFinSum "https://github.com/theashutoshn/daily-five/blob/main/news-triage/public/fin-news-summary.json"
    click NTStoreAll "https://github.com/theashutoshn/daily-five/blob/main/news-triage/public/allItems.json"
    click Ingest "https://github.com/theashutoshn/daily-five/blob/main/multi-source-clustering/ingest.js"
    click Embed "https://github.com/theashutoshn/daily-five/blob/main/multi-source-clustering/embed.js"
    click Cluster "https://github.com/theashutoshn/daily-five/blob/main/multi-source-clustering/cluster.js"
    click Pick "https://github.com/theashutoshn/daily-five/blob/main/multi-source-clustering/pick.js"
    click Build "https://github.com/theashutoshn/daily-five/blob/main/multi-source-clustering/build.js"
    click Xeno2 "https://github.com/theashutoshn/daily-five/blob/main/multi-source-clustering/xenova-summarizer.js"
    click CSFin "https://github.com/theashutoshn/daily-five/blob/main/multi-source-clustering/public/finance-briefs.json"
    click CSTech "https://github.com/theashutoshn/daily-five/blob/main/multi-source-clustering/public/tech-briefs.json"

    %% Classes
    classDef frontend fill:#FFF9C4,stroke:#333,stroke-width:1px
    classDef api fill:#CFD8DC,stroke:#333,stroke-width:1px
    classDef pipeline fill:#E0F7FA,stroke:#333,stroke-width:1px
    classDef external fill:#FFE0B2,stroke:#333,stroke-width:1px
    classDef datastore fill:#C8E6C9,stroke:#333,stroke-width:1px shape:cilinder
```
