# sayantan0008/newsbite architecture

> Architecture diagram of the GitHub repository sayantan0008/newsbite, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sayantan0008/newsbite
- Repository: https://github.com/sayantan0008/newsbite

## Overview

NewsBite fetches news, cleans article text, summarizes and categorizes it, then optimizes and stores the processed articles for an interactive Streamlit dashboard. The dashboard reads saved data and offers browsing features including search, filters, bookmarks, and trending topics. The README documents these capabilities; the source excerpts show the batch runner’s calls and key utility behavior. Dashboard internals are only partially excerpted, so its feature links rely on the dashboard’s indexed references or README documentation. Compatibility patches are included as a supporting runtime layer, not as part of the article-processing sequence.

Main entry points: [`main.py`](https://github.com/sayantan0008/newsbite/blob/main/main.py) (Pipeline Runner), [`app/dashboard.py`](https://github.com/sayantan0008/newsbite/blob/main/app/dashboard.py) (News Dashboard), [`services/news_fetcher.py`](https://github.com/sayantan0008/newsbite/blob/main/services/news_fetcher.py) (News Fetcher), [`utils/bookmark_manager.py`](https://github.com/sayantan0008/newsbite/blob/main/utils/bookmark_manager.py) (Bookmarks)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_pipeline["News Pipeline"]
  node_runner["Pipeline Runner<br/>[main.py]"]
  node_fetcher["News Fetcher<br/>[news_fetcher.py]"]
  node_preprocess["Text Preprocessing<br/>[preprocessing.py]"]
  node_optimizer["Content Optimization<br/>[data_storage.py]"]
end

subgraph group_models["Analysis Models"]
  node_summarizer["Article Summarizer<br/>[summarizer.py]"]
  node_categorizer["Article Categorizer<br/>[categorizer.py]"]
end

subgraph group_dashboard["Dashboard Features"]
  node_dashboard_ui["News Dashboard<br/>[dashboard.py]"]
  node_search_filter["Search and Filters<br/>[dashboard.py]"]
  node_bookmarks["Bookmarks"]
  node_trending["Trending Topics<br/>[trending_topics.py]"]
end

subgraph group_state["Storage and Compatibility"]
  node_storage[("Processed News")]
  node_bookmark_store[("Bookmark File<br/>[bookmarks.json]")]
  node_trending_store[("Trending Data")]
  node_patch["Torch-Streamlit Patch"]
  node_launcher_patch["Streamlit Patch<br/>[streamlit_patch.py]"]
end

node_operator(("Pipeline Operator"))
node_newsapi{{"NewsAPI"}}
node_viewer(("Dashboard User"))

node_operator -->|"runs"| node_runner
node_runner -->|"fetches articles"| node_fetcher
node_fetcher -->|"requests news"| node_newsapi
node_runner -->|"cleans articles"| node_preprocess
node_runner -->|"summarizes articles"| node_summarizer
node_runner -->|"categorizes articles"| node_categorizer
node_runner -->|"optimizes content"| node_optimizer
node_runner -->|"saves results"| node_storage
node_viewer -->|"browses news"| node_dashboard_ui
node_dashboard_ui -->|"loads articles"| node_storage
node_dashboard_ui -->|"provides"| node_search_filter
node_dashboard_ui -->|"uses"| node_bookmarks
node_bookmarks -->|"reads and writes"| node_bookmark_store
node_dashboard_ui -->|"loads topics"| node_trending
node_trending -->|"reads and writes"| node_trending_store
node_runner -->|"applies patches"| node_patch
node_dashboard_ui -.->|"uses compatibility patch"| node_launcher_patch

click node_runner "https://github.com/sayantan0008/newsbite/blob/main/main.py"
click node_fetcher "https://github.com/sayantan0008/newsbite/blob/main/services/news_fetcher.py"
click node_preprocess "https://github.com/sayantan0008/newsbite/blob/main/utils/preprocessing.py"
click node_summarizer "https://github.com/sayantan0008/newsbite/blob/main/model/summarizer.py"
click node_categorizer "https://github.com/sayantan0008/newsbite/blob/main/model/categorizer.py"
click node_optimizer "https://github.com/sayantan0008/newsbite/blob/main/utils/data_storage.py"
click node_storage "https://github.com/sayantan0008/newsbite/tree/main/data"
click node_dashboard_ui "https://github.com/sayantan0008/newsbite/blob/main/app/dashboard.py"
click node_search_filter "https://github.com/sayantan0008/newsbite/blob/main/app/dashboard.py"
click node_bookmarks "https://github.com/sayantan0008/newsbite/blob/main/utils/bookmark_manager.py"
click node_bookmark_store "https://github.com/sayantan0008/newsbite/blob/main/data/bookmarks.json"
click node_trending "https://github.com/sayantan0008/newsbite/blob/main/utils/trending_topics.py"
click node_trending_store "https://github.com/sayantan0008/newsbite/blob/main/data/trending_topics.json"
click node_patch "https://github.com/sayantan0008/newsbite/blob/main/utils/torch_streamlit_patch.py"
click node_launcher_patch "https://github.com/sayantan0008/newsbite/blob/main/streamlit_patch.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_runner,node_fetcher,node_preprocess,node_optimizer,node_viewer toneBlue
class node_summarizer,node_categorizer toneAmber
class node_dashboard_ui,node_search_filter,node_bookmarks,node_trending,node_newsapi toneMint
class node_storage,node_bookmark_store,node_trending_store,node_patch,node_launcher_patch toneRose
class node_operator toneIndigo
```

## Components

### News Pipeline

- [Pipeline Runner](https://github.com/sayantan0008/newsbite/blob/main/main.py): `main.py`
- [News Fetcher](https://github.com/sayantan0008/newsbite/blob/main/services/news_fetcher.py): `services/news_fetcher.py`
- [Text Preprocessing](https://github.com/sayantan0008/newsbite/blob/main/utils/preprocessing.py): `utils/preprocessing.py`
- [Content Optimization](https://github.com/sayantan0008/newsbite/blob/main/utils/data_storage.py): `utils/data_storage.py`

### Analysis Models

- [Article Summarizer](https://github.com/sayantan0008/newsbite/blob/main/model/summarizer.py): `model/summarizer.py`
- [Article Categorizer](https://github.com/sayantan0008/newsbite/blob/main/model/categorizer.py): `model/categorizer.py`

### Dashboard Features

- [News Dashboard](https://github.com/sayantan0008/newsbite/blob/main/app/dashboard.py): `app/dashboard.py`
- [Search and Filters](https://github.com/sayantan0008/newsbite/blob/main/app/dashboard.py): `app/dashboard.py`
- [Bookmarks](https://github.com/sayantan0008/newsbite/blob/main/utils/bookmark_manager.py): `utils/bookmark_manager.py`
- [Trending Topics](https://github.com/sayantan0008/newsbite/blob/main/utils/trending_topics.py): `utils/trending_topics.py`

### Storage and Compatibility

- [Processed News](https://github.com/sayantan0008/newsbite/tree/main/data): `data`
- [Bookmark File](https://github.com/sayantan0008/newsbite/blob/main/data/bookmarks.json): `data/bookmarks.json`
- [Trending Data](https://github.com/sayantan0008/newsbite/blob/main/data/trending_topics.json): `data/trending_topics.json`
- [Torch-Streamlit Patch](https://github.com/sayantan0008/newsbite/blob/main/utils/torch_streamlit_patch.py): `utils/torch_streamlit_patch.py`
- [Streamlit Patch](https://github.com/sayantan0008/newsbite/blob/main/streamlit_patch.py): `streamlit_patch.py`

### Other components

- **Pipeline Operator**
- **NewsAPI**
- **Dashboard User**

## Connections

- Pipeline Operator → Pipeline Runner: runs (evidence: `main.py`)
- Pipeline Runner → News Fetcher: fetches articles (evidence: `main.py`)
- News Fetcher → NewsAPI: requests news (evidence: `services/news_fetcher.py`)
- Pipeline Runner → Text Preprocessing: cleans articles (evidence: `main.py`)
- Pipeline Runner → Article Summarizer: summarizes articles (evidence: `main.py`)
- Pipeline Runner → Article Categorizer: categorizes articles (evidence: `main.py`)
- Pipeline Runner → Content Optimization: optimizes content (evidence: `main.py`)
- Pipeline Runner → Processed News: saves results (evidence: `main.py`)
- Dashboard User → News Dashboard: browses news (evidence: `README.md`)
- News Dashboard → Processed News: loads articles (evidence: `app/dashboard.py`)
- News Dashboard → Search and Filters: provides (evidence: `README.md`)
- News Dashboard → Bookmarks: uses (evidence: `app/dashboard.py`)
- Bookmarks → Bookmark File: reads and writes (evidence: `utils/bookmark_manager.py`)
- News Dashboard → Trending Topics: loads topics (evidence: `app/dashboard.py`)
- Trending Topics → Trending Data: reads and writes (evidence: `utils/trending_topics.py`)
- Pipeline Runner → Torch-Streamlit Patch: applies patches (evidence: `main.py`)
- News Dashboard → Streamlit Patch: uses compatibility patch (evidence: `README.md`)