How jacob5585/rss-summarizer works
Project Type and Purpose
– This is a Python-based full-stack web application (likely using Flask) whose job is to:
• Periodically fetch news articles from various RSS feeds (finance, science, tech, world news).
• Summarize each article using an LLM (via Ollama).
• Optionally convert the summary to audio via a text-to-speech engine.
• Store raw articles, summaries, and audio files in local directories.
• Serve a simple web UI for users to browse summaries and playback audio, plus view logs.
File Structure
Top‐level directories/files:
– src/
• app.py – the Flask (or equivalent) web server, routes, and template rendering.
• main.py – entry point that likely orchestrates scheduled scraping and summarization runs.
• scrape_articles.py – contains RSS-feed-fetching and HTML-parsing logic.
• ollama_sum.py – wraps calls to the Ollama LLM for summarization.
• text2speech.py – handles conversion of text summaries to audio files.
• files.py – manages filesystem I/O (articles/, audio/, logs/).
• logs.py – configures and writes log entries under logs/.
• rss.py – RSS parser or feed-list definitions.
• requirements.txt – Python dependencies (Flask, feedparser, requests, Ollama SDK, TTS libraries, etc.).
– templates/
• main_menu.html – landing page with navigation.
• articles.html – displays article summaries and audio playback links.
• logs.html – displays application logs.
– articles/, audio/, logs/ – persistent storage for data and logs, organized by category.
README
– README is minimal but confirms “python rss feed summarizer application.”
– No explicit mention of database: the app uses flat files.
– No other external services besides RSS feeds and Ollama/TTS libraries.
A. Presentation Layer (Frontend)
• Component: Flask Web Server (app.py)
• Templates: main_menu.html, articles.html, logs.html
B. Business Logic Layer (Backend Services)
1. RSS Scraper Service (scrape_articles.py + rss.py)
– Polls external RSS feed URLs → fetches & normalizes new articles.
2. Summarization Service (ollama_sum.py)
– Takes plain text → calls local/remote Ollama LLM → returns summary.
3. Text-to-Speech Service (text2speech.py)
– Takes summary text → converts to audio files (e.g., MP3/WAV).
4. File Management Service (files.py)
– Writes/reads summaries to articles/; audio files to audio/; log entries via logs/.
5. Logging Service (logs.py)
– Centralized logger → writes structured logs under logs/.
C. Storage Layer
• Local File System
– articles/finance|science|tech|world_news
– audio/finance|science|tech|world_news
– logs/
D. External Systems
• RSS Feed Providers (finance, science, tech, world news sites)
• Ollama LLM (local Docker container or remote API)
• Optional TTS API (if using a cloud service)
Relationships and Data Flows
1. User (Web Browser) → HTTP Request → Flask Web Server
a. “Fetch Latest Summaries” → triggers backend scrape → summarization → TTS → updates storage.
b. “View Articles” → Flask reads articles/ files → renders articles.html.
c. “Play Audio” → browser streams file from audio/ directory.
d. “View Logs” → Flask reads logs/ → renders logs.html.
2. Scraper Service → HTTP Requests → External RSS Feeds → Article HTML/Text
3. Summarization Service → gRPC/HTTP → Ollama → Summary Text
4. TTS Service → API call or library → Audio File
5. File Management Service → Disk I/O → articles/, audio/, logs/
Architectural Patterns and Design Principles
– Layered Architecture (Presentation / Business Logic / Storage).
– Separation of Concerns: each module handles a single responsibility.
– Batch vs. On-Demand: scraping/summarization can be scheduled (main.py) or invoked via UI.
– File-based persistence (no RDBMS) for simplicity.
– Annotations for any scheduling/rate-limiting (if present).