How l34n/ai-researcher works
AI Researcher is a self-hosted AI news and trends dashboard. Users trigger or schedule a refresh, which collects items from configured sources, enriches and clusters them into stories, judges readiness, and produces deep-research pages and a daily brief for dashboard browsing. The graph preserves those distinct stages plus search, saved items, and source/run status. Pipeline and web wiring are based on sampled excerpts and README documentation; unsampled module internals and uncertain links are omitted.
Main entry points: pipeline.py Refresh Pipeline · db.py SQLite Store · app.py Web Dashboard · chat.py Chat Routing
Components 23
Collection
- CLI
src/ai_researcher/cli.py - Refresh Pipeline
src/ai_researcher/pipeline.py - Source Connectors
src/ai_researcher/connectors - Article Fetching
src/ai_researcher/connectors/article.py
Story Analysis
- Item Enrichment
src/ai_researcher/enrich - Chat Routing
src/ai_researcher/enrich/chat.py - Embeddings
src/ai_researcher/enrich/embed.py - Relevance Filter
src/ai_researcher/enrich/relevance.py - Story Clustering
src/ai_researcher/trends/cluster.py - Trend Ranking
src/ai_researcher/trends/score.py - Freshness Checks
src/ai_researcher/trends/freshness.py - Readiness Judgment
src/ai_researcher/enrich/judge.py
Research Outputs
- Deep Research
src/ai_researcher/research/wiki.py - Daily Brief
src/ai_researcher/trends/brief.py
Dashboard
- Web Dashboard
src/ai_researcher/web/app.py - Dashboard Queries
src/ai_researcher/web/queries.py - Dashboard Client
src/ai_researcher/web/static/app.js
Runtime State
- SQLite Store
src/ai_researcher/db.py - Run Progress
src/ai_researcher/progress.py
Other components
- Researcher
- News Sources
- Ollama
- Cloud Models
Connections 35
- Researcher → CLI: runs commands
- CLI → Refresh Pipeline: starts refresh
- Researcher → Web Dashboard: browses dashboard
- Dashboard Client → Web Dashboard: requests actions
- Web Dashboard → Refresh Pipeline: starts refresh
- Refresh Pipeline → Source Connectors: fetches sources
- News Sources → Source Connectors: provides feeds
- Refresh Pipeline → Article Fetching: fetches article text
- Article Fetching → News Sources: requests pages
- Refresh Pipeline → SQLite Store: stores items
- Refresh Pipeline → Item Enrichment: enriches items
- Item Enrichment → Chat Routing: requests model text
- Item Enrichment → Embeddings: embeds items
- Chat Routing → Ollama: routes locally
- Chat Routing → Cloud Models: routes optionally
- Refresh Pipeline → Relevance Filter: filters items
- Refresh Pipeline → Freshness Checks: checks freshness
- Refresh Pipeline → Story Clustering: builds stories
- Refresh Pipeline → Trend Ranking: ranks stories
- Refresh Pipeline → Readiness Judgment: judges readiness
- Refresh Pipeline → Deep Research: researches ready stories
- Deep Research → Chat Routing: generates pages
- Refresh Pipeline → Daily Brief: generates brief
- Daily Brief → Chat Routing: generates summary
- Item Enrichment → SQLite Store: writes enrichment
- Story Clustering → SQLite Store: writes clusters
- Trend Ranking → SQLite Store: stores scores
- Readiness Judgment → SQLite Store: stores judgments
- Deep Research → SQLite Store: stores research
- Daily Brief → SQLite Store: stores brief
- Web Dashboard → Dashboard Queries: loads dashboard data
- Dashboard Queries → SQLite Store: reads records
- Web Dashboard → SQLite Store: updates saved items
- Refresh Pipeline → Run Progress: reports progress
- Web Dashboard → Run Progress: serves run status