# thysrael/horizon architecture

> Architecture diagram of the GitHub repository thysrael/horizon, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thysrael/horizon
- Repository: https://github.com/thysrael/horizon

## Overview

#### Purpose and Entry Points
Horizon is a Python CLI system that turns configured multi-source news into ranked, AI-enriched daily Markdown briefings, then publishes or delivers them. The primary runtime entry point is `src/main.py`, with CLI parsing in `src/_cli.py`; scheduled execution is supported by `scripts/daily-run.sh`. An interactive setup path produces user configuration via `src/setup/wizard.py`.

#### Configuration, Profiles, and Domain Model
Runtime behavior is driven by a data directory: configuration supplies AI provider/model, sources, processing settings, digest limits, and delivery options; persistent output/state lives under that directory. Example presets reside in `data/config.example.json` and `data/presets.json`. Profile definitions partition editorial behavior—matching, analysis, and enrichment prompts—for example `profiles/tech-news/profile.json`, `profiles/tech-news/analysis.md`, and `profiles/tech-news/enrichment.md`. Shared domain objects and pipeline contracts are centered in `src/models.py`.

#### Pipeline Orchestration
`src/orchestrator.py` coordinates the end-to-end control flow: load configuration/profiles, fetch recent source items concurrently, deduplicate, route/filter by profile and score, optionally balance categories, enrich selected items, summarize/localize, persist the briefing, and invoke outputs. `src/processing/profiles.py` resolves configured or AI-selected profiles; `src/processing/content.py` handles selection-oriented processing. This is the primary boundary between source acquisition, AI processing, and delivery.

#### Source Acquisition and Content Extraction
Scrapers implement a common source boundary in `src/scrapers/base.py`; concrete integrations include `src/scrapers/rss.py`, `src/scrapers/hackernews.py`, `src/scrapers/reddit.py`, `src/scrapers/github.py`, `src/scrapers/telegram.py`, `src/scrapers/twitter.py`, and `src/scrapers/openbb.py`. They call external feeds/APIs/services and may collect community discussions. RSS/article-body extraction is decoupled behind the extractor registry: `src/extractors/registry.py`, `src/extractors/base.py`, and optional `src/extractors/trafilatura.py`. URL validation/security is isolated in `src/url_security.py`.

#### AI Processing Layer
AI provider access is abstracted by `src/ai/client.py`, while editorial stages are separated into `src/ai/classifier.py` (profile/category matching and scoring), `src/ai/analyzer.py`, `src/ai/enricher.py`, and `src/ai/summarizer.py`. Prompt construction is versioned as code in `src/ai/prompting/analysis.py`, `src/ai/prompting/classification.py`, and `src/ai/prompting/enrichment.py`. Supported deployments include hosted model providers and local Ollama-compatible use, selected from configuration and environment-provided credentials.

#### Storage and Outputs
`src/storage/manager.py` owns local persistence of generated summaries and related state. The main artifact is Markdown for local use and GitHub Pages publication into `docs/`; the site assets/configuration are in `docs/_config.yml` and `docs/assets/`. Delivery adapters are separate side-effect boundaries: SMTP/IMAP subscription email in `src/services/email.py`, and chat/custom HTTP notifications in `src/services/webhook.py` and `src/services/webhook_cli.py`.

#### MCP and Deployment Boundaries
The optional MCP server exposes pipeline capabilities to compatible AI clients rather than requiring CLI execution: `src/mcp/server.py`, `src/mcp/service.py`, and `src/mcp/horizon_adapter.py`; `src/mcp/run_store.py` tracks MCP run state. Local execution uses Python/uv; containerized execution is defined by `Dockerfile` and `docker-compose.yml`, with optional dependency extras for extraction, OpenBB, and Twitter/Playwright. GitHub Pages documentation deployment is represented by `.github/workflows/deploy-docs.yml`.

Main entry points: [`src/orchestrator.py`](https://github.com/thysrael/horizon/blob/main/src/orchestrator.py) (Pipeline orchestrator), [`src/main.py`](https://github.com/thysrael/horizon/blob/main/src/main.py) (CLI runtime), [`src/scrapers/rss.py`](https://github.com/thysrael/horizon/blob/main/src/scrapers/rss.py) (Source integrations), [`src/processing/content.py`](https://github.com/thysrael/horizon/blob/main/src/processing/content.py) (Profile resolution & selection)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Entry points"]
  node_cli["CLI runtime<br/>Python CLI<br/>[main.py]"]
  node_cli_parser["CLI parser<br/>command parsing<br/>[_cli.py]"]
  node_scheduler["Daily scheduler<br/>shell entry point<br/>[daily-run.sh]"]
  node_setup["Setup wizard<br/>interactive setup<br/>[wizard.py]"]
end

subgraph group_pipeline["Briefing pipeline"]
  node_config["Configuration &amp; presets<br/>runtime settings"]
  node_profiles["Editorial profiles<br/>profile definitions<br/>[profile.json]"]
  node_orchestrator{{"Pipeline orchestrator<br/>workflow coordinator<br/>[orchestrator.py]"}}
  node_models["Domain contracts<br/>shared models<br/>[models.py]"]
  node_selection["Profile resolution &amp; selection<br/>content processing<br/>[content.py]"]
end

subgraph group_acquisition["Content acquisition"]
  node_scraper_boundary["Scraper boundary<br/>source interface<br/>[base.py]"]
  node_source_integrations["Source integrations<br/>[rss.py]"]
  node_extractors["Article extraction<br/>extractor registry<br/>[registry.py]"]
  node_url_security["URL security<br/>URL validation<br/>[url_security.py]"]
end

subgraph group_ai["AI editorial layer"]
  node_ai_client["AI provider client<br/>model provider abstraction<br/>[client.py]"]
  node_classifier["Classification &amp; scoring<br/>AI classifier<br/>[classifier.py]"]
  node_editorial_ai["Analysis, enrichment &amp; summary<br/>AI editorial stages<br/>[analyzer.py]"]
end

subgraph group_delivery["Storage &amp; delivery"]
  node_storage[("Briefing storage<br/>local persistence<br/>[manager.py]")]
  node_pages["GitHub Pages briefing<br/>Markdown publication<br/>[_config.yml]"]
  node_notifications["Email &amp; webhook delivery<br/>delivery adapters<br/>[email.py]"]
end

subgraph group_mcp["MCP interface"]
  node_mcp_server["MCP server<br/>AI client interface<br/>[server.py]"]
end

node_scheduler -->|"runs"| node_cli
node_cli -->|"parses commands"| node_cli_parser
node_setup -->|"creates"| node_config
node_cli -->|"starts briefing run"| node_orchestrator
node_config -->|"configures"| node_orchestrator
node_profiles -->|"defines editorial rules"| node_selection
node_models -.->|"pipeline contracts"| node_orchestrator
node_orchestrator -->|"fetches recent items"| node_scraper_boundary
node_scraper_boundary -->|"implemented by"| node_source_integrations
node_source_integrations -.->|"validates URLs"| node_url_security
node_source_integrations -->|"extracts article bodies"| node_extractors
node_orchestrator -->|"deduplicates and selects"| node_selection
node_selection -->|"classifies and scores"| node_classifier
node_classifier -->|"model calls"| node_ai_client
node_orchestrator -->|"enriches and summarizes"| node_editorial_ai
node_editorial_ai -->|"model calls"| node_ai_client
node_orchestrator -->|"persists briefing"| node_storage
node_storage -->|"publishes Markdown"| node_pages
node_orchestrator -->|"delivers briefing"| node_notifications
node_mcp_server -->|"invokes pipeline"| node_orchestrator

click node_cli "https://github.com/thysrael/horizon/blob/main/src/main.py"
click node_cli_parser "https://github.com/thysrael/horizon/blob/main/src/_cli.py"
click node_scheduler "https://github.com/thysrael/horizon/blob/main/scripts/daily-run.sh"
click node_setup "https://github.com/thysrael/horizon/blob/main/src/setup/wizard.py"
click node_config "https://github.com/thysrael/horizon/blob/main/data/config.example.json"
click node_profiles "https://github.com/thysrael/horizon/blob/main/profiles/tech-news/profile.json"
click node_orchestrator "https://github.com/thysrael/horizon/blob/main/src/orchestrator.py"
click node_models "https://github.com/thysrael/horizon/blob/main/src/models.py"
click node_selection "https://github.com/thysrael/horizon/blob/main/src/processing/content.py"
click node_scraper_boundary "https://github.com/thysrael/horizon/blob/main/src/scrapers/base.py"
click node_source_integrations "https://github.com/thysrael/horizon/blob/main/src/scrapers/rss.py"
click node_extractors "https://github.com/thysrael/horizon/blob/main/src/extractors/registry.py"
click node_url_security "https://github.com/thysrael/horizon/blob/main/src/url_security.py"
click node_ai_client "https://github.com/thysrael/horizon/blob/main/src/ai/client.py"
click node_classifier "https://github.com/thysrael/horizon/blob/main/src/ai/classifier.py"
click node_editorial_ai "https://github.com/thysrael/horizon/blob/main/src/ai/analyzer.py"
click node_storage "https://github.com/thysrael/horizon/blob/main/src/storage/manager.py"
click node_pages "https://github.com/thysrael/horizon/blob/main/docs/_config.yml"
click node_notifications "https://github.com/thysrael/horizon/blob/main/src/services/email.py"
click node_mcp_server "https://github.com/thysrael/horizon/blob/main/src/mcp/server.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_cli_parser,node_scheduler,node_setup toneBlue
class node_config,node_profiles,node_orchestrator,node_models,node_selection toneAmber
class node_scraper_boundary,node_source_integrations,node_extractors,node_url_security toneMint
class node_ai_client,node_classifier,node_editorial_ai toneRose
class node_storage,node_pages,node_notifications toneIndigo
class node_mcp_server toneTeal
```

## Components

### Entry points

- [CLI runtime](https://github.com/thysrael/horizon/blob/main/src/main.py): Python CLI · `src/main.py`
- [CLI parser](https://github.com/thysrael/horizon/blob/main/src/_cli.py): command parsing · `src/_cli.py`
- [Daily scheduler](https://github.com/thysrael/horizon/blob/main/scripts/daily-run.sh): shell entry point · `scripts/daily-run.sh`
- [Setup wizard](https://github.com/thysrael/horizon/blob/main/src/setup/wizard.py): interactive setup · Creates user configuration in the data directory. · `src/setup/wizard.py`

### Briefing pipeline

- [Configuration & presets](https://github.com/thysrael/horizon/blob/main/data/config.example.json): runtime settings · Selects sources, models, processing, limits, and delivery. · `data/config.example.json`
- [Editorial profiles](https://github.com/thysrael/horizon/blob/main/profiles/tech-news/profile.json): profile definitions · Defines matching, analysis, and enrichment behavior. · `profiles/tech-news/profile.json`
- [Pipeline orchestrator](https://github.com/thysrael/horizon/blob/main/src/orchestrator.py): workflow coordinator · Runs acquisition, selection, enrichment, persistence, and outputs. · `src/orchestrator.py`
- [Domain contracts](https://github.com/thysrael/horizon/blob/main/src/models.py): shared models · `src/models.py`
- [Profile resolution & selection](https://github.com/thysrael/horizon/blob/main/src/processing/content.py): content processing · Resolves profiles, filters, scores, deduplicates, and balances items. · `src/processing/content.py`

### Content acquisition

- [Scraper boundary](https://github.com/thysrael/horizon/blob/main/src/scrapers/base.py): source interface · Common interface for configured news sources. · `src/scrapers/base.py`
- [Source integrations](https://github.com/thysrael/horizon/blob/main/src/scrapers/rss.py): RSS, community & market sources · Fetches feeds, Hacker News, Reddit, GitHub, Telegram, Twitter, and OpenBB data. · `src/scrapers/rss.py`
- [Article extraction](https://github.com/thysrael/horizon/blob/main/src/extractors/registry.py): extractor registry · Decouples article-body extraction from source collection. · `src/extractors/registry.py`
- [URL security](https://github.com/thysrael/horizon/blob/main/src/url_security.py): URL validation · `src/url_security.py`

### AI editorial layer

- [AI provider client](https://github.com/thysrael/horizon/blob/main/src/ai/client.py): model provider abstraction · Uses configured hosted or Ollama-compatible models. · `src/ai/client.py`
- [Classification & scoring](https://github.com/thysrael/horizon/blob/main/src/ai/classifier.py): AI classifier · Matches profiles/categories and ranks items. · `src/ai/classifier.py`
- [Analysis, enrichment & summary](https://github.com/thysrael/horizon/blob/main/src/ai/analyzer.py): AI editorial stages · Analyzes, enriches, summarizes, and localizes selected content. · `src/ai/analyzer.py`

### Storage & delivery

- [Briefing storage](https://github.com/thysrael/horizon/blob/main/src/storage/manager.py): local persistence · Persists Markdown briefings and related state. · `src/storage/manager.py`
- [GitHub Pages briefing](https://github.com/thysrael/horizon/blob/main/docs/_config.yml): Markdown publication · Publishes briefing artifacts under docs/. · `docs/_config.yml`
- [Email & webhook delivery](https://github.com/thysrael/horizon/blob/main/src/services/email.py): delivery adapters · Sends subscriptions and chat/custom HTTP notifications. · `src/services/email.py`

### MCP interface

- [MCP server](https://github.com/thysrael/horizon/blob/main/src/mcp/server.py): AI client interface · Exposes pipeline capabilities and tracks MCP runs. · `src/mcp/server.py`

## Connections

- Daily scheduler → CLI runtime: runs
- CLI runtime → CLI parser: parses commands
- Setup wizard → Configuration & presets: creates
- CLI runtime → Pipeline orchestrator: starts briefing run
- Configuration & presets → Pipeline orchestrator: configures
- Editorial profiles → Profile resolution & selection: defines editorial rules
- Domain contracts → Pipeline orchestrator: pipeline contracts
- Pipeline orchestrator → Scraper boundary: fetches recent items
- Scraper boundary → Source integrations: implemented by
- Source integrations → URL security: validates URLs
- Source integrations → Article extraction: extracts article bodies
- Pipeline orchestrator → Profile resolution & selection: deduplicates and selects
- Profile resolution & selection → Classification & scoring: classifies and scores
- Classification & scoring → AI provider client: model calls
- Pipeline orchestrator → Analysis, enrichment & summary: enriches and summarizes
- Analysis, enrichment & summary → AI provider client: model calls
- Pipeline orchestrator → Briefing storage: persists briefing
- Briefing storage → GitHub Pages briefing: publishes Markdown
- Pipeline orchestrator → Email & webhook delivery: delivers briefing
- MCP server → Pipeline orchestrator: invokes pipeline