# spasoye-json/winnow architecture

> Architecture diagram of the GitHub repository spasoye-json/winnow, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/spasoye-json/winnow
- Repository: https://github.com/spasoye-json/winnow

## Overview

#### Purpose and entry points
Winnow is a Python-based YouTube curator: it ingests channel/feed content, obtains transcripts, scores videos, and presents a review/calibration interface. Primary executable entry points are the package launcher and CLI in `src/winnow/__main__.py` and `src/winnow/cli.py`; the web application is centered in `src/winnow/web.py`. A system-service deployment artifact exists at `packaging/winnow.service`.

#### Application runtime and persistence
The core runtime is a single-process Python service, consistent with `docs/adr/0001-single-process-due-check-runtime.md`. Database access and schema/lifecycle concerns are concentrated in `src/winnow/db.py`; application authentication is isolated in `src/winnow/auth.py`. The system’s durable boundary is therefore the database rather than an external job queue or distributed scheduler.

#### Feed ingestion and YouTube boundary
Content acquisition is split between feed discovery, ingestion orchestration, and YouTube-specific API behavior:
- `src/winnow/feed.py`
- `src/winnow/ingest.py`
- `src/winnow/youtube.py`

Ingestion converts upstream channel/video information into locally persisted candidates. `youtube.py` is the external-service adapter boundary, keeping provider interaction separate from downstream scoring and presentation.

#### Transcript acquisition pipeline
Transcript handling is a distinct enrichment stage after video ingestion. `src/winnow/transcript.py` manages transcript retrieval/processing, while `src/winnow/local_transcript.py` provides a local transcript source or fallback. Research documents identify transcript fetching and truncation as material constraints: `docs/research/transcript-fetching.md` and `docs/research/transcript-truncation-strategy.md`.

#### Scoring, verdicts, and calibration
`src/winnow/scoring.py` turns available video/transcript context into a relevance verdict or score. The scoring design is intentionally constrained to a single Flash Lite model according to `docs/adr/0003-flash-lite-single-model-scoring.md`, making the model provider a key external dependency and rate-limit boundary (`docs/research/gemini-free-tier-rate-limits.md`). Scores are final once written, rather than continuously recalibrated, per `docs/adr/0002-organic-calibration-scores-final-once-written.md`. Calibration UI and verdict rendering are represented by `src/winnow/templates/calibration.html` and `src/winnow/templates/_verdict.html`.

#### Scheduling, synchronization, and control flow
`src/winnow/scheduler.py` determines when due checks run; `src/winnow/sync.py` coordinates recurring refresh/synchronization work. The principal flow is: scheduler/CLI triggers sync → feeds and YouTube metadata are ingested → transcripts are acquired → scoring persists a verdict → web views expose queue, feed, details, and calibration. This separation permits manually initiated CLI work and service-driven scheduled work to use the same domain pipeline.

#### Web presentation and operational automation
The server-rendered web layer uses templates under `src/winnow/templates/`, notably `feed.html`, `queue.html`, `detail.html`, and `settings.html`, with static styling in `src/winnow/static/custom.css`. It is a presentation boundary over the persisted ingestion/scoring state, not the core processing engine.

Repository automation is separately implemented as TypeScript Sandcastle agent workflows, invoked by GitHub Actions. Workflow entry implementations include `.sandcastle/agent-workflows/explore/explore.ts`, `.sandcastle/agent-workflows/implement/implement.ts`, and `.sandcastle/agent-workflows/review/review.ts`; CI/agent triggers live in `.github/workflows/ci.yml` and related agent workflow files. These support repository maintenance rather than the Winnow runtime.

Main entry points: [`src/winnow/transcript.py`](https://github.com/spasoye-json/winnow/blob/master/src/winnow/transcript.py) (Transcript enrichment), [`src/winnow/db.py`](https://github.com/spasoye-json/winnow/blob/master/src/winnow/db.py) (Application database), [`src/winnow/web.py`](https://github.com/spasoye-json/winnow/blob/master/src/winnow/web.py) (Web application), [`src/winnow/ingest.py`](https://github.com/spasoye-json/winnow/blob/master/src/winnow/ingest.py) (Video ingestion)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Winnow runtime"]
  node_launcher["Package launcher<br/>Python entry point<br/>[__main__.py]"]
  node_cli["CLI<br/>command interface<br/>[cli.py]"]
  node_scheduler["Due scheduler<br/>[scheduler.py]"]
  node_sync["Sync coordinator<br/>orchestrator<br/>[sync.py]"]
  node_database[("Application database<br/>durable store<br/>[db.py]")]
  node_auth["Authentication<br/>auth boundary<br/>[auth.py]"]
  node_service["System service<br/>deployment artifact<br/>[winnow.service]"]
end

subgraph group_pipeline["Content pipeline"]
  node_feed["Feed discovery<br/>ingestion stage<br/>[feed.py]"]
  node_ingest["Video ingestion<br/>ingestion stage<br/>[ingest.py]"]
  node_youtube{{"YouTube adapter<br/>external API adapter<br/>[youtube.py]"}}
  node_transcript["Transcript enrichment<br/>enrichment stage<br/>[transcript.py]"]
  node_local_transcript["Local transcripts<br/>transcript fallback"]
  node_scoring["Verdict scoring<br/>scoring stage<br/>[scoring.py]"]
  node_gemini{{"Flash Lite model<br/>model provider"}}
end

subgraph group_presentation["Presentation"]
  node_web["Web application<br/>server-rendered web layer<br/>[web.py]"]
  node_views["Review templates<br/>HTML templates"]
end

subgraph group_automation["Repository automation"]
  node_agent_workflows["Sandcastle workflows<br/>TypeScript agents<br/>[explore.ts]"]
  node_github_actions["GitHub Actions<br/>CI trigger<br/>[ci.yml]"]
end

node_launcher -->|"launches"| node_cli
node_cli -->|"triggers"| node_sync
node_scheduler -->|"triggers due work"| node_sync
node_service -->|"runs"| node_scheduler
node_sync -->|"refreshes"| node_feed
node_feed -->|"discovers via"| node_youtube
node_youtube -->|"supplies metadata"| node_ingest
node_ingest -->|"persists candidates"| node_database
node_ingest -->|"enriches videos"| node_transcript
node_transcript -.->|"uses fallback"| node_local_transcript
node_transcript -->|"persists transcripts"| node_database
node_transcript -->|"provides context"| node_scoring
node_scoring -->|"requests score"| node_gemini
node_scoring -->|"writes final verdict"| node_database
node_web -->|"reads review state"| node_database
node_web -->|"authenticates"| node_auth
node_web -->|"renders"| node_views
node_github_actions -->|"invokes"| node_agent_workflows

click node_launcher "https://github.com/spasoye-json/winnow/blob/master/src/winnow/__main__.py"
click node_cli "https://github.com/spasoye-json/winnow/blob/master/src/winnow/cli.py"
click node_scheduler "https://github.com/spasoye-json/winnow/blob/master/src/winnow/scheduler.py"
click node_sync "https://github.com/spasoye-json/winnow/blob/master/src/winnow/sync.py"
click node_database "https://github.com/spasoye-json/winnow/blob/master/src/winnow/db.py"
click node_auth "https://github.com/spasoye-json/winnow/blob/master/src/winnow/auth.py"
click node_feed "https://github.com/spasoye-json/winnow/blob/master/src/winnow/feed.py"
click node_ingest "https://github.com/spasoye-json/winnow/blob/master/src/winnow/ingest.py"
click node_youtube "https://github.com/spasoye-json/winnow/blob/master/src/winnow/youtube.py"
click node_transcript "https://github.com/spasoye-json/winnow/blob/master/src/winnow/transcript.py"
click node_local_transcript "https://github.com/spasoye-json/winnow/blob/master/src/winnow/local_transcript.py"
click node_scoring "https://github.com/spasoye-json/winnow/blob/master/src/winnow/scoring.py"
click node_web "https://github.com/spasoye-json/winnow/blob/master/src/winnow/web.py"
click node_service "https://github.com/spasoye-json/winnow/blob/master/packaging/winnow.service"
click node_agent_workflows "https://github.com/spasoye-json/winnow/blob/master/.sandcastle/agent-workflows/explore/explore.ts"
click node_github_actions "https://github.com/spasoye-json/winnow/blob/master/.github/workflows/ci.yml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_launcher,node_cli,node_scheduler,node_sync,node_database,node_auth,node_service toneBlue
class node_feed,node_ingest,node_youtube,node_transcript,node_local_transcript,node_scoring,node_gemini toneAmber
class node_web,node_views toneMint
class node_agent_workflows,node_github_actions toneRose
```

## Components

### Winnow runtime

Single-process Python service.

- [Package launcher](https://github.com/spasoye-json/winnow/blob/master/src/winnow/__main__.py): Python entry point · `src/winnow/__main__.py`
- [CLI](https://github.com/spasoye-json/winnow/blob/master/src/winnow/cli.py): command interface · Manual trigger for the shared pipeline. · `src/winnow/cli.py`
- [Due scheduler](https://github.com/spasoye-json/winnow/blob/master/src/winnow/scheduler.py): scheduler · `src/winnow/scheduler.py`
- [Sync coordinator](https://github.com/spasoye-json/winnow/blob/master/src/winnow/sync.py): orchestrator · Coordinates recurring refresh work. · `src/winnow/sync.py`
- [Application database](https://github.com/spasoye-json/winnow/blob/master/src/winnow/db.py): durable store · Schema and lifecycle are managed locally. · `src/winnow/db.py`
- [Authentication](https://github.com/spasoye-json/winnow/blob/master/src/winnow/auth.py): auth boundary · `src/winnow/auth.py`
- [System service](https://github.com/spasoye-json/winnow/blob/master/packaging/winnow.service): deployment artifact · `packaging/winnow.service`

### Content pipeline

- [Feed discovery](https://github.com/spasoye-json/winnow/blob/master/src/winnow/feed.py): ingestion stage · `src/winnow/feed.py`
- [Video ingestion](https://github.com/spasoye-json/winnow/blob/master/src/winnow/ingest.py): ingestion stage · Persists discovered channel and video candidates. · `src/winnow/ingest.py`
- [YouTube adapter](https://github.com/spasoye-json/winnow/blob/master/src/winnow/youtube.py): external API adapter · `src/winnow/youtube.py`
- [Transcript enrichment](https://github.com/spasoye-json/winnow/blob/master/src/winnow/transcript.py): enrichment stage · `src/winnow/transcript.py`
- [Local transcripts](https://github.com/spasoye-json/winnow/blob/master/src/winnow/local_transcript.py): transcript fallback · `src/winnow/local_transcript.py`
- [Verdict scoring](https://github.com/spasoye-json/winnow/blob/master/src/winnow/scoring.py): scoring stage · Writes final relevance scores and verdicts. · `src/winnow/scoring.py`
- **Flash Lite model**: model provider · External scoring and rate-limit boundary.

### Presentation

- [Web application](https://github.com/spasoye-json/winnow/blob/master/src/winnow/web.py): server-rendered web layer · Reads persisted state for review and calibration. · `src/winnow/web.py`
- **Review templates**: HTML templates · Feed, queue, detail, settings, and calibration views.

### Repository automation

Maintenance workflows outside the application runtime.

- [Sandcastle workflows](https://github.com/spasoye-json/winnow/blob/master/.sandcastle/agent-workflows/explore/explore.ts): TypeScript agents · Explore, implement, and review maintenance workflows. · `.sandcastle/agent-workflows/explore/explore.ts`
- [GitHub Actions](https://github.com/spasoye-json/winnow/blob/master/.github/workflows/ci.yml): CI trigger · `.github/workflows/ci.yml`

## Connections

- Package launcher → CLI: launches
- CLI → Sync coordinator: triggers
- Due scheduler → Sync coordinator: triggers due work
- System service → Due scheduler: runs
- Sync coordinator → Feed discovery: refreshes
- Feed discovery → YouTube adapter: discovers via
- YouTube adapter → Video ingestion: supplies metadata
- Video ingestion → Application database: persists candidates
- Video ingestion → Transcript enrichment: enriches videos
- Transcript enrichment → Local transcripts: uses fallback
- Transcript enrichment → Application database: persists transcripts
- Transcript enrichment → Verdict scoring: provides context
- Verdict scoring → Flash Lite model: requests score
- Verdict scoring → Application database: writes final verdict
- Web application → Application database: reads review state
- Web application → Authentication: authenticates
- Web application → Review templates: renders
- GitHub Actions → Sandcastle workflows: invokes