# allienna/le-veilleur architecture

> Architecture diagram of the GitHub repository allienna/le-veilleur, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/allienna/le-veilleur
- Repository: https://github.com/allienna/le-veilleur

## Overview

Le Veilleur is a scheduled editorial pipeline that turns newsletter links into a published daily tech-watch article, illustration, and LinkedIn draft. The Minion orchestrates discrete ingestion, generation, and publishing stages; the commit is the publication record, while the Astro site presents committed content. Fiches and podcast production are additional pipeline capabilities. The README documents the end-to-end workflow; source excerpts and the index expose selected implementation relationships, so unverified internal calls are omitted.

Main entry points: [`minion/src/minion/orchestrator.py`](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/orchestrator.py) (Run orchestrator), [`minion/src/minion/steps/generation.py`](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/steps/generation.py) (Article generation), [`minion/src/minion/ingest/scraper.py`](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/ingest/scraper.py) (Link scraping), [`minion/src/minion/steps/ingestion.py`](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/steps/ingestion.py) (Source validation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ops["Scheduling and control"]
  node_minion["Run orchestrator<br/>[orchestrator.py]"]
  node_runstate[("Run and lock state<br/>[memory.py]")]
  node_logging["Structured logging<br/>[logging.py]"]
  node_killfn["Budget kill-switch<br/>[main.py]"]
end

subgraph group_pipeline["Editorial pipeline"]
  node_gmail["Newsletter ingestion<br/>[ingestion.py]"]
  node_scraper["Link scraping<br/>[scraper.py]"]
  node_inputgate["Source validation<br/>[ingestion.py]"]
  node_assemble["Context assembly<br/>[assemble.py]"]
  node_generate["Article generation<br/>[generation.py]"]
  node_outputgate["Article validation<br/>[validate.py]"]
  node_fiches["Source fiches<br/>[runner.py]"]
end

subgraph group_outputs["Publishing and media"]
  node_publish["Article and draft commit<br/>[publish.py]"]
  node_imagen["Illustration generation<br/>[imagen.py]"]
  node_github["Repository publishing<br/>[github.py]"]
  node_podcast["Podcast production<br/>[script.py]"]
  node_gcs[("Audio object storage<br/>[gcs.py]")]
end

subgraph group_site["Reader experience"]
  node_gitcontent["Committed site content<br/>[config.ts]"]
  node_astro["Astro site"]
  node_rss["RSS and podcast feed<br/>[rss.xml.ts]"]
end

node_scheduler(("Daily scheduler"))
node_claude{{"Claude CLI"}}
node_billing(("Billing budget events"))

node_scheduler -->|"starts daily"| node_minion
node_minion -->|"records run"| node_runstate
node_minion -->|"emits events"| node_logging
node_minion -->|"dispatches steps"| node_gmail
node_minion -.->|"dispatches steps"| node_scraper
node_minion -.->|"dispatches steps"| node_inputgate
node_minion -.->|"dispatches steps"| node_assemble
node_minion -.->|"dispatches steps"| node_generate
node_minion -.->|"dispatches steps"| node_outputgate
node_minion -.->|"dispatches steps"| node_publish
node_minion -.->|"dispatches steps"| node_fiches
node_minion -.->|"dispatches steps"| node_podcast
node_gmail -->|"passes URLs"| node_scraper
node_scraper -->|"provides sources"| node_inputgate
node_inputgate -->|"passes accepted sources"| node_assemble
node_assemble -->|"builds context"| node_generate
node_generate -->|"invokes generation"| node_claude
node_generate -->|"validates article"| node_outputgate
node_minion -->|"generates illustration"| node_imagen
node_minion -->|"commits outputs"| node_github
node_publish -->|"writes article"| node_gitcontent
node_github -->|"commits content"| node_gitcontent
node_gitcontent -->|"renders content"| node_astro
node_podcast -.->|"stores audio"| node_gcs
node_gcs -->|"serves episode links"| node_rss
node_billing -->|"sends budget event"| node_killfn
node_killfn -->|"pauses job"| node_scheduler

click node_minion "https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/orchestrator.py"
click node_runstate "https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/store/memory.py"
click node_logging "https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/logging.py"
click node_gmail "https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/steps/ingestion.py"
click node_scraper "https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/ingest/scraper.py"
click node_inputgate "https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/steps/ingestion.py"
click node_assemble "https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/generate/assemble.py"
click node_generate "https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/steps/generation.py"
click node_outputgate "https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/generate/validate.py"
click node_fiches "https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/fiches/runner.py"
click node_publish "https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/steps/publish.py"
click node_imagen "https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/publish/imagen.py"
click node_github "https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/publish/github.py"
click node_podcast "https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/podcast/script.py"
click node_gcs "https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/podcast/gcs.py"
click node_gitcontent "https://github.com/allienna/le-veilleur/blob/main/site/src/content/config.ts"
click node_astro "https://github.com/allienna/le-veilleur/tree/main/site"
click node_rss "https://github.com/allienna/le-veilleur/blob/main/site/src/pages/rss.xml.ts"
click node_killfn "https://github.com/allienna/le-veilleur/blob/main/functions/budget-killswitch/main.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_minion,node_runstate,node_logging,node_killfn toneBlue
class node_gmail,node_scraper,node_inputgate,node_assemble,node_generate,node_outputgate,node_fiches toneAmber
class node_publish,node_imagen,node_github,node_podcast,node_gcs toneMint
class node_gitcontent,node_astro,node_rss,node_scheduler toneRose
class node_claude,node_billing toneIndigo
```

## Components

### Scheduling and control

- [Run orchestrator](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/orchestrator.py): `minion/src/minion/orchestrator.py`
- [Run and lock state](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/store/memory.py): `minion/src/minion/store/memory.py`
- [Structured logging](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/logging.py): `minion/src/minion/logging.py`
- [Budget kill-switch](https://github.com/allienna/le-veilleur/blob/main/functions/budget-killswitch/main.py): `functions/budget-killswitch/main.py`

### Editorial pipeline

- [Newsletter ingestion](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/steps/ingestion.py): `minion/src/minion/steps/ingestion.py`
- [Link scraping](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/ingest/scraper.py): `minion/src/minion/ingest/scraper.py`
- [Source validation](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/steps/ingestion.py): `minion/src/minion/steps/ingestion.py`
- [Context assembly](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/generate/assemble.py): `minion/src/minion/generate/assemble.py`
- [Article generation](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/steps/generation.py): `minion/src/minion/steps/generation.py`
- [Article validation](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/generate/validate.py): `minion/src/minion/generate/validate.py`
- [Source fiches](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/fiches/runner.py): `minion/src/minion/fiches/runner.py`

### Publishing and media

- [Article and draft commit](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/steps/publish.py): `minion/src/minion/steps/publish.py`
- [Illustration generation](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/publish/imagen.py): `minion/src/minion/publish/imagen.py`
- [Repository publishing](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/publish/github.py): `minion/src/minion/publish/github.py`
- [Podcast production](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/podcast/script.py): `minion/src/minion/podcast/script.py`
- [Audio object storage](https://github.com/allienna/le-veilleur/blob/main/minion/src/minion/podcast/gcs.py): `minion/src/minion/podcast/gcs.py`

### Reader experience

- [Committed site content](https://github.com/allienna/le-veilleur/blob/main/site/src/content/config.ts): `site/src/content/config.ts`
- [Astro site](https://github.com/allienna/le-veilleur/tree/main/site): `site`
- [RSS and podcast feed](https://github.com/allienna/le-veilleur/blob/main/site/src/pages/rss.xml.ts): `site/src/pages/rss.xml.ts`

### Other components

- **Daily scheduler**
- **Claude CLI**
- **Billing budget events**

## Connections

- Daily scheduler → Run orchestrator: starts daily (evidence: `README.md`)
- Run orchestrator → Run and lock state: records run (evidence: `minion/src/minion/orchestrator.py`)
- Run orchestrator → Structured logging: emits events (evidence: `minion/src/minion/orchestrator.py`)
- Run orchestrator → Newsletter ingestion: dispatches steps (evidence: `minion/src/minion/orchestrator.py`)
- Run orchestrator → Link scraping: dispatches steps
- Run orchestrator → Source validation: dispatches steps
- Run orchestrator → Context assembly: dispatches steps
- Run orchestrator → Article generation: dispatches steps
- Run orchestrator → Article validation: dispatches steps
- Run orchestrator → Article and draft commit: dispatches steps
- Run orchestrator → Source fiches: dispatches steps
- Run orchestrator → Podcast production: dispatches steps
- Newsletter ingestion → Link scraping: passes URLs (evidence: `minion/src/minion/steps/ingestion.py`)
- Link scraping → Source validation: provides sources (evidence: `minion/src/minion/steps/ingestion.py`)
- Source validation → Context assembly: passes accepted sources (evidence: `README.md`)
- Context assembly → Article generation: builds context (evidence: `minion/src/minion/steps/generation.py`)
- Article generation → Claude CLI: invokes generation (evidence: `minion/src/minion/steps/generation.py`)
- Article generation → Article validation: validates article (evidence: `minion/src/minion/steps/generation.py`)
- Run orchestrator → Illustration generation: generates illustration (evidence: `README.md`)
- Run orchestrator → Repository publishing: commits outputs (evidence: `README.md`)
- Article and draft commit → Committed site content: writes article (evidence: `README.md`)
- Repository publishing → Committed site content: commits content (evidence: `README.md`)
- Committed site content → Astro site: renders content (evidence: `README.md`)
- Podcast production → Audio object storage: stores audio
- Audio object storage → RSS and podcast feed: serves episode links
- Billing budget events → Budget kill-switch: sends budget event (evidence: `functions/budget-killswitch/main.py`)
- Budget kill-switch → Daily scheduler: pauses job (evidence: `functions/budget-killswitch/main.py`)
