How nahimmora/news-auto-publisher-lavozriojana works
Purpose and execution surface
La Voz Riojana is a Windows-first Python automation system that ingests regional news, applies editorial validation/rewrite, generates social assets, and conditionally publishes to a Node CMS, Facebook, and Instagram. Operational state is local JSON, not a database. Primary entry points are cli.py, continuous supervision in run_24x7.py, batch execution in run_all.py, and section-specific runners such as main_locales.py.
Orchestration, safety, and runtime control
cli.py exposes operational commands: diagnostics, preflight, one-cycle execution, canary, queue cutover, backup, alerts, and reconciliation. Deployment modes constrain channel availability but cannot override kill switches. Result states and evidence are standardized for supervisor/automation use.
• cli.py
• run_24x7.py
• utils/deployment.py
• utils/preflight.py
• utils/operation_result.py
Ingestion and normalization
Section scrapers discover links and extract articles from Nueva Rioja and Tiempo Popular. A common runner coordinates sources; source contracts, URL normalization, deduplication, and filters form the boundary between unreliable external HTML and internal news candidates.
• scraping/runner.py
• scraping/base_nuevarioja.py
• scraping/base_tiempopopular.py
• utils/scraper_contract.py
• utils/news_dedup.py
Editorial processing and routing
Content is screened by policy and blocked-keyword rules, classified/prioritized, and may be rewritten or captioned through OpenAI-facing modules. Editorial routing can divert eligible items into a separate manual premium workflow; it is disabled by default to preserve automatic Instagram selection.
• openIA/rewrite_news.py
• openIA/caption_generator.py
• utils/editorial_policy.py
• utils/editorial_router.py
• pipeline/node_webapp/editorial.py
Media and rendering pipeline
The system obtains/processes images and creates cards, reels, and premium static assets. Python/Pillow-style rendering is the automatic default; a separate TypeScript/Remotion project renders premium studio artifacts. Video production depends on system ffmpeg/ffprobe and Python-installed yt-dlp.
• layout/image_generator.py
• openIA/reel_generator.py
• utils/video_renderer.py
• utils/remotion_renderer.py
• remotion/src/Root.tsx
Queues, durable state, and recovery
Local JSON-backed durable queues coordinate automatic social work, manual posts/videos, and premium submissions. They support events, throttling, dead-letter/recovery-oriented behavior, cutover archiving, and quarantine rather than a central database. Environment-configurable data/log/output paths enable isolated runs.
• utils/durable_queue.py
• utils/social_queue.py
• utils/premium_post_queue.py
• utils/queue_cutover.py
• utils/manual_video_queue.py
Publishing boundaries and external services
Web publication is isolated behind the Node webapp adapter; Meta clients manage Facebook Graph and Instagram calls, with token handling and stage-level results. Facebook has a pre-publication public-page/og:image verification boundary. R2-compatible object storage is used for media and reversible health checks. External publication requires enabled switches, authorization, capacity limits, and channel-specific validation.
• pipeline/publish_web.py
• pipeline/node_webapp/publisher.py
• meta/fb_client.py
• meta/ig_client.py
• utils/r2_storage.py
Operations and observability
The 24x7 profile runs under Windows PowerShell/task scheduling, guarded by supervisor diagnostics and idempotent process startup. Heartbeats, structured logging, alert detection/deduplication, and a durable alert outbox decouple monitoring from optional webhook delivery. CI runs on Windows and validates dry-run end-to-end contracts with external doubles.
• scripts/start_24x7_production.ps1
• utils/heartbeat.py
• utils/alerts.py
• utils/logging_setup.py
• .github/workflows/reliability.yml
Main entry points: durable_queue.py Durable Social Queues · deployment.py Deployment Guards · editorial_router.py Editorial Processing · r2_storage.py R2 Object Storage
Components 20
Runtime Control
- Operations CLI · Runs diagnostics, preflight, cycles, cutover, backup, alerts, and reconciliation.
cli.py - 24x7 Supervisor · Continuously supervises idempotent automation cycles.
run_24x7.py - Deployment Guards · Deployment modes constrain channels alongside kill switches.
utils/deployment.py - Preflight Checks
utils/preflight.py - Windows Scheduled Runtime · Starts the production 24x7 profile under Windows scheduling.
scripts/start_24x7_production.ps1
News Pipeline
- Scraper Runner
scraping/runner.py - Regional News Sources · Nueva Rioja and Tiempo Popular extraction boundary.
scraping/base_nuevarioja.py - Source Contracts · Normalizes URLs and applies source-level candidate rules.
utils/scraper_contract.py - Deduplication
utils/news_dedup.py
Editorial & Media
- Editorial Processing · Screens, prioritizes, rewrites, captions, and routes candidates.
utils/editorial_router.py - OpenAI Content Tools · Generates rewritten news and social captions.
openIA/rewrite_news.py - Automatic Media Renderer · Produces images, cards, and reels using Python tooling.
layout/image_generator.py - Premium Studio Renderer · Renders premium static artifacts in a separate TypeScript project.
utils/remotion_renderer.py
Durable State
- Durable Social Queues · Coordinates automatic, manual, video, and premium work with recovery behavior.
utils/durable_queue.py - Queue Cutover · Archives and quarantines queue work during cutover.
utils/queue_cutover.py
Publishing & Operations
- Web Publishing Adapter · Isolates publication through the Node webapp adapter.
pipeline/publish_web.py - Facebook Publisher · Verifies public page and og:image before publication.
meta/fb_client.py - Instagram Publisher
meta/ig_client.py - R2 Object Storage · Stores media and supports reversible health checks.
utils/r2_storage.py - Monitoring & Alerts · Provides heartbeats, structured logs, deduplicated alerts, and durable alert delivery.
utils/alerts.py
Connections 28
- Windows Scheduled Runtime → 24x7 Supervisor: starts
- Operations CLI → Preflight Checks: runs
- Operations CLI → Deployment Guards: enforces
- 24x7 Supervisor → Scraper Runner: executes cycles
- 24x7 Supervisor → Monitoring & Alerts: emits health
- Deployment Guards → Web Publishing Adapter: gates
- Deployment Guards → Facebook Publisher: gates
- Deployment Guards → Instagram Publisher: gates
- Scraper Runner → Regional News Sources: extracts
- Regional News Sources → Source Contracts: raw articles
- Source Contracts → Deduplication: normalized candidates
- Deduplication → Editorial Processing: news candidates
- Editorial Processing → OpenAI Content Tools: rewrite or caption
- OpenAI Content Tools → Automatic Media Renderer: approved content
- Editorial Processing → Premium Studio Renderer: premium route
- Automatic Media Renderer → R2 Object Storage: media assets
- Premium Studio Renderer → R2 Object Storage: premium assets
- Editorial Processing → Durable Social Queues: queues approved work
- Automatic Media Renderer → Durable Social Queues: attaches assets
- Durable Social Queues → Queue Cutover: archive or quarantine
- Durable Social Queues → Web Publishing Adapter: web jobs
- Durable Social Queues → Facebook Publisher: Facebook jobs
- Durable Social Queues → Instagram Publisher: Instagram jobs
- R2 Object Storage → Facebook Publisher: hosted media
- R2 Object Storage → Instagram Publisher: hosted media
- Web Publishing Adapter → Monitoring & Alerts: stage results
- Facebook Publisher → Monitoring & Alerts: stage results
- Instagram Publisher → Monitoring & Alerts: stage results