How engineervix/zed-news works
Main Components and Their Responsibilities
• Core (Backend):
• Code located in the app/core/ directory.
• Responsible for fetching news (via libraries like requests, feedparser, beautifulsoup4).
• Summarization using multiple LLM backends (cohere, openai, together).
• Audio processing including text-to-speech conversion (via AWS Polly) and audio processing (ffmpeg).
• Database interactions with configurations and models (app/core/db).
• Additional functionality: social media integrations (e.g., make_video.sh, post.py) and utilities.
• Web (Frontend):
• Code located in app/web/.
• An 11ty project generating a static site.
• Contains templates, pages, CSS/JS assets, and assets such as images, favicons, and feed generation logic.
• Supporting Components and Infrastructure:
• Docker & Docker-compose: Containerization (Dockerfile, docker-compose.yml) ensuring portability.
• External Services:
• AWS S3 for hosting mp3 files.
• External news sources.
• AWS Polly for text-to-speech conversion.
• Summarization services via OpenAI, Cohere, and Together.
• Continuous integration via GitHub Workflows.
• Cron jobs (cron.sh) for automated deployment.
• Deployment environment: including requirements for a Unix machine, Python virtual environment, and environment variable management (.env).
Relationships and Data Flow
• How news flows from external Zambian sources into the Core module (news fetching).
• That the generated content (mp3 files, transcripts, website content) flows into the Web component.
• Docker's role: two primary containers may exist (one for the core tasks and one for potentially serving the web content, or at least managing environment consistency during development).
Architectural Patterns and Design Principles
• Modular Design: Separate “core” logic (Python and data processing) from the “web” static site generation.
• Layering: Separate the concerns of data fetching, processing (summarization and audio production), and delivery (static site production).
• Testing and Automation: Highlight the tests directory in app/tests, and include CI/CD pipeline (GitHub Actions) as part of the overall architecture.
• DevOps and Containerization: Emphasize Docker usage, cron jobs for scheduled tasks, and deployment automation via scripts and container orchestration.
Relevant Technologies and Their Locations
• Python (backend), Node.js with 11ty (frontend).
• Docker, Docker Compose, and associated configuration files.
• AWS Polly and S3 as external cloud services.
• GitHub Actions for CI/CD.
• Tools such as Poetry for dependency management in Python and npm for the web project.