# engineervix/zed-news architecture

> Architecture diagram of the GitHub repository engineervix/zed-news, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/engineervix/zed-news
- Repository: https://github.com/engineervix/zed-news

## Overview

##### Main Components and Their Responsibilities
- Core (Backend):
- Code located in the app/core/ directory.
- Responsible for fetching news (via libraries like requests, feedparser, beautifulsoup4).
- Summarization using multiple LLM backends (cohere, openai, together).
- Audio processing including text-to-speech conversion (via AWS Polly) and audio processing (ffmpeg).
- Database interactions with configurations and models (app/core/db).
- Additional functionality: social media integrations (e.g., make_video.sh, post.py) and utilities.
- Web (Frontend):
- Code located in app/web/.
- An 11ty project generating a static site.
- Contains templates, pages, CSS/JS assets, and assets such as images, favicons, and feed generation logic.
- Supporting Components and Infrastructure:
- Docker & Docker-compose: Containerization (Dockerfile, docker-compose.yml) ensuring portability.
- External Services:
- AWS S3 for hosting mp3 files.
- External news sources.
- AWS Polly for text-to-speech conversion.
- Summarization services via OpenAI, Cohere, and Together.
- Continuous integration via GitHub Workflows.
- Cron jobs (cron.sh) for automated deployment.
- Deployment environment: including requirements for a Unix machine, Python virtual environment, and environment variable management (.env).

##### Relationships and Data Flow
- How news flows from external Zambian sources into the Core module (news fetching).
- That the generated content (mp3 files, transcripts, website content) flows into the Web component.
- Docker's role: two primary containers may exist (one for the core tasks and one for potentially serving the web content, or at least managing environment consistency during development).

##### Architectural Patterns and Design Principles
- Modular Design: Separate “core” logic (Python and data processing) from the “web” static site generation.
- Layering: Separate the concerns of data fetching, processing (summarization and audio production), and delivery (static site production).
- Testing and Automation: Highlight the tests directory in app/tests, and include CI/CD pipeline (GitHub Actions) as part of the overall architecture.
- DevOps and Containerization: Emphasize Docker usage, cron jobs for scheduled tasks, and deployment automation via scripts and container orchestration.

##### Relevant Technologies and Their Locations
- Python (backend), Node.js with 11ty (frontend).
- Docker, Docker Compose, and associated configuration files.
- AWS Polly and S3 as external cloud services.
- GitHub Actions for CI/CD.
- Tools such as Poetry for dependency management in Python and npm for the web project.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Services
    extNews["External News Sources"]:::external
    awsPolly["AWS Polly (Text-to-Speech)"]:::external
    awsS3["AWS S3 (Media Hosting)"]:::external
    cohere["Cohere"]:::external
    openai["OpenAI"]:::external
    together["Together"]:::external

    %% DevOps / Automation
    docker["Docker & Containerization"]:::devops
    ciCd["CI/CD Pipeline (GitHub Actions)"]:::devops
    cron["Cron Jobs (Automated Deployment)"]:::devops

    %% Backend (Core)
    subgraph "Backend (Core)"
        core["Core (Backend)"]:::backend
        newsFetcher["News Fetcher"]:::backend
        summarizer["Summarization Engine"]:::backend
        podcast["Podcast & Audio Processing"]:::backend
        database["Database Interactions"]:::backend
        social["Social Media Integrations"]:::backend
    end

    %% Frontend (Web)
    subgraph "Frontend (Web)"
        web["Static Site Generator (11ty)"]:::frontend
    end

    %% Data Flow
    extNews -->|"fetches"| newsFetcher
    newsFetcher -->|"sends news"| summarizer
    summarizer -->|"uses"| cohere
    summarizer -->|"uses"| openai
    summarizer -->|"uses"| together
    summarizer -->|"processes summary"| podcast
    podcast -->|"generates audio"| awsPolly
    awsPolly -->|"stores audio"| awsS3
    podcast -->|"stores content"| database
    podcast -->|"triggers posts"| social
    podcast -->|"publishes content"| web

    %% DevOps Relations
    docker --> core
    docker --> web
    ciCd --> core
    ciCd --> web
    cron --> core

    %% Click Events
    click core "https://github.com/engineervix/zed-news/tree/main/app/core/"
    click newsFetcher "https://github.com/engineervix/zed-news/tree/main/app/core/news/"
    click summarizer "https://github.com/engineervix/zed-news/tree/main/app/core/summarization/"
    click podcast "https://github.com/engineervix/zed-news/tree/main/app/core/podcast/"
    click database "https://github.com/engineervix/zed-news/tree/main/app/core/db/"
    click social "https://github.com/engineervix/zed-news/tree/main/app/core/social/"
    click web "https://github.com/engineervix/zed-news/tree/main/app/web/"
    click docker "https://github.com/engineervix/zed-news/tree/main/Dockerfile"
    click ciCd "https://github.com/engineervix/zed-news/tree/main/.github/workflows/"
    click cron "https://github.com/engineervix/zed-news/blob/main/cron.sh"

    %% Style Classes
    classDef backend fill:#add8e6,stroke:#000,stroke-width:1px;
    classDef frontend fill:#90ee90,stroke:#000,stroke-width:1px;
    classDef external fill:#ffdab9,stroke:#000,stroke-width:1px;
    classDef devops fill:#dda0dd,stroke:#000,stroke-width:1px;
```
