# touhi99/tldr-ai-news-crew architecture

> Architecture diagram of the GitHub repository touhi99/tldr-ai-news-crew, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/touhi99/tldr-ai-news-crew
- Repository: https://github.com/touhi99/tldr-ai-news-crew

## Overview

- User Interface (UI) Layer – Represented by main.py, which launches the Streamlit UI. This component gathers user input (such as date ranges) and displays the processed AI news data.
- Tools & Functionalities – Under the tools directory you have individual modules like:
- crawler.py for crawling TLDR news data.
- qa.py for processing Q&A functionalities.
- speech.py and voice.py for TTS and voice capabilities (with voice mode extended through Elevenlabs).
- vectordb.py for managing interactions with a vector database.
- Configuration & Environment – The config directory (with agents.yaml, config.json, and tasks.yaml) holds essential configuration details for agent behaviors and tasks. Also highlight the use of .env.example for API key management. Group these as part of the “Configuration Layer”.

##### Relationships and Data Flow
- The CrewAI Engine processes the input, using configuration from the config folder, and delegates subtasks to the respective tool modules. For example:
- The output then flows back from the agents and tool modules into UI (main.py) for display.

##### Architectural Patterns & Principles
- Separation of Concerns: The design clearly segregates the presentation layer (Streamlit UI), the business logic/agent orchestration (CrewAI Engine), and specialized tools (crawling, Q&A, speech, etc.).
- Configuration Management: A central configuration layer (with YAML and JSON files) that the core engine reads to adapt agent behavior.
- Modularity & Extensibility: The tools directory is modular, making it simple to add or replace functionalities (e.g., integrating a new TTS service).
- External Dependency Management: The use of .env and README instructions indicates proper management of API keys and sensitive information.

##### Relevant Technologies & Frameworks
- Streamlit for the UI.
- Python modules for agent orchestration and tool functionalities.
- External AI/LLM APIs (e.g., OpenAI, Elevenlabs) that are accessed via tools.
- Possibly a vector database integration as indicated in tools/vectordb.py.

- In the center, show the CrewAI Engine (crew.py and llms.py) which drives the logic.
- A configuration store symbol (with the config folder) linked to the engine to imply it provides runtime parameters.
- On the periphery, place external API services linked to modules that rely on them (like OpenAI and Elevenlabs).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% UI Layer
    UI["Streamlit UI"]:::ui

    %% CrewAI Engine Subgraph
    subgraph "CrewAI Engine"
        Crew["Crew (crew.py)"]:::engine
        LLM["LLMs (llms.py)"]:::engine
        Crew -->|"communicatesWith"| LLM
    end

    %% Configuration & Environment Subgraph
    subgraph "Configuration & Environment"
        Config["Configuration Files"]:::config
        Env[".env Example"]:::config
    end

    %% Tool Modules Subgraph
    subgraph "Tool Modules"
        Crawler["Crawler"]:::tool
        QA["Q&A"]:::tool
        Speech["Speech"]:::tool
        Voice["Voice"]:::tool
        VectorDB["VectorDB"]:::tool
    end

    %% External APIs Subgraph
    subgraph "External APIs"
        OpenAI["OpenAI API"]:::external
        Elevenlabs["Voice API (Elevenlabs)"]:::external
    end

    %% Processed Data Node
    Processed["Processed Data"]:::tool

    %% Connections from UI to CrewAI Engine
    UI -->|"triggers"| Crew
    UI -->|"triggers"| LLM

    %% CrewAI Engine calls configuration
    Crew -->|"readConfig"| Config
    Crew -->|"readConfig"| Env
    LLM -->|"readConfig"| Config
    LLM -->|"readConfig"| Env

    %% CrewAI Engine delegates tasks to Tool Modules
    Crew -->|"delegates"| Crawler
    Crew -->|"delegates"| QA
    Crew -->|"delegates"| Speech
    Crew -->|"delegates"| Voice
    Crew -->|"delegates"| VectorDB
    LLM -->|"delegates"| QA

    %% Tools interacting with External APIs
    LLM -->|"calls"| OpenAI
    Speech -->|"calls"| Elevenlabs
    Voice -->|"calls"| Elevenlabs

    %% Tools aggregate output to Processed Data
    Crawler -->|"returns"| Processed
    QA -->|"returns"| Processed
    Speech -->|"returns"| Processed
    Voice -->|"returns"| Processed
    VectorDB -->|"returns"| Processed

    %% Processed Data is sent back to UI
    Processed -->|"displays"| UI

    %% Styles
    classDef ui fill:#b3cde0,stroke:#03396c,stroke-width:2px;
    classDef engine fill:#fdd49e,stroke:#d95f02,stroke-width:2px;
    classDef tool fill:#c7e9c0,stroke:#31a354,stroke-width:2px;
    classDef config fill:#fdae6b,stroke:#e6550d,stroke-width:2px;
    classDef external fill:#fbb4ae,stroke:#b30000,stroke-width:2px;

    %% Click Events
    click UI "https://github.com/touhi99/tldr-ai-news-crew/blob/main/main.py"
    click Crew "https://github.com/touhi99/tldr-ai-news-crew/blob/main/crew.py"
    click LLM "https://github.com/touhi99/tldr-ai-news-crew/blob/main/llms.py"
    click Crawler "https://github.com/touhi99/tldr-ai-news-crew/blob/main/tools/crawler.py"
    click QA "https://github.com/touhi99/tldr-ai-news-crew/blob/main/tools/qa.py"
    click Speech "https://github.com/touhi99/tldr-ai-news-crew/blob/main/tools/speech.py"
    click Voice "https://github.com/touhi99/tldr-ai-news-crew/blob/main/tools/voice.py"
    click VectorDB "https://github.com/touhi99/tldr-ai-news-crew/blob/main/tools/vectordb.py"
    click Config "https://github.com/touhi99/tldr-ai-news-crew/tree/main/config"
    click Env "https://github.com/touhi99/tldr-ai-news-crew/blob/main/.env.example"
```
