# rahul-gaikwad-007/youtube-trend-analysis architecture

> Architecture diagram of the GitHub repository rahul-gaikwad-007/youtube-trend-analysis, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rahul-gaikwad-007/youtube-trend-analysis
- Repository: https://github.com/rahul-gaikwad-007/youtube-trend-analysis

## Overview

##### Main Components
- Web Interface (Frontend): Represented by app.py, this is the Streamlit-based UI that users interact with.
- YouTube Data Scraper: Indicated by brightdata_scrapper.py, this component uses the BrightData API to scrape video data from YouTube.
- AI Analysis Engine: Though no dedicated module file is provided for CrewAI, the README explains that CrewAI is used to analyze video transcripts and generate summaries. This should be shown as a separate processing component (you might call it the "CrewAI Analysis Module").
- Configuration & Environment: The config.yaml (along with .env instructions in the README) represents where configuration and secrets (e.g., the BrightData API key) are stored. Treat these as supporting components.
- External Services:
- BrightData: The external API service used for scraping YouTube videos.
- CrewAI (and its toolset): The external AI-based analysis service.
- Ollama (Llama 3.2 model): Although not directly in the file tree, the README instructions for installing and pulling the model indicate that it plays a role in the AI analysis process.

##### Relationships and Data Flow
- The user accesses the application via the Streamlit web interface (app.py).
- The web interface then calls the BrightData scraper (brightdata_scrapper.py) to fetch YouTube video data.
- The scraped data (including video transcripts) is then forwarded to the CrewAI Analysis Module, which may also interact with the Ollama-based model.
- Results (e.g., trend summaries or insights) are collected and rendered back to the user via the web interface.
- Configuration details (from config.yaml and .env) are used to securely manage API keys and other settings across the modules.

##### Architectural Patterns and Design Principles
- There is a clear separation of concerns:
- Frontend: Handles user interactions and presentation.
- Backend: Contains two main functions, data scraping and AI analysis.

##### Relevant Technologies and Frameworks
- Python 3.11+: The base programming language.
- Streamlit: For building the web interface.
- BrightData API: For accessing YouTube video data.
- CrewAI and crewai-tools: For transcript analysis and summary generation.
- Ollama: For deploying the Llama 3.2 model, which may underpin parts of the AI analysis process.
- YAML and environment management: config.yaml and .env files for setup and configuration.

- Each component should be clearly labeled (e.g., “Streamlit Frontend (app.py)”, “BrightData Scraper (brightdata_scrapper.py)”, “CrewAI Analysis Module”, “Config & Environment”).
- Data flow from user input to scraping and then to analysis.
- Interaction with external services (BrightData API, CrewAI, and Ollama).
- External services (e.g., rounded rectangles or differently shaded components).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Nodes
    U("User"):::user
    FE("Streamlit Web Interface (app.py)"):::frontend
    BS("BrightData Scraper (brightdata_scrapper.py)"):::scraper
    CA("CrewAI Analysis Module"):::analysis
    CE("Configuration & Environment (config.yaml)"):::config
    BAPI("BrightData API"):::external
    CAAPI("CrewAI Service"):::external
    OLL("Ollama Llama Model"):::external

    %% Internal Data Flow
    U -->|"user_interaction"| FE
    FE -->|"fetch_data"| BS
    BS -->|"retrieve_videos"| BAPI
    BS -->|"send_scraped_data"| CA
    CA -->|"analyze_transcript"| CAAPI
    CA -->|"invoke_llama"| OLL
    CA -->|"return_insights"| FE

    %% Configuration Dependency (dashed lines)
    CE -.-> FE
    CE -.-> BS
    CE -.-> CA

    %% Click Events
    click FE "https://github.com/rahul-gaikwad-007/youtube-trend-analysis/blob/main/app.py"
    click BS "https://github.com/rahul-gaikwad-007/youtube-trend-analysis/blob/main/brightdata_scrapper.py"
    click CE "https://github.com/rahul-gaikwad-007/youtube-trend-analysis/blob/main/config.yaml"

    %% Styles
    classDef user fill:#ddf,stroke:#333,stroke-width:2px;
    classDef frontend fill:#f9f,stroke:#333,stroke-width:2px;
    classDef scraper fill:#bbf,stroke:#333,stroke-width:2px;
    classDef analysis fill:#bfb,stroke:#333,stroke-width:2px;
    classDef config fill:#ffb,stroke:#333,stroke-width:2px;
    classDef external fill:#faa,stroke:#333,stroke-width:2px;
```
