# jatin-mehra119/ai-powered-youtube-video-summarizer-and-fact-checker architecture

> Architecture diagram of the GitHub repository jatin-mehra119/ai-powered-youtube-video-summarizer-and-fact-checker, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jatin-mehra119/ai-powered-youtube-video-summarizer-and-fact-checker
- Repository: https://github.com/jatin-mehra119/ai-powered-youtube-video-summarizer-and-fact-checker

## Overview

##### Project type and purpose
- This is a full-stack AI-powered web application that extracts YouTube video captions, builds searchable embeddings, provides summarization and fact-checking, and exposes both a Streamlit UI and a FastAPI backend.

##### File structure
- At the root you have:
- Docker-related files (.dockerignore, Dockerfile)
- The FastAPI app entrypoint (main.py) and its docs (fastapi_APP_info.md)
- A Streamlit UI under src/UI/app.py
- A caption-capture module under src/CC_capture
- A “Database” folder with FAISS index logic
- A pipelines folder with embedding, search, and fact_checker logic
- Requirements, license, README, and a sample captions.csv
- Patterns:
- Separation of concerns: UI (Streamlit), API (FastAPI), data pipelines, data storage (CSV + FAISS), external integrations (YouTube via yt-dlp, Groq API, Crawl4AI).

##### Extract key architectural notes from the README
- Users enter a YouTube URL in the UI
- The system extracts, cleans, and stores captions (CSV)
- Captions → sentence embeddings → FAISS index
- UI search queries hit the FAISS index for nearest neighbors
- Summaries and fact-checks use an LLM via GroqAPI + external crawl (Crawl4AI)
- Deployed as a container on a GCP VM

##### Main components
a. Frontend: Streamlit App
b. Backend: FastAPI App
c. Caption Capture Service (yt-dlp + load_cc)
d. Embedding Pipeline (SentenceTransformers)
e. FAISS Index (search)
f. Fact-Checking Pipeline (Groq API + Crawl4AI)
g. Data Storage: CSV on disk + FAISS index file
h. External Services: YouTube, Groq API, Crawl4AI
i. Orchestration / Hosting: Docker container(s) on GCP VM

##### Relationships and interactions
- User → Streamlit UI: inputs URL or search query
- UI → FastAPI (REST calls) for:
- Fetch & store captions
- Build embeddings & FAISS index
- Search captions
- Summarize + fact-check
- FastAPI → CC_capture to fetch captions from YouTube
- FastAPI → pipelines/embedding to generate embeddings
- FastAPI → Database/faiss_search to build and query FAISS
- FastAPI → pipelines/fact_checker to call Groq API + Crawl4AI
- FastAPI → Disk (captions.csv, faiss_index.bin)

##### Architectural patterns & design principles
- Layered architecture: UI / API / Data / External Integrations
- Modular pipelines for each major function (extraction, embedding, search, fact-checking)
- Cohesion: each module has a single responsibility
- Separation of deployment concerns: containerizable services

##### Relevant technologies & frameworks
- Python, FastAPI (backend), Streamlit (frontend)
- yt-dlp for caption retrieval
- pandas for CSV processing
- SentenceTransformers + FAISS for embedding & search
- Groq API & Crawl4AI for fact-checking
- Docker, GCP VM for deployment

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interaction
    User[/"User"/]:::external
    User -->|"Input URL / Search Query"| UI

    %% Frontend Layer
    subgraph "Frontend Layer"
        UI["Streamlit UI\nsrc/UI/app.py"]:::frontend
    end

    %% Backend Layer
    subgraph "Backend Layer"
        API["FastAPI App\nmain.py"]:::backend
        Docs["FastAPI Docs\nfastapi_APP_info.md"]:::backend
    end

    UI -->|"REST API Calls"| API
    API -->|"serves docs"| Docs

    %% Caption Extraction
    subgraph "Caption Extraction" 
        CC["CC.py\nsrc/CC_capture/CC.py"]:::backend
        LoadCC["load_cc.py\nsrc/CC_capture/load_cc.py"]:::backend
    end

    API -->|"fetch captions"| CC
    CC -->|"call load_cc"| LoadCC
    LoadCC -->|"yt-dlp"| YouTube

    %% Data Storage
    subgraph "Data Storage"
        CSV["captions.csv"]:::data
        FAISSFile["faiss_index.bin"]:::database
    end

    LoadCC -->|"store captions"| CSV

    %% Pipelines
    subgraph "Pipelines"
        Embed["Embedding Pipeline\nsrc/pipelines/embedding_pipeline.py"]:::pipelines
        Search["Search Pipeline\nsrc/pipelines/search_pipeline.py"]:::pipelines
        FactCheck["Fact-Checking Pipeline\nsrc/pipelines/fact_checker.py"]:::pipelines
    end

    API -->|"generate embeddings"| Embed
    Embed -->|"build index"| FAISSFile

    API -->|"search captions"| Search
    Search -->|"query index"| FAISSFile
    Search -->|"return results"| API

    API -->|"summarize & fact-check"| FactCheck
    FactCheck -->|"call GroqAPI"| GroqAPI
    FactCheck -->|"call Crawl4AI"| Crawl4AI
    FactCheck -->|"return summary"| API

    %% External Services
    subgraph "External Services"
        YouTube[("YouTube")]:::external
        GroqAPI[("Groq API")]:::external
        Crawl4AI[("Crawl4AI")]:::external
    end

    %% Deployment
    subgraph "Deployment"
        Dockerfile["Dockerfile"]:::deployment
        DockerIgnore[".dockerignore"]:::deployment
        GCP["GCP VM"]:::deployment
    end

    API -->|"containerize"| Dockerfile
    API --> DockerIgnore
    Dockerfile --> GCP
    DockerIgnore --> GCP

    %% Click Events
    click UI "https://github.com/jatin-mehra119/ai-powered-youtube-video-summarizer-and-fact-checker/blob/main/src/UI/app.py"
    click API "https://github.com/jatin-mehra119/ai-powered-youtube-video-summarizer-and-fact-checker/blob/main/main.py"
    click Docs "https://github.com/jatin-mehra119/ai-powered-youtube-video-summarizer-and-fact-checker/blob/main/fastapi_APP_info.md"
    click CC "https://github.com/jatin-mehra119/ai-powered-youtube-video-summarizer-and-fact-checker/blob/main/src/CC_capture/CC.py"
    click LoadCC "https://github.com/jatin-mehra119/ai-powered-youtube-video-summarizer-and-fact-checker/blob/main/src/CC_capture/load_cc.py"
    click Embed "https://github.com/jatin-mehra119/ai-powered-youtube-video-summarizer-and-fact-checker/blob/main/src/pipelines/embedding_pipeline.py"
    click Search "https://github.com/jatin-mehra119/ai-powered-youtube-video-summarizer-and-fact-checker/blob/main/src/pipelines/search_pipeline.py"
    click FactCheck "https://github.com/jatin-mehra119/ai-powered-youtube-video-summarizer-and-fact-checker/blob/main/src/pipelines/fact_checker.py"
    click CSV "https://github.com/jatin-mehra119/ai-powered-youtube-video-summarizer-and-fact-checker/blob/main/captions.csv"
    click Dockerfile "https://github.com/jatin-mehra119/ai-powered-youtube-video-summarizer-and-fact-checker/tree/main/Dockerfile"
    click DockerIgnore "https://github.com/jatin-mehra119/ai-powered-youtube-video-summarizer-and-fact-checker/blob/main/.dockerignore"

    %% Styles
    classDef frontend fill:#b2f2bb,stroke:#2f9e44
    classDef backend fill:#a5d8ff,stroke:#339af0
    classDef pipelines fill:#ffd8a8,stroke:#fd7e14
    classDef data fill:#ffe066,stroke:#fab005
    classDef database fill:#ffa94d,stroke:#f76707
    classDef external fill:#dee2e6,stroke:#868e96
    classDef deployment fill:#e7f5ff,stroke:#5c7cfa
```
