# gideon-nobbs7/youtube-transcript-pod architecture

> Architecture diagram of the GitHub repository gideon-nobbs7/youtube-transcript-pod, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gideon-nobbs7/youtube-transcript-pod
- Repository: https://github.com/gideon-nobbs7/youtube-transcript-pod

## Overview

##### Project Type and Purpose
- This is a Python‑based data‑ingestion and processing microservice focused on fetching YouTube video transcripts, summarizing/categorizing them via ChatGPT, and persisting results.
- It exposes a RESTful API (FastAPI) for triggering or retrieving processed transcripts and metadata.

##### Main Components
a. API Layer (src/api)
- `main.py`: Bootstraps FastAPI server.
- `routes.py`: Defines HTTP endpoints—for example, `/scrape`, `/videos`, `/download`.
- `model.py`: ORM-like definitions of database tables (video metadata, categories).
- `db_config.py`: Supabase (Postgres) connection settings.
- `supa_store.py`: Wrapper around Supabase Storage for saving transcript files/PDFs.
b. Service/Utility Layer (src/utils)
- `youtube.py`: Calls YouTube Data API to list channel videos and download transcripts.
- `chatgpt.py`: Sends prompts to OpenAI API to categorize or summarize transcripts.
- `csv_writer.py`: Builds the `video_details.csv` report.
- `txt_to_pdf.py`: (Optional) Converts transcript text files into PDF format.
- `main.py` (in utils): CLI entrypoint that orchestrates the end‑to‑end pipeline when run standalone.
c. Data Storage
- Supabase Postgres (via `db_config.py` and `model.py`) for structured metadata.
- Supabase Storage bucket (via `supa_store.py`) for transcript or PDF files.
- Local filesystem (e.g., `transcripts/Category/...`) if running as standalone script.
- CSV file output.
d. External Services
- YouTube Data API v3
- OpenAI API (ChatGPT)
- Supabase (Postgres + Storage)

##### Relationships and Data Flows
1. Client (CLI or HTTP) issues request →
2. FastAPI `main.py` routes through `routes.py` →
3. Route handlers invoke utility functions (`youtube.py`, `chatgpt.py`, etc.) →
4. `youtube.py` fetches video list and transcripts from YouTube Data API →
- Raw transcripts passed to `chatgpt.py` → OpenAI API categorization →
5. Categorized transcripts written:
- Persisted to Supabase Storage via `supa_store.py` (or to local `transcripts/` directory)
- Metadata ingested into Supabase Postgres via `model.py`
- CSV aggregated by `csv_writer.py`
6. Responses returned to client (status, metadata, download links)

##### Architectural Patterns and Principles
- Layered Architecture:
- Presentation/API layer (FastAPI)
- Service layer (utils)
- Data layer (Supabase + Storage)
- Separation of Concerns: each utility module has a single responsibility.
- External‑Service Facade: wrappers (`supa_store.py`, `chatgpt.py`) encapsulate third‑party APIs.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Clients
    Client["Client (HTTP/CLI)"]:::external

    %% API Layer
    subgraph "API Layer"
        API_Main["main.py\n(FastAPI Server)"]:::internal
        Routes["routes.py\n(Endpoints)"]:::internal
        Model["model.py\n(DB Models)"]:::internal
        DB_Config["db_config.py\n(DB Config)"]:::internal
        SupaStore["supa_store.py\n(Storage Facade)"]:::internal
    end

    %% Service Layer
    subgraph "Service Layer"
        YT["youtube.py"]:::internal
        ChatGPT["chatgpt.py"]:::internal
        CSV["csv_writer.py"]:::internal
        PDF["txt_to_pdf.py"]:::internal
        CLI["main.py\n(CLI Orchestrator)"]:::internal
    end

    %% Data Stores
    subgraph "Data Stores"
        Postgres["Supabase Postgres"]:::datastore
        StorageBucket["Supabase Storage"]:::datastore
        LocalFS["Local Filesystem"]:::datastore
        CSVFile["video_details.csv"]:::datastore
    end

    %% External Services
    subgraph "External Services"
        YouTubeAPI["YouTube Data API"]:::external
        OpenAI["OpenAI API"]:::external
    end

    %% Flows
    Client -->|"HTTP GET /scrape"| API_Main
    Client -->|"invoke CLI"| CLI

    API_Main --> Routes
    API_Main --> DB_Config
    Routes -->|"call youtube.py"| YT
    YT -->|"Fetch transcripts"| YouTubeAPI
    YouTubeAPI -->|"Raw transcripts"| YT
    YT -->|"pass transcripts"| ChatGPT
    ChatGPT -->|"Categorize/Summarize"| OpenAI
    OpenAI -->|"Processed text"| ChatGPT

    ChatGPT -->|"save metadata"| Model
    Model -->|"persist to DB"| Postgres

    ChatGPT -->|"save files"| SupaStore
    SupaStore --> StorageBucket

    ChatGPT -->|"generate CSV entry"| CSV
    CSV --> CSVFile

    ChatGPT -->|"optional PDF conversion"| PDF
    PDF --> LocalFS

    Routes -->|"return metadata/download links"| Client
    CLI --> YT
    CLI --> ChatGPT
    CLI --> Model
    CLI --> SupaStore
    CLI --> CSV
    CLI --> PDF

    %% Click Events
    click API_Main "https://github.com/gideon-nobbs7/youtube-transcript-pod/blob/main/src/api/main.py"
    click Routes "https://github.com/gideon-nobbs7/youtube-transcript-pod/blob/main/src/api/routes.py"
    click Model "https://github.com/gideon-nobbs7/youtube-transcript-pod/blob/main/src/api/model.py"
    click DB_Config "https://github.com/gideon-nobbs7/youtube-transcript-pod/blob/main/src/api/db_config.py"
    click SupaStore "https://github.com/gideon-nobbs7/youtube-transcript-pod/blob/main/src/api/supa_store.py"
    click YT "https://github.com/gideon-nobbs7/youtube-transcript-pod/blob/main/src/utils/youtube.py"
    click ChatGPT "https://github.com/gideon-nobbs7/youtube-transcript-pod/blob/main/src/utils/chatgpt.py"
    click CSV "https://github.com/gideon-nobbs7/youtube-transcript-pod/blob/main/src/utils/csv_writer.py"
    click PDF "https://github.com/gideon-nobbs7/youtube-transcript-pod/blob/main/src/utils/txt_to_pdf.py"
    click CLI "https://github.com/gideon-nobbs7/youtube-transcript-pod/blob/main/src/utils/main.py"

    %% Styles
    classDef internal fill:#D6EAF8,stroke:#3498DB,color:#1B4F72;
    classDef datastore fill:#ABEBC6,stroke:#28B463,color:#145A32;
    classDef external fill:#F9E79F,stroke:#D68910,color:#7E5109;
```
