How gideon-nobbs7/youtube-transcript-pod works
Project Type and Purpose
• This is a Python‑based data‑ingestion and processing microservice focused on fetching YouTube video transcripts, summarizing/categorizing them via ChatGPT, and persisting results.
• It exposes a RESTful API (FastAPI) for triggering or retrieving processed transcripts and metadata.
Main Components
a. API Layer (src/api)
• main.py: Bootstraps FastAPI server.
• routes.py: Defines HTTP endpoints—for example, /scrape, /videos, /download.
• model.py: ORM-like definitions of database tables (video metadata, categories).
• db_config.py: Supabase (Postgres) connection settings.
• supa_store.py: Wrapper around Supabase Storage for saving transcript files/PDFs.
b. Service/Utility Layer (src/utils)
• youtube.py: Calls YouTube Data API to list channel videos and download transcripts.
• chatgpt.py: Sends prompts to OpenAI API to categorize or summarize transcripts.
• csv_writer.py: Builds the video_details.csv report.
• txt_to_pdf.py: (Optional) Converts transcript text files into PDF format.
• main.py (in utils): CLI entrypoint that orchestrates the end‑to‑end pipeline when run standalone.
c. Data Storage
• Supabase Postgres (via db_config.py and model.py) for structured metadata.
• Supabase Storage bucket (via supa_store.py) for transcript or PDF files.
• Local filesystem (e.g., transcripts/Category/...) if running as standalone script.
• CSV file output.
d. External Services
• YouTube Data API v3
• OpenAI API (ChatGPT)
• Supabase (Postgres + Storage)
Relationships and Data Flows
1. Client (CLI or HTTP) issues request →
2. FastAPI main.py routes through routes.py →
3. Route handlers invoke utility functions (youtube.py, chatgpt.py, etc.) →
4. youtube.py fetches video list and transcripts from YouTube Data API →
• Raw transcripts passed to chatgpt.py → OpenAI API categorization →
5. Categorized transcripts written:
• Persisted to Supabase Storage via supa_store.py (or to local transcripts/ directory)
• Metadata ingested into Supabase Postgres via model.py
• CSV aggregated by csv_writer.py
6. Responses returned to client (status, metadata, download links)
Architectural Patterns and Principles
• Layered Architecture:
• Presentation/API layer (FastAPI)
• Service layer (utils)
• Data layer (Supabase + Storage)
• Separation of Concerns: each utility module has a single responsibility.
• External‑Service Facade: wrappers (supa_store.py, chatgpt.py) encapsulate third‑party APIs.