How saitejasrivilli/youtube-transcript-query-tool works
Main Components
• User Interface & Application Server:
• The Streamlit-based front end serves as both the UI and the entry point for user requests (e.g., entering a YouTube URL and specifying a query task). This is implemented in the file (app.py) which orchestrates the overall workflow.
• YouTube Transcript Fetcher:
• This component interacts with the YouTube Transcript API to fetch the transcript of a given video.
• Transcript Chunking Module:
• A logical layer within the application that handles splitting the full transcript into manageable chunks, especially considering GPT-3.5’s token limitations.
• Pinecone Storage Service:
• Acts as the vector database where the processed (chunked) transcript data is stored for efficient retrieval and similarity searches.
• Query & Summarization Engine:
• Utilizes OpenAI GPT-3.5 to process the stored transcript chunks when a query or summarization request is made.
Relationships and Interactions
• The user interacts with the Streamlit UI from their web browser.
• When a YouTube URL is submitted, the application first fetches the transcript using the YouTube Transcript API.
• The fetched transcript is then passed to the chunking module; after processing, these chunks are stored in Pinecone.
• Upon receiving a user query (i.e., “Summarize the video” or similar task), the system retrieves the relevant transcript chunks from Pinecone and sends these, along with the query, to the OpenAI GPT-3.5 API.
• The response generated by GPT-3.5 is then displayed back to the user via the Streamlit UI.
Architectural Patterns & Design Principles
• The System Follows a Modular Architecture: each functionality (fetching, chunking, storing, querying) is handled as a separate logical component.
• Service-Oriented Integration: external APIs (YouTube, Pinecone, OpenAI GPT-3.5) are integrated as services.
• Separation of Concerns: the UI (user interactions via Streamlit) is clearly separated from data processing and storage.
• Lightweight, Single-Page App: given that the entire codebase is encapsulated in a few files, the design should emphasize clarity in data flow over overly complex inter-module interactions.
Relevant Technologies
• Streamlit – for building the interactive web user interface.
• OpenAI GPT-3.5 API – for natural language processing tasks.
• Pinecone – for scalable vector storage and similarity search.
• YouTube Transcript API – for fetching video transcripts.