How manikantha-asam/text_summarization works
Project Type and Purpose
• This is a lightweight full-stack web application (single-page) built with Streamlit (front-end + app server in one) and Python.
• Purpose: Accept a YouTube video URL or website URL, fetch or scrape content, and produce a summarized output via the Groq API (Gemma model).
Main Components
a. Client / UI (Browser)
• Streamlit front-end served by app.py
• Input fields for URL or video link
• “Summarize” trigger button
b. Application Server (Streamlit / Python)
• app.py orchestrates the workflow
• dotenv loads .env (API key)
• validators module checks URL syntax
• LangChain chains handle content loading and summarization
c. Content Sources
• YouTube video transcripts (via LangChain’s YouTube loader)
• Website HTML text scraper (via LangChain’s web loader)
d. External Summarization Service
• Groq API endpoint (Gemma model)
• Authenticated via API key stored in .env
• Could be Redis or a simple file cache to avoid re-scraping
• Not present today but call out as a future extensibility point
f. Local Environment
• .env file for configuration (API_KEY)
• requirements.txt lists Python dependencies
Interactions & Data Flow
1. User enters a YouTube URL or website URL in the Streamlit UI.
2. Streamlit (app.py) receives the request and:
– Reads .env (dotenv) for the Groq API key.
– Validates the URL with the validators library.
– Passes the URL to the appropriate LangChain “loader” chain.
3. LangChain loader fetches the raw text (video transcripts or scraped HTML).
4. Loader output is sent to the Groq API via langchain_groq:
– HTTP request to Groq endpoint with the Gemma model.
– Receives summarized text response.
5. Streamlit displays the summary back to the user.
Architectural Patterns & Design Principles
• Orchestration Pattern: app.py orchestrates a sequence of responsibility (load, validate, summarize).
• Thin UI Layer: Streamlit handles UI and routing in one process—no separate web server.
• Externalized Configuration: .env file for secrets keeps code free from keys.
• Single Responsibility Modules: validators for input, LangChain chains for content loading & summarization, dotenv for config.
Final Checklist
[ ] Clearly labeled components (UI, app.py, validators, dotenv, LangChain Loaders, Groq API)
[ ] Grouping of related sub-components (e.g., inside a “LangChain” container)