How pritam-parali/document-parsing-using-rag-major-project works
This is a Streamlit document assistant: users upload supported files, ask questions, and receive CSV analysis or retrieval-grounded LLM answers. Its separate YouTube workflow fetches transcripts and produces downloadable notes; OCR and Mermaid diagrams provide additional document/screenshot capabilities. The main RAG, persistence, and YouTube modules are sampled; some UI wiring is omitted, so uncertain links are not shown.
Main entry points: app.py Streamlit app · search.py RAG search · vectorestore.py FAISS store · youtube_loader.py Transcript loader
Components 21
User interface
- Streamlit app
app.py - Chat history
src/chat_history.py - Conversation JSON
chat_history.json
Document RAG
- File loader
src/data_loader.py - Chunk and embed
src/embedding.py - FAISS store
src/vectorestore.py - RAG search
src/search.py - CSV analysis
src/csv_agent.py
YouTube notes
- Transcript loader
src/youtube_loader.py - Transcript chunker
src/youtube_chunker.py - Notes generation
src/youtube_notes.py - Notes export
src/youtube_export.py
Assistant features
- Screenshot OCR
app.py - Mermaid generation
src/diagram_generator.py - Text-to-speech
app.py
Other components
- User
- Groq Llama
- YouTube Transcript API
- Tesseract OCR
- Mermaid renderer
- gTTS
Connections 30
- User → Streamlit app: uses
- Streamlit app → Chat history: loads and saves
- Chat history → Conversation JSON: reads and writes
- Streamlit app → File loader: loads documents
- File loader → Streamlit app: returns documents
- Streamlit app → FAISS store: builds index
- FAISS store → Chunk and embed: chunks and embeds
- Streamlit app → RAG search: submits query
- RAG search → CSV analysis: checks CSVs
- RAG search → FAISS store: queries vectors
- FAISS store → RAG search: returns matches
- RAG search → Groq Llama: requests answer
- Groq Llama → RAG search: returns answer
- RAG search → Streamlit app: returns response
- User → Streamlit app: provides video URL
- Streamlit app → Transcript loader: requests transcript
- Transcript loader → YouTube Transcript API: fetches transcript
- Transcript loader → Streamlit app: returns transcript
- Streamlit app → Transcript chunker: splits transcript
- Streamlit app → Notes generation: summarizes chunks
- Notes generation → Groq Llama: requests summaries
- Streamlit app → Notes generation: generates final notes
- Streamlit app → Notes export: exports notes
- User → Streamlit app: provides screenshot
- Streamlit app → Tesseract OCR: extracts text
- Streamlit app → Mermaid generation: requests flowchart
- Mermaid generation → Groq Llama: generates Mermaid
- Streamlit app → Mermaid renderer: renders diagram
- Streamlit app → Text-to-speech: reads response aloud
- Text-to-speech → gTTS: generates audio