How pacificrm/rag-architecture works
Project Type and Purpose
• This is a full-stack Python Streamlit application that supports multimodal document ingestion (audio, video, image, text, CSV, PDF, DOCX, JSON, YAML) and provides a natural-language query interface via a Retrieval-Augmented Generation (RAG) pipeline.
• The purpose is to extract text from files, store vector embeddings in a Milvus database, and answer queries by retrieving relevant embeddings and generating responses via an LLM.
File Structure
• Top-level files:
• app.py – Main Streamlit application (UI + controller).
• requirements.txt – Python dependencies.
• milvus_database.db – Local Milvus vector database file.
• Dataset/Images/ – Raw data storage.
• utils/ – Collection of processing modules:
– audio_utils.py, video_utils.py, image_utils.py, document_loaders.py, milvus_client.py.
• No separate “frontend” folder: Streamlit handles UI and server logic in app.py.
• Configs: requirements.txt; no Docker/Kubernetes scripts.
README for Additional Insights
• Dependencies: LangChain, HuggingFaceEmbeddings, Milvus, EasyOCR, speech_recognition, pydub, MoviePy, transformers.
• Features: upload & process, embed & store, query & generate.
Main Components and Relationships
a. Components
1. Client Browser – User interacts with Streamlit UI.
2. Streamlit App (app.py)
• Upload Mode: accepts file → routes to appropriate utils module.
• Query Mode: accepts question → triggers RAG pipeline.
3. Utility Modules (utils/)
• audio_utils, video_utils, image_utils, document_loaders: extract raw text.
• milvus_client: connect, insert, search embeddings.
4. Embedding Service
• LangChain + HuggingFaceEmbeddings for vectorization.
5. Milvus Database
• Stores and retrieves embeddings.
6. LLM / RAG Pipeline
• LangChain chains for retrieval and generation.
7. External ML Models/Services
• EasyOCR, speech_recognition, pydub, MoviePy, transformer models from Hugging Face.
8. Local Storage
• Dataset/ folder, Images/ folder, milvus_database.db.
b. Data Flows and Interactions
• Upload Flow:
Browser → Streamlit UI → app.py → utils module → raw text → embeddings → milvus_client → Milvus → confirmation back to UI.
• Query Flow:
Browser → UI → app.py → milvus_client search → retrieved passages → LLM (RAG) → generated answer → UI.
• External Models invoked by utils modules and embedding service.
c. Architectural Patterns and Principles
• Separation of Concerns: UI vs. processing vs. storage.
• Modular design: one utils module per modality.
• Retrieval-Augmented Generation: vector DB + LLM chain.
• Stateful database for embeddings; stateless processing modules.
d. Key Technologies / Frameworks
• Streamlit (UI + server)
• LangChain (orchestration, embedding wrapper)
• HuggingFaceEmbeddings (vector creation)
• Milvus (vector database)
• EasyOCR, speech_recognition, pydub, MoviePy (data extraction)
• Python 3.8+, various LangChain document loaders