How vincentkoc/airgapped-offfline-rag works
Main Components
a. User Interface (UI) – the Streamlit front-end
b. Backend Application – “app/” codebase
• DocumentProcessor (app/document_processor.py)
• RAG Orchestrator (app/rag.py) using LangChain
• ModelHandler (app/model_handler.py) invoking local Llama C++/Gemini/Mistral
• Utility functions (app/utils.py)
c. Vector Storage – ChromaDB (or Qdrant via Chromadb API)
d. Model Store – local filesystem folder “models/” containing GGUF files
e. Configuration – config.yaml, .env, example.env
f. Persistence – “documents/” for uploaded PDFs
g. Infrastructure – Docker, Docker-Compose, Makefile
Relationships & Data Flow
1. User uploads a PDF via Streamlit UI.
2. UI sends file + query request to the Backend Application.
3. DocumentProcessor reads PDF → splits into chunks → calls Sentence-Transformer to embed.
4. Embeddings are written to Vector Storage (ChromaDB).
5. On question submit, RAG Orchestrator:
a. Queries Vector Storage for top-k relevant chunks
b. Builds prompt (including retrieved chunks)
c. Calls ModelHandler to invoke llama.cpp (or analogous local binaries)
d. Receives streamed response
6. Response is returned to UI for display.
7. All components read parameters (model paths, chunk sizes) from config.yaml/.env.
Architectural Patterns & Technologies
• Modular monolith: single container running Streamlit + Python modules
• Retrieval-Augmented Generation via LangChain (app/rag.py)
• Local LLM inference with llama.cpp + GGUF models (app/model_handler.py)
• Vector database integration (ChromaDB) for offline embeddings
• Docker Compose orchestration for service lifecycle
• Test suite (pytest + tox) for component validation
– Clearly labeled components and containers
– Annotations of key libraries/frameworks (LangChain, llama.cpp, Streamlit)
– Configuration flow from config.yaml/.env into modules
– Optional: include ports (e.g., Streamlit:8501, ChromaDB:8000)