How paragd004/llm_and_rag works
Project type and purpose
• This is an “AI demo” repository showcasing several Jupyter-based pipelines that combine local LLMs (via Ollama) with Retrieval-Augmented Generation (RAG).
• Key use cases provided as notebooks:
• Airline AI Assistant
• Expert Knowledge Worker (with a local markdown knowledge base)
• Webpage Summarizer
File structure
• Top-level notebooks (Airline AI Assistant.ipynb, Web Page Summarizer.ipynb) and one subfolder “Expert Knowledge worker” containing:
• A Jupyter notebook for the Expert Knowledge Worker workflow
• A “knowledge-base” directory full of markdown docs grouped into company, contracts, employees, products
• Configuration file .env.local.txt for secrets (vector DB credentials, Ollama settings)
README for additional insights
• States reliance on Ollama to run open-source LLMs locally
• Requires Python + the ollama client library for RAG code
• Mentions a Vector DB storing embeddings (“RAG enhances by retrieving relevant docs…”)
Main components
a. User Interface (Jupyter Notebooks)
b. RAG Orchestration Service (Python code invoked in the notebooks)
c. Knowledge Base (local markdown files)
d. Vector Database (e.g., Pinecone, Weaviate) storing your document embeddings
e. Ollama LLM Engine (local LLaMA-3 process managed by Ollama)
f. Configuration Layer (.env.local.txt for secrets/config)
Relationships and interactions
1. The user writes/executes code in a notebook (Airline AI Assistant / Expert Knowledge Worker / Webpage Summarizer).
2. Notebook code reads .env.local.txt → initializes Vector DB client and Ollama client.
3. RAG pipeline:
• Load selected docs from Knowledge Base → compute or retrieve embeddings → query Vector DB for top-k relevant docs
• Assemble prompt: user question + retrieved docs context
• Send prompt via Ollama Python client → Ollama spawns or routes to local LLaMA-3 model → returns generated answer
4. Notebook renders the “Informed Answer.”
Architectural patterns/design principles
• Separation of concerns: UI (notebook) vs. orchestration (RAG code) vs. data (markdown KB + vector store) vs. compute (LLM engine)
• RAG as a two-step pipeline: Retrieve then Generate
• Local-first AI: all heavy compute runs in your environment (no external API calls)