How tdtheautomator/local-llm-01 works
Project Type & Purpose
• This is a CLI + Gradio demo application that runs entirely locally.
• It implements a two‑phase Retrieval‑Augmented Generation (RAG) pipeline:
a. Embedding Creation (ingest PDFs → vector store)
b. Query/Chat (retrieve relevant embeddings → local LLM → answers)
• Uses Ollama to serve models, ChromaDB for vector storage, LangChain library for orchestration, and Gradio for UI.
File Structure
• Root scripts:
• create_embeddings.py → loads PDFs (docs/) with pypdf, calls mxbai-embed-large to compute embeddings, stores them in ChromaDB.
• query01.py / query02.py → CLI interfaces for RAG queries (different prompts or flows).
• chatbot_gradio_context.py → Gradio app for interactive chat.
• docs/ → source PDF files to index.
• images/ → sample output screenshots (for docs, not in runtime).
• tools/ → precompiled standalone binaries of a simple chatbot (optional extras).
• .gradio/ → Gradio TLS certs & config.
• README.md → instructions & high‑level workflow.
a. Data Ingestion & Embedding Pipeline
• PDFs (docs/)
• create_embeddings.py (LangChain PDF loader + embedding model client)
• ChromaDB local store
b. Model Serving
• Ollama server (deepseek-r1:1.5b for generation; mxbai-embed-large for embeddings)
c. Query Layer
• CLI scripts (query01.py, query02.py)
• Gradio app (chatbot_gradio_context.py)
• LangChain Retriever & LLMChain
d. Vector Store
• ChromaDB files/directories (on disk)
e. External Dependencies
• Ollama daemon (runs locally, exposes gRPC/REST)
• Python environment & libraries (LangChain, Chromadb, Gradio, pypdf)
Relationships & Data Flow
• Embedding script → Ollama embedding model → embeddings
• Query script/Gradio UI → LangChain Retriever → ChromaDB (similarity search) → retrieved docs
• Retrieved docs + user query → LangChain LLMChain → Ollama generation model → answer
• UI displays answer back to user
Architectural Patterns & Principles
• Pipeline pattern (two distinct jobs: embed vs. query)
• Separation of concerns: storage (ChromaDB), compute (Ollama), orchestration (LangChain scripts), UI (CLI/Gradio)
• Local-first design (everything hosted on the same machine)