How thomasjanssen-tech/local-rag-with-ollama works
1. Overall shape
This repo is a three-stage local RAG pipeline built around Ollama. It separates data acquisition, indexing, and chat/query serving into distinct scripts rather than one monolithic app.
2. Data acquisition layer
1_scraping_wikipedia.py is the ingestion entrypoint. It scrapes Wikipedia content, likely using keywords.xlsx as the source list of topics/pages to fetch. This is the raw document generation step for the knowledge base.
3. Preprocessing and document preparation
2_chunking_embedding_ingestion.py turns scraped text into retrieval-ready units. The key architectural boundary here is between “documents” and “chunks”: content is split into smaller pieces before embedding so retrieval works at passage level, not whole-article level.
4. Embedding and vector indexing
The repo is centered on embedding text locally through Ollama, then storing vectors in a vector database for similarity search. The README points to Chroma and Ollama embedding models, so the likely indexing path is: chunk text → generate embeddings → persist in a local vector store.
5. Retrieval layer
example_retriever.py and the ingestion script imply a retrieval subsystem that performs nearest-neighbor search over the stored embeddings. This is the core RAG boundary: user questions are converted into vectors and matched against indexed chunks to supply context.
6. Chat / generation layer
3_chatbot.py is the interactive application. It likely uses Streamlit for the UI and queries the retrieval layer to build a context-aware prompt for a local Ollama chat model such as Qwen3. This is the final answer-generation stage.
7. Runtime and UI
The repo is Python 3.11+ and the user-facing app is launched with Streamlit. That makes the architecture split between batch scripts for offline preparation and a lightweight interactive frontend for runtime Q&A.
8. External services and dependencies
Bright Data is used for scraping access, with credentials provided through .env. The README also mentions optional ChatGPT or Anthropic API keys, which suggests the system can swap the generation backend, but Ollama is the default/local path and does not require those external LLM APIs.
9. Configuration boundary
.env.example indicates environment-driven configuration rather than hardcoded secrets. This is important architecturally because API keys and model choices are externalized, keeping the scripts portable across local setups.
10. Reference / example modules
example_chunking.py, example_embedding.py, and example_retriever.py look like focused demos or prototypes for individual pipeline stages. They likely mirror the production scripts in smaller units and are useful for understanding how the chunking, embedding, and retrieval boundaries fit together.
11. End-to-end data flow
The main flow is: scrape Wikipedia content → chunk it → embed it locally → store embeddings in a vector database → run a Streamlit chatbot that retrieves relevant chunks and passes them to an Ollama model for response generation. This is a classic offline-indexed, online-retrieval RAG architecture.
12. What to diagram
An accurate architecture diagram should show three layers: a batch ingestion pipeline, a persistent vector store, and an online chat app. Around those, show external dependencies for Bright Data and Ollama, plus optional third-party LLM APIs, with the vector store as the central shared data boundary between indexing and chat.
Main entry points: 2_chunking_embedding_ingestion.py Chunk & index · 1_scraping_wikipedia.py Scrape Wikipedia · 3_chatbot.py Chatbot · .env.example Env config
Components 13
Offline pipeline
Batch ingestion and indexing stages that prepare the knowledge base.
- Keywords · Source list of Wikipedia topics/pages to scrape.
keywords.xlsx - Scrape Wikipedia · Fetches raw Wikipedia content for the knowledge base.
1_scraping_wikipedia.py - Chunk & index · Splits documents into chunks, embeds them, and writes vectors.
2_chunking_embedding_ingestion.py - Chunking demo · Reference implementation for text splitting.
example_chunking.py - Embedding demo · Reference implementation for local embedding calls.
example_embedding.py
Runtime app
Interactive query and answer generation path.
- Retrieve chunks · Nearest-neighbor lookup over the vector store.
example_retriever.py - Chatbot · Interactive RAG UI that retrieves context and generates answers.
3_chatbot.py - Retriever demo · Reference implementation for vector search and retrieval.
example_retriever.py
External systems
Local models, databases, and third-party services used by the pipeline.
- Vector store · Local similarity-search store for embedded chunks.
- Ollama embeddings · Local embedding backend used during indexing and retrieval.
- Ollama chat · Local chat model used to answer user questions.
- Bright Data · External scraping infrastructure and credentials source.
- Env config · Environment-driven configuration for keys and model settings.
.env.example
Connections 16
- Keywords → Scrape Wikipedia: drives topics · Topic list determines what content gets scraped.
- Bright Data → Scrape Wikipedia: access · Provides scraping access and credentials.
- Env config → Scrape Wikipedia: config · Supplies runtime secrets and settings.
- Scrape Wikipedia → Chunk & index: raw docs · Scraped pages become the document input for preprocessing.
- Chunk & index → Ollama embeddings: embed text · Chunks are converted into vectors locally.
- Chunk & index → Vector store: persist vectors · Indexed chunks are stored for later retrieval.
- Ollama embeddings → Vector store: vectorize · Embeddings are written into the local index.
- Vector store → Retrieve chunks: search corpus · Retriever queries the persisted vector index.
- Chunking demo → Chunk & index: reference · Example code mirrors the chunking boundary.
- Embedding demo → Ollama embeddings: reference · Example code mirrors local embedding usage.
- Retriever demo → Retrieve chunks: reference · Example code mirrors the retrieval boundary.
- Env config → Chatbot: config · Supplies model and API settings for runtime.
- Vector store → Chatbot: context · Chatbot pulls relevant chunks from the shared index.
- Chatbot → Ollama chat: generate · Retrieved context is sent to the local chat model.
- Retriever demo → Chatbot: inspires · Retriever pattern feeds the chat application's query flow.
- Env config → Ollama chat: model choice · Environment can switch or configure generation backends.