# abhi-sys511/llama3-rag-knowledge-assistant architecture

> Architecture diagram of the GitHub repository abhi-sys511/llama3-rag-knowledge-assistant, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/abhi-sys511/llama3-rag-knowledge-assistant
- Repository: https://github.com/abhi-sys511/llama3-rag-knowledge-assistant

## Overview

1. This repository is a local Retrieval-Augmented Generation (RAG) research assistant, centered on a CLI workflow rather than a web app. The primary path is: user input → document retrieval → prompt construction → local LLM generation → streamed response → structured logging.

2. `main.py` is described in the README as the main controller. It likely owns the interactive loop, command parsing, timing, streaming, and the handoff between retrieval and generation. It is the orchestration boundary, not the knowledge layer itself.

3. `rag_engine.py` is the core “memory” subsystem. It handles PDF ingestion, text extraction, chunking, embedding creation, and persistence into a Chroma vector store. It also provides retrieval: converting a question into a semantic search against stored chunks and returning the most relevant context.

4. `rag.py` is the prompt-augmentation layer. Its job is to merge retrieved document context with the user’s question into the final prompt sent to the LLM. This is the boundary that turns raw retrieval into grounded generation.

5. The local model runtime is Ollama with Llama3, as stated in the README. Python does not call a remote API; instead it talks to the local Ollama service, which loads the model and streams tokens back. That makes the system offline, private, and dependent on local CPU/GPU capacity rather than cloud billing.

6. Persistent knowledge storage is a Chroma vector database under `knowledge_db/` per the README. That directory is the long-term semantic memory: embedded chunks and metadata are stored there so PDFs do not need to be reprocessed on every run.

7. The document ingestion flow is multi-step and important for any architecture diagram: PDF file → text extraction → chunk splitting → embeddings → vector DB persistence. The sample PDF `ABAP_New.pdf` in the repo is likely used as a test document for this pipeline.

8. The query-time flow is the mirror image: user question → embedding/search → top matching chunks → prompt assembly in `rag.py` → Ollama/Llama3 generation → streamed answer in the terminal. This is the central RAG loop that reduces hallucination by grounding the answer in loaded documents.

9. Logging and observability are first-class concerns here. The README describes structured JSON output with timestamp, query, response, and response time; the file tree shows `research_output.txt`, suggesting the logging artifact may have changed over time. Either way, the architecture includes response telemetry as a separate concern from generation.

10. `evaluate.py` appears to be an analysis or benchmarking utility, likely used to assess responses, performance, or retrieval quality. `tools.py` is probably shared helper code for file, text, or logging utilities. These are support modules around the main pipeline, not core runtime dependencies.

11. `app.py` is present alongside `main.py`, so there are likely multiple entrypoints or an earlier wrapper layer. For diagram purposes, treat `main.py` as the interactive controller and `app.py` as either an alternate launcher or packaging entrypoint unless code inspection shows otherwise.

12. The architectural boundaries are clear: controller layer (`main.py`/`app.py`), retrieval/memory layer (`rag_engine.py` + Chroma storage), prompt layer (`rag.py`), model runtime (Ollama/Llama3), and observability/output (`research_output.*`, timing, evaluation). That separation is what makes the project modular and diagram-friendly.

Main entry points: [`main.py`](https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/main.py) (Main loop), [`rag_engine.py`](https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/rag_engine.py) (RAG engine), [`tools.py`](https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/tools.py) (Tools), [`rag.py`](https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/rag.py) (Prompt builder)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Entry points"]
  node_app_py["App<br/>entrypoint<br/>[app.py]"]
  node_main_py["Main loop<br/>controller<br/>[main.py]"]
  node_stream_output["Streamed answer<br/>terminal output"]
end

subgraph group_pipeline["RAG pipeline"]
  node_rag_engine_py["RAG engine<br/>memory core<br/>[rag_engine.py]"]
  node_rag_py["Prompt builder<br/>augmentation<br/>[rag.py]"]
  node_ingestion_flow{{"Ingest flow<br/>pipeline"}}
  node_retrieval_flow{{"Search flow<br/>pipeline"}}
end

subgraph group_runtime["Model runtime"]
  node_llama3_runtime["Llama3<br/>local LLM"]
  node_ollama_service["Ollama<br/>model host"]
end

subgraph group_storage["Knowledge store"]
  node_chroma_store[("Chroma DB<br/>vector store")]
  node_structured_log["Research log<br/>telemetry"]
  node_abap_pdf["Sample PDF<br/>source document<br/>[ABAP_New.pdf]"]
end

subgraph group_ops["Support"]
  node_evaluate_py["Evaluate<br/>analysis<br/>[evaluate.py]"]
  node_tools_py["Tools<br/>helpers<br/>[tools.py]"]
end

node_app_py -->|"launches"| node_main_py
node_main_py -->|"ingest/retrieve"| node_rag_engine_py
node_main_py -->|"build prompt"| node_rag_py
node_main_py -->|"generate"| node_ollama_service
node_ollama_service -->|"hosts"| node_llama3_runtime
node_main_py -->|"streams"| node_stream_output
node_main_py -->|"logs"| node_structured_log
node_abap_pdf -->|"source"| node_ingestion_flow
node_ingestion_flow -->|"indexes"| node_rag_engine_py
node_rag_engine_py -->|"persists"| node_chroma_store
node_main_py -->|"asks"| node_retrieval_flow
node_retrieval_flow -->|"searches"| node_rag_engine_py
node_rag_engine_py -->|"queries"| node_chroma_store
node_retrieval_flow -->|"supplies context"| node_rag_py
node_rag_py -->|"format prompt"| node_ollama_service
node_evaluate_py -.->|"analyzes"| node_structured_log
node_tools_py -.->|"supports"| node_main_py
node_tools_py -.->|"supports"| node_rag_engine_py
node_tools_py -.->|"supports"| node_evaluate_py

click node_app_py "https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/app.py"
click node_main_py "https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/main.py"
click node_rag_engine_py "https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/rag_engine.py"
click node_rag_py "https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/rag.py"
click node_structured_log "https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/research_output.txt"
click node_evaluate_py "https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/evaluate.py"
click node_tools_py "https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/tools.py"
click node_abap_pdf "https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/ABAP_New.pdf"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app_py,node_main_py,node_stream_output toneBlue
class node_rag_engine_py,node_rag_py,node_ingestion_flow,node_retrieval_flow toneAmber
class node_llama3_runtime,node_ollama_service toneMint
class node_chroma_store,node_structured_log,node_abap_pdf toneRose
class node_evaluate_py,node_tools_py toneIndigo
```

## Components

### Entry points

Command-line and launch surfaces

- [App](https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/app.py): entrypoint · Alternate launcher or wrapper entrypoint · `app.py`
- [Main loop](https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/main.py): controller · Interactive CLI controller and orchestration boundary · `main.py`
- **Streamed answer**: terminal output · Token stream returned to the CLI

### RAG pipeline

Ingestion, retrieval, and prompt assembly

- [RAG engine](https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/rag_engine.py): memory core · PDF ingestion, chunking, embeddings, persistence, and retrieval · `rag_engine.py`
- [Prompt builder](https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/rag.py): augmentation · Merges retrieved context with the user question · `rag.py`
- **Ingest flow**: pipeline · Text extraction, splitting, and embedding preparation
- **Search flow**: pipeline · Semantic lookup of relevant chunks for a query

### Model runtime

Local generation backend

- **Llama3**: local LLM · Model loaded through the local Ollama service
- **Ollama**: model host · Local token-streaming runtime used for generation

### Knowledge store

Persistent semantic memory and outputs

- **Chroma DB**: vector store · Persistent chunk and metadata storage for long-term memory
- [Research log](https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/research_output.txt): telemetry · Structured response timing and result logging · `research_output.txt`
- [Sample PDF](https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/ABAP_New.pdf): source document · Example knowledge document used to seed retrieval · `ABAP_New.pdf`

### Support

Utilities, evaluation, and shared helpers

- [Evaluate](https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/evaluate.py): analysis · Benchmarking or response-quality analysis utility · `evaluate.py`
- [Tools](https://github.com/abhi-sys511/llama3-rag-knowledge-assistant/blob/main/tools.py): helpers · Shared utility helpers for file, text, or logging tasks · `tools.py`

## Connections

- App → Main loop: launches · Alternate entrypoint delegates into the CLI controller
- Main loop → RAG engine: ingest/retrieve · Controller calls the memory subsystem for indexing and search
- Main loop → Prompt builder: build prompt · Controller passes retrieved context into prompt augmentation
- Main loop → Ollama: generate · Controller sends the final prompt to the local model runtime
- Ollama → Llama3: hosts · Ollama loads and serves the Llama3 model locally
- Main loop → Streamed answer: streams · Generated tokens are emitted back to the terminal
- Main loop → Research log: logs · Query, response, and timing are recorded as telemetry
- Sample PDF → Ingest flow: source · PDF content is the document input for ingestion
- Ingest flow → RAG engine: indexes · Ingestion pipeline is implemented by the RAG engine
- RAG engine → Chroma DB: persists · Embedded chunks are stored in the vector database
- Main loop → Search flow: asks · User questions trigger retrieval against the knowledge base
- Search flow → RAG engine: searches · Semantic search returns the most relevant chunks
- RAG engine → Chroma DB: queries · Retrieval reads from persisted embeddings and metadata
- Search flow → Prompt builder: supplies context · Top matches are fed into prompt construction
- Prompt builder → Ollama: format prompt · Augmented prompt is handed to the local model host
- Evaluate → Research log: analyzes · Evaluation likely consumes stored run outputs and timings
- Tools → Main loop: supports · Shared helper functions support the CLI workflow
- Tools → RAG engine: supports · Utilities assist file and text handling in the memory pipeline
- Tools → Evaluate: supports · Utilities may be reused by analysis workflows
