# yash2010/ai_rag architecture

> Architecture diagram of the GitHub repository yash2010/ai_rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yash2010/ai_rag
- Repository: https://github.com/yash2010/ai_rag

## Overview

#### Purpose and Entry Points
This is a local, CLI-driven retrieval-augmented-generation agent. It ingests local documents and optional URLs into a persistent vector store, then routes each chat query to document retrieval, web search, both, or direct local-model generation. Primary entry point: `main.py`; core workflow implementation: `agent.py`.

#### Runtime and Configuration
Python 3.10+ runs the application. Settings are centralized in `config.yaml` and loaded through `config.py`, including model names, retrieval chunking, storage locations, search provider, and routing defaults. The process depends on a locally running Ollama service for Llama3 generation and `nomic-embed-text` embeddings.

#### Ingestion and Knowledge Boundary
Ingestion is invoked through `python main.py --ingest`. Source material resides under `docs/`, currently including `docs/test.txt`; supported sources also include PDFs, DOCX, and URL lists. Content is parsed, chunked, embedded, and persisted in Chroma at `data/chroma/`. This creates the local knowledge boundary: documents become searchable semantic chunks rather than being sent to a cloud LLM.

#### Stateful Agent Graph
`agent.py` defines the LangGraph-based workflow around a shared `AgentState`. Its architecture is a router followed by conditional stages:
- router classifies a query as `rag`, `web`, `both`, or `direct`;
- retrieval queries Chroma for relevant local chunks;
- web search fetches current external evidence;
- generation synthesizes an answer and citations.
The graph owns transitions; individual nodes read and update shared state.

#### Retrieval and Persistence
Chroma is the durable document-index backend, represented by `data/chroma/chroma.sqlite3` plus its vector index files under `data/chroma/52dc6bd8-1df5-4af8-9357-daadb2c3258a/`. Retrieval returns the configured top-k chunks to the generation stage. Conversation history is intended to persist in the configured SQLite memory database under `data/`, enabling continuity across CLI sessions.

#### External Services and Trust Boundaries
Ollama is a local inference dependency, keeping LLM prompts and embeddings on the machine. Web retrieval crosses the external boundary through the configured search provider: Tavily, SerpAPI, or DuckDuckGo; paid/keyed providers use environment-supplied API credentials. Web results and local retrieved chunks converge only at answer generation, where source citations are produced.

#### User Interaction
Without `--ingest`, `main.py` starts an interactive terminal session. It accepts natural-language questions and session commands such as reset, help, and quit. The CLI exposes the selected strategy, retrieval status, cited sources, and final generated response while delegating all decision logic to the agent graph in `agent.py`.

Main entry points: [`main.py`](https://github.com/yash2010/ai_rag/blob/main/main.py) (Interactive CLI), [`agent.py`](https://github.com/yash2010/ai_rag/blob/main/agent.py) (Answer generation), [`config.py`](https://github.com/yash2010/ai_rag/blob/main/config.py) (Settings loader), [`config.yaml`](https://github.com/yash2010/ai_rag/blob/main/config.yaml) (Application settings)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Local Runtime"]
  node_cli["Interactive CLI<br/>Python entry point<br/>[main.py]"]
  node_settings["Settings loader<br/>configuration<br/>[config.py]"]
  node_config["Application settings<br/>YAML configuration<br/>[config.yaml]"]
  node_ollama{{"Ollama<br/>local model service"}}
end

subgraph group_agent["Agent Workflow"]
  node_agent_graph{{"LangGraph workflow<br/>agent orchestrator<br/>[agent.py]"}}
  node_agent_state["AgentState<br/>shared workflow state<br/>[agent.py]"]
  node_router["Query router<br/>workflow stage<br/>[agent.py]"]
  node_local_retrieval["Local retrieval<br/>workflow stage<br/>[agent.py]"]
  node_web_search["Web search<br/>workflow stage<br/>[agent.py]"]
  node_generation["Answer generation<br/>workflow stage<br/>[agent.py]"]
end

subgraph group_knowledge["Knowledge &amp; Memory"]
  node_source_docs["Local source documents<br/>ingestion input"]
  node_chroma[("Chroma vector store<br/>durable vector database")]
  node_session_memory[("Conversation memory<br/>SQLite session store")]
end

subgraph group_external["External Retrieval"]
  node_search_provider{{"Search provider<br/>external API"}}
end

node_config -->|"loaded by"| node_settings
node_settings -->|"configures"| node_cli
node_cli -->|"--ingest reads"| node_source_docs
node_source_docs -->|"parse, chunk, embed, persist"| node_chroma
node_chroma -->|"embeddings generated by"| node_ollama
node_cli -->|"submits query"| node_agent_graph
node_cli -->|"loads and persists sessions"| node_session_memory
node_agent_graph -->|"maintains"| node_agent_state
node_agent_graph -->|"starts with"| node_router
node_router -->|"rag or both"| node_local_retrieval
node_router -->|"web or both"| node_web_search
node_router -->|"direct"| node_generation
node_local_retrieval -->|"queries chunks"| node_chroma
node_local_retrieval -->|"local evidence"| node_generation
node_web_search -->|"requests results"| node_search_provider
node_web_search -->|"web evidence"| node_generation
node_generation -->|"generates with"| node_ollama
node_generation -->|"answer and citations"| node_cli

click node_cli "https://github.com/yash2010/ai_rag/blob/main/main.py"
click node_settings "https://github.com/yash2010/ai_rag/blob/main/config.py"
click node_config "https://github.com/yash2010/ai_rag/blob/main/config.yaml"
click node_agent_graph "https://github.com/yash2010/ai_rag/blob/main/agent.py"
click node_agent_state "https://github.com/yash2010/ai_rag/blob/main/agent.py"
click node_router "https://github.com/yash2010/ai_rag/blob/main/agent.py"
click node_local_retrieval "https://github.com/yash2010/ai_rag/blob/main/agent.py"
click node_web_search "https://github.com/yash2010/ai_rag/blob/main/agent.py"
click node_generation "https://github.com/yash2010/ai_rag/blob/main/agent.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_settings,node_config,node_ollama toneBlue
class node_agent_graph,node_agent_state,node_router,node_local_retrieval,node_web_search,node_generation toneAmber
class node_source_docs,node_chroma,node_session_memory toneMint
class node_search_provider toneRose
```

## Components

### Local Runtime

CLI process and local inference dependencies.

- [Interactive CLI](https://github.com/yash2010/ai_rag/blob/main/main.py): Python entry point · Starts chat or ingestion from command-line flags. · `main.py`
- [Settings loader](https://github.com/yash2010/ai_rag/blob/main/config.py): configuration · Loads model, storage, routing, and search settings. · `config.py`
- [Application settings](https://github.com/yash2010/ai_rag/blob/main/config.yaml): YAML configuration · `config.yaml`
- **Ollama**: local model service · Provides Llama3 generation and nomic-embed-text embeddings locally.

### Agent Workflow

LangGraph stages operating on shared state.

- [LangGraph workflow](https://github.com/yash2010/ai_rag/blob/main/agent.py): agent orchestrator · Owns conditional transitions over AgentState. · `agent.py`
- [AgentState](https://github.com/yash2010/ai_rag/blob/main/agent.py): shared workflow state · Carries query, strategy, evidence, and answer between stages. · `agent.py`
- [Query router](https://github.com/yash2010/ai_rag/blob/main/agent.py): workflow stage · Selects rag, web, both, or direct generation. · `agent.py`
- [Local retrieval](https://github.com/yash2010/ai_rag/blob/main/agent.py): workflow stage · Returns configured top-k semantic chunks. · `agent.py`
- [Web search](https://github.com/yash2010/ai_rag/blob/main/agent.py): workflow stage · Fetches current external evidence when routed. · `agent.py`
- [Answer generation](https://github.com/yash2010/ai_rag/blob/main/agent.py): workflow stage · Synthesizes response and source citations. · `agent.py`

### Knowledge & Memory

Persistent local retrieval and session storage.

- **Local source documents**: ingestion input · Text, PDF, DOCX, and URL-list sources.
- **Chroma vector store**: durable vector database · Persists embedded document chunks.
- **Conversation memory**: SQLite session store · Configured persistence for continuity across CLI sessions.

### External Retrieval

Optional internet-facing search boundary.

- **Search provider**: external API · Tavily, SerpAPI, or DuckDuckGo; keyed providers use environment credentials.

## Connections

- Application settings → Settings loader: loaded by
- Settings loader → Interactive CLI: configures
- Interactive CLI → Local source documents: --ingest reads
- Local source documents → Chroma vector store: parse, chunk, embed, persist
- Chroma vector store → Ollama: embeddings generated by
- Interactive CLI → LangGraph workflow: submits query
- Interactive CLI → Conversation memory: loads and persists sessions
- LangGraph workflow → AgentState: maintains
- LangGraph workflow → Query router: starts with
- Query router → Local retrieval: rag or both
- Query router → Web search: web or both
- Query router → Answer generation: direct
- Local retrieval → Chroma vector store: queries chunks
- Local retrieval → Answer generation: local evidence
- Web search → Search provider: requests results
- Web search → Answer generation: web evidence
- Answer generation → Ollama: generates with
- Answer generation → Interactive CLI: answer and citations