yash2010/ai_rag

Purpose and Entry Points

This is a local, CLI-driven retrieval-augmented-generation agent. It ingests local documents and optional URLs into a persistent vector store, then routes each chat query to document retrieval, web search, both, or direct local-model generation. Primary entry point: main.py; core workflow implementation: agent.py.

Runtime and Configuration

Python 3.10+ runs the application. Settings are centralized in config.yaml and loaded through config.py, including model names, retrieval chunking, storage locations, search provider, and routing defaults. The process depends on a locally running Ollama service for Llama3 generation and nomic-embed-text embeddings.

Ingestion and Knowledge Boundary

Ingestion is invoked through python main.py --ingest. Source material resides under docs/, currently including docs/test.txt; supported sources also include PDFs, DOCX, and URL lists. Content is parsed, chunked, embedded, and persisted in Chroma at data/chroma/. This creates the local knowledge boundary: documents become searchable semantic chunks rather than being sent to a cloud LLM.

Stateful Agent Graph

agent.py defines the LangGraph-based workflow around a shared AgentState. Its architecture is a router followed by conditional stages:

• router classifies a query as rag, web, both, or direct;

• retrieval queries Chroma for relevant local chunks;

• web search fetches current external evidence;

• generation synthesizes an answer and citations.

The graph owns transitions; individual nodes read and update shared state.

Retrieval and Persistence

Chroma is the durable document-index backend, represented by data/chroma/chroma.sqlite3 plus its vector index files under data/chroma/52dc6bd8-1df5-4af8-9357-daadb2c3258a/. Retrieval returns the configured top-k chunks to the generation stage. Conversation history is intended to persist in the configured SQLite memory database under data/, enabling continuity across CLI sessions.

External Services and Trust Boundaries

Ollama is a local inference dependency, keeping LLM prompts and embeddings on the machine. Web retrieval crosses the external boundary through the configured search provider: Tavily, SerpAPI, or DuckDuckGo; paid/keyed providers use environment-supplied API credentials. Web results and local retrieved chunks converge only at answer generation, where source citations are produced.

User Interaction

Without --ingest, main.py starts an interactive terminal session. It accepts natural-language questions and session commands such as reset, help, and quit. The CLI exposes the selected strategy, retrieval status, cited sources, and final generated response while delegating all decision logic to the agent graph in agent.py.