# kingmaker1234/tn-agri-rag architecture

> Architecture diagram of the GitHub repository kingmaker1234/tn-agri-rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kingmaker1234/tn-agri-rag
- Repository: https://github.com/kingmaker1234/tn-agri-rag

## Overview

Uzhavan Assist is a Tamil Nadu farm-scheme question-answering app: users ask in Tamil, English, or Tanglish and receive answers grounded in government documents with citations and retrieval traces. Its main workflow runs from the browser’s Ask page through the API, query planning, hybrid retrieval and reranking, then answer generation. A separate ingestion workflow accepts files and URLs, extracts and chunks content, enriches it with document context, embeds it, and indexes it. The README also documents corpus discovery, comparison, library, source, and system views. Unsampled API wiring is shown conservatively.

Main entry points: [`backend/app/main.py`](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/main.py) (FastAPI routes), [`backend/app/ingest/pipeline.py`](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/ingest/pipeline.py) (Ingest pipeline), [`backend/app/retrieval/hybrid.py`](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/retrieval/hybrid.py) (Hybrid retrieval), [`frontend/src/App.tsx`](https://github.com/kingmaker1234/tn-agri-rag/blob/main/frontend/src/App.tsx) (Navigation shell)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_web["Web experience"]
  node_app["Navigation shell<br/>[App.tsx]"]
  node_ask["Ask and citations<br/>[Ask.tsx]"]
  node_explore["Retrieval comparison<br/>[Explore.tsx]"]
  node_ingestui["Ingest controls<br/>[Ingest.tsx]"]
  node_library["Document library<br/>[Library.tsx]"]
  node_sources["Source catalog<br/>[Sources.tsx]"]
  node_system["System status<br/>[System.tsx]"]
  node_client["Typed API client<br/>[api.ts]"]
end

subgraph group_api["API orchestration"]
  node_api_server["FastAPI routes<br/>[main.py]"]
  node_jobs["Ingestion jobs<br/>[jobs.py]"]
end

subgraph group_ingest["Corpus ingestion"]
  node_crawler["Source crawler<br/>[crawler.py]"]
  node_loader["File loaders<br/>[loaders.py]"]
  node_splitter["Chunk splitter<br/>[splitter.py]"]
  node_context["Document context<br/>[doc_context.py]"]
  node_pipeline["Ingest pipeline<br/>[pipeline.py]"]
end

subgraph group_retrieve["Retrieval and answers"]
  node_planner["Query planner<br/>[query_planner.py]"]
  node_hybrid["Hybrid retrieval<br/>[hybrid.py]"]
  node_reranker["Passage reranker<br/>[reranker.py]"]
  node_generator["Grounded answer<br/>[generator.py]"]
end

subgraph group_state["Index and operations"]
  node_embedder["Embedding backends<br/>[embeddings.py]"]
  node_store[("FAISS and SQLite index<br/>[vector_store.py]")]
  node_bm25[("BM25 index<br/>[bm25.py]")]
  node_telemetry["Usage and telemetry<br/>[telemetry.py]"]
  node_usage["Token cost accounting<br/>[usage.py]"]
  node_text["Tamil text utilities<br/>[text_utils.py]"]
end

node_farmer(("Farmer or admin"))
node_openai{{"OpenAI-compatible model API"}}

node_farmer -->|"uses"| node_app
node_app -->|"routes to"| node_ask
node_app -->|"routes to"| node_explore
node_app -->|"routes to"| node_ingestui
node_app -->|"routes to"| node_library
node_app -->|"routes to"| node_sources
node_app -->|"routes to"| node_system
node_ask -.->|"requests"| node_client
node_explore -.->|"requests"| node_client
node_ingestui -.->|"requests"| node_client
node_library -.->|"requests"| node_client
node_sources -.->|"requests"| node_client
node_system -.->|"requests"| node_client
node_client -->|"HTTP requests"| node_api_server
node_api_server -.->|"searches"| node_hybrid
node_api_server -.->|"starts ingestion"| node_pipeline
node_api_server -.->|"manages jobs"| node_jobs
node_api_server -.->|"discovers sources"| node_crawler
node_api_server -.->|"serves documents"| node_store
node_api_server -.->|"reports status"| node_telemetry
node_pipeline -->|"ingests discovered"| node_crawler
node_pipeline -->|"loads files"| node_loader
node_pipeline -->|"splits sections"| node_splitter
node_pipeline -->|"adds context"| node_context
node_pipeline -->|"embeds chunks"| node_embedder
node_pipeline -->|"indexes chunks"| node_store
node_loader -->|"cleans Tamil text"| node_text
node_context -->|"summarizes documents"| node_openai
node_hybrid -->|"plans variants"| node_planner
node_hybrid -->|"embeds queries"| node_embedder
node_hybrid -->|"dense search"| node_store
node_hybrid -.->|"keyword search"| node_bm25
node_hybrid -->|"reranks candidates"| node_reranker
node_hybrid -.->|"supplies retrieved context"| node_generator
node_planner -->|"rewrites queries"| node_openai
node_reranker -.->|"scores passages"| node_openai
node_generator -->|"generates answers"| node_openai
node_embedder -.->|"creates vectors"| node_openai
node_pipeline -.->|"records cost"| node_usage
node_hybrid -->|"records cost"| node_usage
node_api_server -.->|"reports usage"| node_usage

click node_app "https://github.com/kingmaker1234/tn-agri-rag/blob/main/frontend/src/App.tsx"
click node_ask "https://github.com/kingmaker1234/tn-agri-rag/blob/main/frontend/src/pages/Ask.tsx"
click node_explore "https://github.com/kingmaker1234/tn-agri-rag/blob/main/frontend/src/pages/Explore.tsx"
click node_ingestui "https://github.com/kingmaker1234/tn-agri-rag/blob/main/frontend/src/pages/Ingest.tsx"
click node_library "https://github.com/kingmaker1234/tn-agri-rag/blob/main/frontend/src/pages/Library.tsx"
click node_sources "https://github.com/kingmaker1234/tn-agri-rag/blob/main/frontend/src/pages/Sources.tsx"
click node_system "https://github.com/kingmaker1234/tn-agri-rag/blob/main/frontend/src/pages/System.tsx"
click node_client "https://github.com/kingmaker1234/tn-agri-rag/blob/main/frontend/src/api.ts"
click node_api_server "https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/main.py"
click node_jobs "https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/jobs.py"
click node_crawler "https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/ingest/crawler.py"
click node_loader "https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/ingest/loaders.py"
click node_splitter "https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/ingest/splitter.py"
click node_context "https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/ingest/doc_context.py"
click node_pipeline "https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/ingest/pipeline.py"
click node_planner "https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/retrieval/query_planner.py"
click node_hybrid "https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/retrieval/hybrid.py"
click node_reranker "https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/retrieval/reranker.py"
click node_generator "https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/llm/generator.py"
click node_embedder "https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/store/embeddings.py"
click node_store "https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/store/vector_store.py"
click node_bm25 "https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/store/bm25.py"
click node_telemetry "https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/telemetry.py"
click node_usage "https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/llm/usage.py"
click node_text "https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/text_utils.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_ask,node_explore,node_ingestui,node_library,node_sources,node_system,node_client toneBlue
class node_api_server,node_jobs toneAmber
class node_crawler,node_loader,node_splitter,node_context,node_pipeline,node_openai toneMint
class node_planner,node_hybrid,node_reranker,node_generator toneRose
class node_embedder,node_store,node_bm25,node_telemetry,node_usage,node_text,node_farmer toneIndigo
```

## Components

### Web experience

- [Navigation shell](https://github.com/kingmaker1234/tn-agri-rag/blob/main/frontend/src/App.tsx): `frontend/src/App.tsx`
- [Ask and citations](https://github.com/kingmaker1234/tn-agri-rag/blob/main/frontend/src/pages/Ask.tsx): `frontend/src/pages/Ask.tsx`
- [Retrieval comparison](https://github.com/kingmaker1234/tn-agri-rag/blob/main/frontend/src/pages/Explore.tsx): `frontend/src/pages/Explore.tsx`
- [Ingest controls](https://github.com/kingmaker1234/tn-agri-rag/blob/main/frontend/src/pages/Ingest.tsx): `frontend/src/pages/Ingest.tsx`
- [Document library](https://github.com/kingmaker1234/tn-agri-rag/blob/main/frontend/src/pages/Library.tsx): `frontend/src/pages/Library.tsx`
- [Source catalog](https://github.com/kingmaker1234/tn-agri-rag/blob/main/frontend/src/pages/Sources.tsx): `frontend/src/pages/Sources.tsx`
- [System status](https://github.com/kingmaker1234/tn-agri-rag/blob/main/frontend/src/pages/System.tsx): `frontend/src/pages/System.tsx`
- [Typed API client](https://github.com/kingmaker1234/tn-agri-rag/blob/main/frontend/src/api.ts): `frontend/src/api.ts`

### API orchestration

- [FastAPI routes](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/main.py): `backend/app/main.py`
- [Ingestion jobs](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/jobs.py): `backend/app/jobs.py`

### Corpus ingestion

- [Source crawler](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/ingest/crawler.py): `backend/app/ingest/crawler.py`
- [File loaders](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/ingest/loaders.py): `backend/app/ingest/loaders.py`
- [Chunk splitter](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/ingest/splitter.py): `backend/app/ingest/splitter.py`
- [Document context](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/ingest/doc_context.py): `backend/app/ingest/doc_context.py`
- [Ingest pipeline](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/ingest/pipeline.py): `backend/app/ingest/pipeline.py`

### Retrieval and answers

- [Query planner](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/retrieval/query_planner.py): `backend/app/retrieval/query_planner.py`
- [Hybrid retrieval](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/retrieval/hybrid.py): `backend/app/retrieval/hybrid.py`
- [Passage reranker](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/retrieval/reranker.py): `backend/app/retrieval/reranker.py`
- [Grounded answer](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/llm/generator.py): `backend/app/llm/generator.py`

### Index and operations

- [Embedding backends](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/store/embeddings.py): `backend/app/store/embeddings.py`
- [FAISS and SQLite index](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/store/vector_store.py): `backend/app/store/vector_store.py`
- [BM25 index](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/store/bm25.py): `backend/app/store/bm25.py`
- [Usage and telemetry](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/telemetry.py): `backend/app/telemetry.py`
- [Token cost accounting](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/llm/usage.py): `backend/app/llm/usage.py`
- [Tamil text utilities](https://github.com/kingmaker1234/tn-agri-rag/blob/main/backend/app/text_utils.py): `backend/app/text_utils.py`

### Other components

- **Farmer or admin**
- **OpenAI-compatible model API**

## Connections

- Farmer or admin → Navigation shell: uses
- Navigation shell → Ask and citations: routes to (evidence: `frontend/src/App.tsx`)
- Navigation shell → Retrieval comparison: routes to (evidence: `frontend/src/App.tsx`)
- Navigation shell → Ingest controls: routes to (evidence: `frontend/src/App.tsx`)
- Navigation shell → Document library: routes to (evidence: `frontend/src/App.tsx`)
- Navigation shell → Source catalog: routes to (evidence: `frontend/src/App.tsx`)
- Navigation shell → System status: routes to (evidence: `frontend/src/App.tsx`)
- Ask and citations → Typed API client: requests (evidence: `frontend/src/pages/Ask.tsx`)
- Retrieval comparison → Typed API client: requests (evidence: `frontend/src/pages/Explore.tsx`)
- Ingest controls → Typed API client: requests (evidence: `frontend/src/pages/Ingest.tsx`)
- Document library → Typed API client: requests (evidence: `frontend/src/pages/Library.tsx`)
- Source catalog → Typed API client: requests (evidence: `frontend/src/pages/Sources.tsx`)
- System status → Typed API client: requests (evidence: `frontend/src/pages/System.tsx`)
- Typed API client → FastAPI routes: HTTP requests (evidence: `frontend/src/api.ts`)
- FastAPI routes → Hybrid retrieval: searches (evidence: `backend/app/main.py`)
- FastAPI routes → Ingest pipeline: starts ingestion (evidence: `backend/app/main.py`)
- FastAPI routes → Ingestion jobs: manages jobs (evidence: `backend/app/main.py`)
- FastAPI routes → Source crawler: discovers sources (evidence: `backend/app/main.py`)
- FastAPI routes → FAISS and SQLite index: serves documents (evidence: `backend/app/main.py`)
- FastAPI routes → Usage and telemetry: reports status (evidence: `backend/app/main.py`)
- Ingest pipeline → Source crawler: ingests discovered (evidence: `backend/app/ingest/pipeline.py`)
- Ingest pipeline → File loaders: loads files (evidence: `backend/app/ingest/pipeline.py`)
- Ingest pipeline → Chunk splitter: splits sections (evidence: `backend/app/ingest/pipeline.py`)
- Ingest pipeline → Document context: adds context (evidence: `backend/app/ingest/pipeline.py`)
- Ingest pipeline → Embedding backends: embeds chunks (evidence: `backend/app/ingest/pipeline.py`)
- Ingest pipeline → FAISS and SQLite index: indexes chunks (evidence: `backend/app/ingest/pipeline.py`)
- File loaders → Tamil text utilities: cleans Tamil text (evidence: `backend/app/ingest/loaders.py`)
- Document context → OpenAI-compatible model API: summarizes documents (evidence: `backend/app/ingest/doc_context.py`)
- Hybrid retrieval → Query planner: plans variants (evidence: `backend/app/retrieval/hybrid.py`)
- Hybrid retrieval → Embedding backends: embeds queries (evidence: `backend/app/retrieval/hybrid.py`)
- Hybrid retrieval → FAISS and SQLite index: dense search (evidence: `backend/app/retrieval/hybrid.py`)
- Hybrid retrieval → BM25 index: keyword search
- Hybrid retrieval → Passage reranker: reranks candidates (evidence: `backend/app/retrieval/hybrid.py`)
- Hybrid retrieval → Grounded answer: supplies retrieved context
- Query planner → OpenAI-compatible model API: rewrites queries (evidence: `backend/app/retrieval/query_planner.py`)
- Passage reranker → OpenAI-compatible model API: scores passages
- Grounded answer → OpenAI-compatible model API: generates answers (evidence: `backend/app/llm/generator.py`)
- Embedding backends → OpenAI-compatible model API: creates vectors
- Ingest pipeline → Token cost accounting: records cost (evidence: `backend/app/ingest/pipeline.py`)
- Hybrid retrieval → Token cost accounting: records cost (evidence: `backend/app/retrieval/hybrid.py`)
- FastAPI routes → Token cost accounting: reports usage (evidence: `backend/app/main.py`)
