# harsha1509/ai_project_1 architecture

> Architecture diagram of the GitHub repository harsha1509/ai_project_1, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/harsha1509/ai_project_1
- Repository: https://github.com/harsha1509/ai_project_1

## Overview

This repository implements a local PDF question-answering system: PDF text is converted into chunks and embeddings stored in SQLite, then user questions retrieve relevant passages and produce citation-checked answers. A project-aware ingestion path preserves project and section metadata for targeted resource answers. The graph includes the separately documented evaluation capability; several modules are indexed but unsampled, so their internal relationships are omitted unless supported by shown imports or calls.

Main entry points: [`production_rag/app/rag_pipeline.py`](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/rag_pipeline.py) (RAG orchestration), [`production_rag/app/vector_store.py`](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/vector_store.py) (SQLite chunk store), [`production_rag/app/ingest_documents.py`](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/ingest_documents.py) (PDF ingestion), [`production_rag/app/reingest_project_chunks.py`](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/reingest_project_chunks.py) (Project reingestion)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingestion["Document ingestion"]
  node_reader["PDF reader<br/>[document_reader.py]"]
  node_pipeline["PDF chunking"]
  node_text_chunks["Text chunker<br/>[text_chunker.py]"]
  node_project_chunks["Project chunker<br/>[project_chunker.py]"]
  node_ingest["PDF ingestion"]
  node_reingest["Project reingestion"]
end

subgraph group_retrieval["Search and storage"]
  node_embed["Embedding generator"]
  node_store[("SQLite chunk store<br/>[vector_store.py]")]
  node_semantic["Semantic search<br/>[retriever.py]"]
  node_keyword["Keyword search"]
  node_hybrid["Hybrid retrieval"]
end

subgraph group_answering["Question answering"]
  node_resource_extract["Resource extraction"]
  node_rag["RAG orchestration<br/>[rag_pipeline.py]"]
  node_generator["Answer generation"]
end

subgraph group_evaluation["Retrieval evaluation"]
  node_evaluation["RAG evaluation<br/>[evaluate_rag.py]"]
end

node_user(("Questioner"))
node_pdf["Source PDF<br/>[sample.pdf]"]
node_ollama_embed["Ollama embeddings"]
node_ollama_generate["Ollama generation"]

node_ingest -->|"reads"| node_pdf
node_ingest -->|"chunks via"| node_pipeline
node_pipeline -->|"extracts pages"| node_reader
node_pipeline -->|"splits text"| node_text_chunks
node_ingest -->|"embeds chunks"| node_embed
node_embed -->|"requests vectors"| node_ollama_embed
node_ingest -->|"saves chunks"| node_store
node_reingest -->|"reads"| node_pdf
node_reingest -->|"extracts chunks"| node_project_chunks
node_project_chunks -->|"reads pages"| node_reader
node_reingest -->|"embeds chunks"| node_embed
node_reingest -->|"replaces records"| node_store
node_user -->|"asks question"| node_rag
node_rag -->|"retrieves candidates"| node_hybrid
node_rag -->|"checks similarity"| node_semantic
node_hybrid -->|"gets vector results"| node_semantic
node_hybrid -->|"gets keyword results"| node_keyword
node_semantic -->|"embeds query"| node_embed
node_semantic -->|"reads chunks"| node_store
node_keyword -->|"reads chunks"| node_store
node_rag -->|"reads resource chunk"| node_store
node_rag -->|"extracts resources"| node_resource_extract
node_rag -->|"generates answer"| node_generator
node_generator -->|"requests completion"| node_ollama_generate
node_rag -->|"validates citations"| node_generator
node_evaluation -->|"evaluates retrieval"| node_hybrid

click node_pdf "https://github.com/harsha1509/ai_project_1/blob/main/production_rag/data/sample.pdf"
click node_reader "https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/document_reader.py"
click node_pipeline "https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/document_pipeline.py"
click node_text_chunks "https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/text_chunker.py"
click node_project_chunks "https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/project_chunker.py"
click node_ingest "https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/ingest_documents.py"
click node_reingest "https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/reingest_project_chunks.py"
click node_embed "https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/embedding_generator.py"
click node_store "https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/vector_store.py"
click node_semantic "https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/retriever.py"
click node_keyword "https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/keyword_retriever.py"
click node_hybrid "https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/hybrid_retriever.py"
click node_resource_extract "https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/resource_extractor.py"
click node_rag "https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/rag_pipeline.py"
click node_generator "https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/answer_generator.py"
click node_evaluation "https://github.com/harsha1509/ai_project_1/blob/main/production_rag/evals/evaluate_rag.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_reader,node_pipeline,node_text_chunks,node_project_chunks,node_ingest,node_reingest toneBlue
class node_embed,node_store,node_semantic,node_keyword,node_hybrid toneAmber
class node_resource_extract,node_rag,node_generator toneMint
class node_evaluation toneRose
class node_user,node_ollama_embed,node_ollama_generate toneIndigo
class node_pdf toneTeal
```

## Components

### Document ingestion

- [PDF reader](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/document_reader.py): `production_rag/app/document_reader.py`
- [PDF chunking](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/document_pipeline.py): `production_rag/app/document_pipeline.py`
- [Text chunker](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/text_chunker.py): `production_rag/app/text_chunker.py`
- [Project chunker](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/project_chunker.py): `production_rag/app/project_chunker.py`
- [PDF ingestion](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/ingest_documents.py): `production_rag/app/ingest_documents.py`
- [Project reingestion](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/reingest_project_chunks.py): `production_rag/app/reingest_project_chunks.py`

### Search and storage

- [Embedding generator](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/embedding_generator.py): `production_rag/app/embedding_generator.py`
- [SQLite chunk store](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/vector_store.py): `production_rag/app/vector_store.py`
- [Semantic search](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/retriever.py): `production_rag/app/retriever.py`
- [Keyword search](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/keyword_retriever.py): `production_rag/app/keyword_retriever.py`
- [Hybrid retrieval](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/hybrid_retriever.py): `production_rag/app/hybrid_retriever.py`

### Question answering

- [Resource extraction](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/resource_extractor.py): `production_rag/app/resource_extractor.py`
- [RAG orchestration](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/rag_pipeline.py): `production_rag/app/rag_pipeline.py`
- [Answer generation](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/app/answer_generator.py): `production_rag/app/answer_generator.py`

### Retrieval evaluation

- [RAG evaluation](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/evals/evaluate_rag.py): `production_rag/evals/evaluate_rag.py`

### Other components

- **Questioner**
- [Source PDF](https://github.com/harsha1509/ai_project_1/blob/main/production_rag/data/sample.pdf): `production_rag/data/sample.pdf`
- **Ollama embeddings**
- **Ollama generation**

## Connections

- PDF ingestion → Source PDF: reads (evidence: `production_rag/app/ingest_documents.py`)
- PDF ingestion → PDF chunking: chunks via (evidence: `production_rag/app/ingest_documents.py`)
- PDF chunking → PDF reader: extracts pages (evidence: `production_rag/app/document_pipeline.py`)
- PDF chunking → Text chunker: splits text (evidence: `production_rag/app/document_pipeline.py`)
- PDF ingestion → Embedding generator: embeds chunks (evidence: `production_rag/app/ingest_documents.py`)
- Embedding generator → Ollama embeddings: requests vectors (evidence: `production_rag/app/embedding_generator.py`)
- PDF ingestion → SQLite chunk store: saves chunks (evidence: `production_rag/app/ingest_documents.py`)
- Project reingestion → Source PDF: reads (evidence: `production_rag/app/reingest_project_chunks.py`)
- Project reingestion → Project chunker: extracts chunks (evidence: `production_rag/app/reingest_project_chunks.py`)
- Project chunker → PDF reader: reads pages (evidence: `production_rag/app/project_chunker.py`)
- Project reingestion → Embedding generator: embeds chunks (evidence: `production_rag/app/reingest_project_chunks.py`)
- Project reingestion → SQLite chunk store: replaces records (evidence: `production_rag/app/reingest_project_chunks.py`)
- Questioner → RAG orchestration: asks question (evidence: `production_rag/app/rag_pipeline.py`)
- RAG orchestration → Hybrid retrieval: retrieves candidates (evidence: `production_rag/app/rag_pipeline.py`)
- RAG orchestration → Semantic search: checks similarity (evidence: `production_rag/app/rag_pipeline.py`)
- Hybrid retrieval → Semantic search: gets vector results (evidence: `production_rag/app/hybrid_retriever.py`)
- Hybrid retrieval → Keyword search: gets keyword results (evidence: `production_rag/app/hybrid_retriever.py`)
- Semantic search → Embedding generator: embeds query (evidence: `production_rag/app/retriever.py`)
- Semantic search → SQLite chunk store: reads chunks (evidence: `production_rag/app/retriever.py`)
- Keyword search → SQLite chunk store: reads chunks (evidence: `production_rag/app/keyword_retriever.py`)
- RAG orchestration → SQLite chunk store: reads resource chunk (evidence: `production_rag/app/rag_pipeline.py`)
- RAG orchestration → Resource extraction: extracts resources (evidence: `production_rag/app/rag_pipeline.py`)
- RAG orchestration → Answer generation: generates answer (evidence: `production_rag/app/rag_pipeline.py`)
- Answer generation → Ollama generation: requests completion (evidence: `production_rag/app/answer_generator.py`)
- RAG orchestration → Answer generation: validates citations (evidence: `production_rag/app/rag_pipeline.py`)
- RAG evaluation → Hybrid retrieval: evaluates retrieval (evidence: `production_rag/evals/evaluate_rag.py`)