# venkatkollu/localknowledge architecture

> Architecture diagram of the GitHub repository venkatkollu/localknowledge, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/venkatkollu/localknowledge
- Repository: https://github.com/venkatkollu/localknowledge

## Overview

Local KnowledgeOS is a privacy-preserving, local-first document RAG system. Its main workflow is authenticated document ingestion and indexing, followed by access-controlled hybrid retrieval, evidence selection, answer generation, and citation checking. The graph also preserves index lifecycle, Git ingestion, security, observability, and evaluation capabilities described in the README. Several connections are documented rather than visible in sampled implementation excerpts; those use the README as evidence.

Main entry points: [`app/api.py`](https://github.com/venkatkollu/localknowledge/blob/main/app/api.py) (FastAPI), [`app/retrieval.py`](https://github.com/venkatkollu/localknowledge/blob/main/app/retrieval.py) (Query Pipeline), [`app/ingestion.py`](https://github.com/venkatkollu/localknowledge/blob/main/app/ingestion.py) (Ingestion Pipeline), [`app/generation.py`](https://github.com/venkatkollu/localknowledge/blob/main/app/generation.py) (Answer Generation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["API and Jobs"]
  node_api["FastAPI<br/>[api.py]"]
  node_jobs["Ingestion Jobs<br/>[jobs.py]"]
end

subgraph group_ingest["Document Ingestion"]
  node_ingest["Ingestion Pipeline<br/>[ingestion.py]"]
  node_repositories["Git Ingestion<br/>[repositories.py]"]
  node_parsing["Document Parsing<br/>[parsing.py]"]
  node_chunking["Structure and Chunks<br/>[chunking.py]"]
  node_embedding["Embedding Models<br/>[embeddings.py]"]
end

subgraph group_answer["Retrieval and Answers"]
  node_retrieval["Query Pipeline<br/>[retrieval.py]"]
  node_reranking["Reranking<br/>[reranking.py]"]
  node_evidence["Evidence Decisions<br/>[evidence.py]"]
  node_generation["Answer Generation<br/>[generation.py]"]
  node_citations["Citation Validation<br/>[citations.py]"]
end

subgraph group_platform["State and Security"]
  node_config["Configuration<br/>[config.py]"]
  node_storage[("SQLite and FTS5<br/>[storage.py]")]
  node_security["Access and Screening<br/>[security.py]"]
  node_auth["Authentication<br/>[auth.py]"]
end

subgraph group_quality["Evaluation and Tracing"]
  node_evaluation["RAG Evaluation<br/>[evaluation.py]"]
  node_metrics["Retrieval Metrics<br/>[metrics.py]"]
  node_observability["Traces and Metrics<br/>[observability.py]"]
end

node_user(("User"))
node_ollama["Ollama Models"]

node_user -->|"submits requests"| node_api
node_api -->|"authenticates"| node_auth
node_api -->|"submits ingestion"| node_jobs
node_jobs -.->|"runs jobs"| node_ingest
node_api -->|"routes queries"| node_retrieval
node_api -->|"manages indexes"| node_storage
node_api -->|"starts Git ingestion"| node_repositories
node_ingest -->|"parses documents"| node_parsing
node_ingest -->|"structures chunks"| node_chunking
node_ingest -->|"creates vectors"| node_embedding
node_ingest -->|"contextualizes chunks"| node_generation
node_ingest -->|"writes index"| node_storage
node_repositories -.->|"supplies repository files"| node_ingest
node_retrieval -->|"embeds queries"| node_embedding
node_retrieval -->|"searches index"| node_storage
node_retrieval -->|"filters access"| node_security
node_retrieval -->|"reranks candidates"| node_reranking
node_retrieval -->|"selects evidence"| node_evidence
node_retrieval -->|"requests answer"| node_generation
node_generation -.->|"calls local model"| node_ollama
node_generation -.->|"validates answer"| node_citations
node_evidence -->|"supports checking"| node_citations
node_api -.->|"loads settings"| node_config
node_auth -->|"resolves principal"| node_security
node_api -->|"runs evaluation"| node_evaluation
node_evaluation -->|"checks claims"| node_citations
node_evaluation -->|"computes measures"| node_metrics
node_api -->|"records traces"| node_observability
node_api -->|"reads trace data"| node_storage

click node_api "https://github.com/venkatkollu/localknowledge/blob/main/app/api.py"
click node_jobs "https://github.com/venkatkollu/localknowledge/blob/main/app/jobs.py"
click node_ingest "https://github.com/venkatkollu/localknowledge/blob/main/app/ingestion.py"
click node_repositories "https://github.com/venkatkollu/localknowledge/blob/main/app/repositories.py"
click node_parsing "https://github.com/venkatkollu/localknowledge/blob/main/app/parsing.py"
click node_chunking "https://github.com/venkatkollu/localknowledge/blob/main/app/chunking.py"
click node_embedding "https://github.com/venkatkollu/localknowledge/blob/main/app/embeddings.py"
click node_retrieval "https://github.com/venkatkollu/localknowledge/blob/main/app/retrieval.py"
click node_reranking "https://github.com/venkatkollu/localknowledge/blob/main/app/reranking.py"
click node_evidence "https://github.com/venkatkollu/localknowledge/blob/main/app/evidence.py"
click node_generation "https://github.com/venkatkollu/localknowledge/blob/main/app/generation.py"
click node_citations "https://github.com/venkatkollu/localknowledge/blob/main/app/citations.py"
click node_config "https://github.com/venkatkollu/localknowledge/blob/main/app/config.py"
click node_storage "https://github.com/venkatkollu/localknowledge/blob/main/app/storage.py"
click node_security "https://github.com/venkatkollu/localknowledge/blob/main/app/security.py"
click node_auth "https://github.com/venkatkollu/localknowledge/blob/main/app/auth.py"
click node_evaluation "https://github.com/venkatkollu/localknowledge/blob/main/app/evaluation.py"
click node_metrics "https://github.com/venkatkollu/localknowledge/blob/main/eval/metrics.py"
click node_observability "https://github.com/venkatkollu/localknowledge/blob/main/app/observability.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_api,node_jobs,node_user toneBlue
class node_ingest,node_repositories,node_parsing,node_chunking,node_embedding toneAmber
class node_retrieval,node_reranking,node_evidence,node_generation,node_citations toneMint
class node_config,node_storage,node_security,node_auth toneRose
class node_evaluation,node_metrics,node_observability,node_ollama toneIndigo
```

## Components

### API and Jobs

- [FastAPI](https://github.com/venkatkollu/localknowledge/blob/main/app/api.py): `app/api.py`
- [Ingestion Jobs](https://github.com/venkatkollu/localknowledge/blob/main/app/jobs.py): `app/jobs.py`

### Document Ingestion

- [Ingestion Pipeline](https://github.com/venkatkollu/localknowledge/blob/main/app/ingestion.py): `app/ingestion.py`
- [Git Ingestion](https://github.com/venkatkollu/localknowledge/blob/main/app/repositories.py): `app/repositories.py`
- [Document Parsing](https://github.com/venkatkollu/localknowledge/blob/main/app/parsing.py): `app/parsing.py`
- [Structure and Chunks](https://github.com/venkatkollu/localknowledge/blob/main/app/chunking.py): `app/chunking.py`
- [Embedding Models](https://github.com/venkatkollu/localknowledge/blob/main/app/embeddings.py): `app/embeddings.py`

### Retrieval and Answers

- [Query Pipeline](https://github.com/venkatkollu/localknowledge/blob/main/app/retrieval.py): `app/retrieval.py`
- [Reranking](https://github.com/venkatkollu/localknowledge/blob/main/app/reranking.py): `app/reranking.py`
- [Evidence Decisions](https://github.com/venkatkollu/localknowledge/blob/main/app/evidence.py): `app/evidence.py`
- [Answer Generation](https://github.com/venkatkollu/localknowledge/blob/main/app/generation.py): `app/generation.py`
- [Citation Validation](https://github.com/venkatkollu/localknowledge/blob/main/app/citations.py): `app/citations.py`

### State and Security

- [Configuration](https://github.com/venkatkollu/localknowledge/blob/main/app/config.py): `app/config.py`
- [SQLite and FTS5](https://github.com/venkatkollu/localknowledge/blob/main/app/storage.py): `app/storage.py`
- [Access and Screening](https://github.com/venkatkollu/localknowledge/blob/main/app/security.py): `app/security.py`
- [Authentication](https://github.com/venkatkollu/localknowledge/blob/main/app/auth.py): `app/auth.py`

### Evaluation and Tracing

- [RAG Evaluation](https://github.com/venkatkollu/localknowledge/blob/main/app/evaluation.py): `app/evaluation.py`
- [Retrieval Metrics](https://github.com/venkatkollu/localknowledge/blob/main/eval/metrics.py): `eval/metrics.py`
- [Traces and Metrics](https://github.com/venkatkollu/localknowledge/blob/main/app/observability.py): `app/observability.py`

### Other components

- **User**
- **Ollama Models**

## Connections

- User → FastAPI: submits requests (evidence: `README.md`)
- FastAPI → Authentication: authenticates (evidence: `app/api.py`)
- FastAPI → Ingestion Jobs: submits ingestion (evidence: `app/api.py`)
- Ingestion Jobs → Ingestion Pipeline: runs jobs
- FastAPI → Query Pipeline: routes queries (evidence: `README.md`)
- FastAPI → SQLite and FTS5: manages indexes (evidence: `app/api.py`)
- FastAPI → Git Ingestion: starts Git ingestion (evidence: `README.md`)
- Ingestion Pipeline → Document Parsing: parses documents (evidence: `app/ingestion.py`)
- Ingestion Pipeline → Structure and Chunks: structures chunks (evidence: `app/ingestion.py`)
- Ingestion Pipeline → Embedding Models: creates vectors (evidence: `README.md`)
- Ingestion Pipeline → Answer Generation: contextualizes chunks (evidence: `app/ingestion.py`)
- Ingestion Pipeline → SQLite and FTS5: writes index (evidence: `app/ingestion.py`)
- Git Ingestion → Ingestion Pipeline: supplies repository files
- Query Pipeline → Embedding Models: embeds queries (evidence: `app/retrieval.py`)
- Query Pipeline → SQLite and FTS5: searches index (evidence: `README.md`)
- Query Pipeline → Access and Screening: filters access (evidence: `README.md`)
- Query Pipeline → Reranking: reranks candidates (evidence: `app/retrieval.py`)
- Query Pipeline → Evidence Decisions: selects evidence (evidence: `README.md`)
- Query Pipeline → Answer Generation: requests answer (evidence: `README.md`)
- Answer Generation → Ollama Models: calls local model (evidence: `README.md`)
- Answer Generation → Citation Validation: validates answer (evidence: `README.md`)
- Evidence Decisions → Citation Validation: supports checking (evidence: `app/evidence.py`)
- FastAPI → Configuration: loads settings (evidence: `app/api.py`)
- Authentication → Access and Screening: resolves principal (evidence: `app/auth.py`)
- FastAPI → RAG Evaluation: runs evaluation (evidence: `README.md`)
- RAG Evaluation → Citation Validation: checks claims (evidence: `app/evaluation.py`)
- RAG Evaluation → Retrieval Metrics: computes measures (evidence: `app/evaluation.py`)
- FastAPI → Traces and Metrics: records traces (evidence: `app/api.py`)
- FastAPI → SQLite and FTS5: reads trace data (evidence: `app/api.py`)
