# darkerror64/retrivo architecture

> Architecture diagram of the GitHub repository darkerror64/retrivo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/darkerror64/retrivo
- Repository: https://github.com/darkerror64/retrivo

## Overview

PKOS ingests PDFs into structurally and semantically enriched vector chunks, then retrieves relevant passages for direct search or a multi-step ReAct answer. The API/controller and CLI provide interaction paths; an asynchronous worker supports background ingestion. The graph preserves parsing, chunking, token optimization, embedding, hybrid retrieval, reranking, and LLM capabilities. Some route and frontend wiring is not shown in the excerpts, so uncertain connections are omitted.

Main entry points: [`app/controllers/main_controller.py`](https://github.com/darkerror64/retrivo/blob/main/app/controllers/main_controller.py) (Main controller), [`app/engines/hierarchical_chunker.py`](https://github.com/darkerror64/retrivo/blob/main/app/engines/hierarchical_chunker.py) (Hierarchical chunker), [`app/db/retrieval_manager.py`](https://github.com/darkerror64/retrivo/blob/main/app/db/retrieval_manager.py) (Hybrid retrieval), [`app/engines/agent_engine.py`](https://github.com/darkerror64/retrivo/blob/main/app/engines/agent_engine.py) (ReAct agent)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["Interfaces &amp; orchestration"]
  node_cli["CLI<br/>[cli.py]"]
  node_web["Web UI"]
  node_api_server["FastAPI<br/>[main.py]"]
  node_controller["Main controller<br/>[main_controller.py]"]
  node_worker["Ingestion worker<br/>[worker.py]"]
end

subgraph group_ingest["Document ingestion"]
  node_pdf["PDF extraction<br/>[pdf_extractor.py]"]
  node_hierarchical["Hierarchical chunker"]
  node_parser["Markdown AST parser<br/>[markdown_parser.py]"]
  node_semantic["Semantic chunker &amp; embeddings"]
  node_tokens["Token optimizer<br/>[token_optimizer.py]"]
end

subgraph group_search["Retrieval &amp; ranking"]
  node_retrieval["Hybrid retrieval"]
  node_reranker["Cross-encoder reranker<br/>[reranker.py]"]
end

subgraph group_agent["Agent &amp; language models"]
  node_agent["ReAct agent<br/>[agent_engine.py]"]
  node_llm["LLM service<br/>[llm.py]"]
end

subgraph group_storage["Storage &amp; configuration"]
  node_ingestion["Ingestion manager"]
  node_database[("PostgreSQL pgvector")]
  node_config["Runtime settings<br/>[config.py]"]
end

node_user(("User"))
node_llm_provider{{"LLM provider"}}

node_user -.->|"queries"| node_cli
node_user -.->|"uses"| node_web
node_cli -->|"invokes"| node_controller
node_api_server -->|"routes through"| node_controller
node_api_server -->|"dispatches async jobs"| node_worker
node_controller -->|"extracts PDF"| node_pdf
node_controller -->|"ingests document"| node_ingestion
node_controller -->|"queries chunks"| node_retrieval
node_controller -->|"routes agent query"| node_agent
node_worker -->|"extracts PDF"| node_pdf
node_worker -->|"ingests document"| node_ingestion
node_ingestion -->|"chunks and embeds"| node_hierarchical
node_hierarchical -->|"parses structure"| node_parser
node_hierarchical -->|"detects boundaries"| node_semantic
node_hierarchical -->|"optimizes chunks"| node_tokens
node_hierarchical -->|"embeds chunks"| node_semantic
node_ingestion -->|"stores chunks"| node_database
node_retrieval -->|"searches vectors and text"| node_database
node_retrieval -->|"embeds queries"| node_semantic
node_retrieval -->|"reranks candidates"| node_reranker
node_agent -->|"searches documents"| node_retrieval
node_agent -->|"reads documents and chunks"| node_database
node_agent -->|"requests reasoning"| node_llm
node_llm -->|"calls configured provider"| node_llm_provider
node_config -->|"configures provider"| node_llm

click node_cli "https://github.com/darkerror64/retrivo/blob/main/cli.py"
click node_web "https://github.com/darkerror64/retrivo/tree/main/frontend/app"
click node_api_server "https://github.com/darkerror64/retrivo/blob/main/app/main.py"
click node_controller "https://github.com/darkerror64/retrivo/blob/main/app/controllers/main_controller.py"
click node_worker "https://github.com/darkerror64/retrivo/blob/main/app/worker.py"
click node_pdf "https://github.com/darkerror64/retrivo/blob/main/app/utils/pdf_extractor.py"
click node_hierarchical "https://github.com/darkerror64/retrivo/blob/main/app/engines/hierarchical_chunker.py"
click node_parser "https://github.com/darkerror64/retrivo/blob/main/app/engines/markdown_parser.py"
click node_semantic "https://github.com/darkerror64/retrivo/blob/main/app/engines/semantic_chunker.py"
click node_tokens "https://github.com/darkerror64/retrivo/blob/main/app/utils/token_optimizer.py"
click node_ingestion "https://github.com/darkerror64/retrivo/blob/main/app/db/database_manager.py"
click node_database "https://github.com/darkerror64/retrivo/blob/main/app/db/database_manager.py"
click node_retrieval "https://github.com/darkerror64/retrivo/blob/main/app/db/retrieval_manager.py"
click node_reranker "https://github.com/darkerror64/retrivo/blob/main/app/engines/reranker.py"
click node_agent "https://github.com/darkerror64/retrivo/blob/main/app/engines/agent_engine.py"
click node_llm "https://github.com/darkerror64/retrivo/blob/main/app/core/llm.py"
click node_config "https://github.com/darkerror64/retrivo/blob/main/app/core/config.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_web,node_api_server,node_controller,node_worker,node_user toneBlue
class node_pdf,node_hierarchical,node_parser,node_semantic,node_tokens toneAmber
class node_retrieval,node_reranker toneMint
class node_agent,node_llm toneRose
class node_ingestion,node_database,node_config,node_llm_provider toneIndigo
```

## Components

### Interfaces & orchestration

- [CLI](https://github.com/darkerror64/retrivo/blob/main/cli.py): `cli.py`
- [Web UI](https://github.com/darkerror64/retrivo/tree/main/frontend/app): `frontend/app`
- [FastAPI](https://github.com/darkerror64/retrivo/blob/main/app/main.py): `app/main.py`
- [Main controller](https://github.com/darkerror64/retrivo/blob/main/app/controllers/main_controller.py): `app/controllers/main_controller.py`
- [Ingestion worker](https://github.com/darkerror64/retrivo/blob/main/app/worker.py): `app/worker.py`

### Document ingestion

- [PDF extraction](https://github.com/darkerror64/retrivo/blob/main/app/utils/pdf_extractor.py): `app/utils/pdf_extractor.py`
- [Hierarchical chunker](https://github.com/darkerror64/retrivo/blob/main/app/engines/hierarchical_chunker.py): `app/engines/hierarchical_chunker.py`
- [Markdown AST parser](https://github.com/darkerror64/retrivo/blob/main/app/engines/markdown_parser.py): `app/engines/markdown_parser.py`
- [Semantic chunker & embeddings](https://github.com/darkerror64/retrivo/blob/main/app/engines/semantic_chunker.py): `app/engines/semantic_chunker.py`
- [Token optimizer](https://github.com/darkerror64/retrivo/blob/main/app/utils/token_optimizer.py): `app/utils/token_optimizer.py`

### Retrieval & ranking

- [Hybrid retrieval](https://github.com/darkerror64/retrivo/blob/main/app/db/retrieval_manager.py): `app/db/retrieval_manager.py`
- [Cross-encoder reranker](https://github.com/darkerror64/retrivo/blob/main/app/engines/reranker.py): `app/engines/reranker.py`

### Agent & language models

- [ReAct agent](https://github.com/darkerror64/retrivo/blob/main/app/engines/agent_engine.py): `app/engines/agent_engine.py`
- [LLM service](https://github.com/darkerror64/retrivo/blob/main/app/core/llm.py): `app/core/llm.py`

### Storage & configuration

- [Ingestion manager](https://github.com/darkerror64/retrivo/blob/main/app/db/database_manager.py): `app/db/database_manager.py`
- [PostgreSQL pgvector](https://github.com/darkerror64/retrivo/blob/main/app/db/database_manager.py): `app/db/database_manager.py`
- [Runtime settings](https://github.com/darkerror64/retrivo/blob/main/app/core/config.py): `app/core/config.py`

### Other components

- **User**
- **LLM provider**

## Connections

- User → CLI: queries
- User → Web UI: uses
- CLI → Main controller: invokes (evidence: `cli.py`)
- FastAPI → Main controller: routes through (evidence: `app/main.py`)
- FastAPI → Ingestion worker: dispatches async jobs (evidence: `app/main.py`)
- Main controller → PDF extraction: extracts PDF (evidence: `app/controllers/main_controller.py`)
- Main controller → Ingestion manager: ingests document (evidence: `app/controllers/main_controller.py`)
- Main controller → Hybrid retrieval: queries chunks (evidence: `app/controllers/main_controller.py`)
- Main controller → ReAct agent: routes agent query (evidence: `app/controllers/main_controller.py`)
- Ingestion worker → PDF extraction: extracts PDF (evidence: `app/worker.py`)
- Ingestion worker → Ingestion manager: ingests document (evidence: `app/worker.py`)
- Ingestion manager → Hierarchical chunker: chunks and embeds (evidence: `app/db/database_manager.py`)
- Hierarchical chunker → Markdown AST parser: parses structure (evidence: `app/engines/hierarchical_chunker.py`)
- Hierarchical chunker → Semantic chunker & embeddings: detects boundaries (evidence: `app/engines/hierarchical_chunker.py`)
- Hierarchical chunker → Token optimizer: optimizes chunks (evidence: `app/engines/hierarchical_chunker.py`)
- Hierarchical chunker → Semantic chunker & embeddings: embeds chunks (evidence: `app/engines/hierarchical_chunker.py`)
- Ingestion manager → PostgreSQL pgvector: stores chunks (evidence: `app/db/database_manager.py`)
- Hybrid retrieval → PostgreSQL pgvector: searches vectors and text (evidence: `README.md`)
- Hybrid retrieval → Semantic chunker & embeddings: embeds queries (evidence: `app/db/retrieval_manager.py`)
- Hybrid retrieval → Cross-encoder reranker: reranks candidates (evidence: `app/db/retrieval_manager.py`)
- ReAct agent → Hybrid retrieval: searches documents (evidence: `app/engines/agent_engine.py`)
- ReAct agent → PostgreSQL pgvector: reads documents and chunks (evidence: `app/engines/agent_engine.py`)
- ReAct agent → LLM service: requests reasoning (evidence: `app/engines/agent_engine.py`)
- LLM service → LLM provider: calls configured provider (evidence: `app/core/llm.py`)
- Runtime settings → LLM service: configures provider (evidence: `app/core/llm.py`)
