How darkerror64/retrivo works
PKOS ingests PDFs into structurally and semantically enriched vector chunks, then retrieves relevant passages for direct search or a multi-step ReAct answer. The API/controller and CLI provide interaction paths; an asynchronous worker supports background ingestion. The graph preserves parsing, chunking, token optimization, embedding, hybrid retrieval, reranking, and LLM capabilities. Some route and frontend wiring is not shown in the excerpts, so uncertain connections are omitted.
Main entry points: main_controller.py Main controller · hierarchical_chunker.py Hierarchical chunker · retrieval_manager.py Hybrid retrieval · agent_engine.py ReAct agent
Components 19
Interfaces & orchestration
- CLI
cli.py - Web UI
frontend/app - FastAPI
app/main.py - Main controller
app/controllers/main_controller.py - Ingestion worker
app/worker.py
Document ingestion
- PDF extraction
app/utils/pdf_extractor.py - Hierarchical chunker
app/engines/hierarchical_chunker.py - Markdown AST parser
app/engines/markdown_parser.py - Semantic chunker & embeddings
app/engines/semantic_chunker.py - Token optimizer
app/utils/token_optimizer.py
Retrieval & ranking
- Hybrid retrieval
app/db/retrieval_manager.py - Cross-encoder reranker
app/engines/reranker.py
Agent & language models
- ReAct agent
app/engines/agent_engine.py - LLM service
app/core/llm.py
Storage & configuration
- Ingestion manager
app/db/database_manager.py - PostgreSQL pgvector
app/db/database_manager.py - Runtime settings
app/core/config.py
Other components
- User
- LLM provider
Connections 25
- User → CLI: queries
- User → Web UI: uses
- CLI → Main controller: invokes
cli.py - FastAPI → Main controller: routes through
app/main.py - FastAPI → Ingestion worker: dispatches async jobs
app/main.py - Main controller → PDF extraction: extracts PDF
app/controllers/main_controller.py - Main controller → Ingestion manager: ingests document
app/controllers/main_controller.py - Main controller → Hybrid retrieval: queries chunks
app/controllers/main_controller.py - Main controller → ReAct agent: routes agent query
app/controllers/main_controller.py - Ingestion worker → PDF extraction: extracts PDF
app/worker.py - Ingestion worker → Ingestion manager: ingests document
app/worker.py - Ingestion manager → Hierarchical chunker: chunks and embeds
app/db/database_manager.py - Hierarchical chunker → Markdown AST parser: parses structure
app/engines/hierarchical_chunker.py - Hierarchical chunker → Semantic chunker & embeddings: detects boundaries
app/engines/hierarchical_chunker.py - Hierarchical chunker → Token optimizer: optimizes chunks
app/engines/hierarchical_chunker.py - Hierarchical chunker → Semantic chunker & embeddings: embeds chunks
app/engines/hierarchical_chunker.py - Ingestion manager → PostgreSQL pgvector: stores chunks
app/db/database_manager.py - Hybrid retrieval → PostgreSQL pgvector: searches vectors and text
README.md - Hybrid retrieval → Semantic chunker & embeddings: embeds queries
app/db/retrieval_manager.py - Hybrid retrieval → Cross-encoder reranker: reranks candidates
app/db/retrieval_manager.py - ReAct agent → Hybrid retrieval: searches documents
app/engines/agent_engine.py - ReAct agent → PostgreSQL pgvector: reads documents and chunks
app/engines/agent_engine.py - ReAct agent → LLM service: requests reasoning
app/engines/agent_engine.py - LLM service → LLM provider: calls configured provider
app/core/llm.py - Runtime settings → LLM service: configures provider
app/core/llm.py