How d3byn/cortex works
Cortex is a document-ingestion and grounded question-answering platform. Users upload PDF, TXT, or Markdown files for background parsing, chunking, embedding, and indexing; they can then search or ask questions and receive cited answers with a groundedness check. The graph preserves the separate API, ingestion, retrieval, and answer-generation stages. Several subsystem links rely on the README or SOURCE INDEX references; the frontend-to-API connection is documented only at a high level.
Main entry points: hybrid.py Hybrid Retrieval · background.py Background Ingestion · main.py FastAPI Application · documents.py Document Management
Components 24
User Interface & API
- Chat Frontend
frontend/app.py - FastAPI Application
backend/app/main.py - Ingest Endpoint
backend/app/api/ingest.py - Search Endpoint
backend/app/api/search.py - Query Endpoint
backend/app/api/query.py - Document Management
backend/app/api/documents.py - API Schemas
backend/app/schemas/models.py
Document Ingestion
- Background Ingestion
backend/app/ingestion/background.py - File Parsing
backend/app/ingestion/parsers.py - Text Chunking
backend/app/ingestion/chunker.py
Search & Retrieval
- Embedding Service
backend/app/retrieval/embeddings.py - Hybrid Retrieval
backend/app/retrieval/hybrid.py - Rank Fusion
backend/app/retrieval/fusion.py - Chunk Lookup
backend/app/retrieval/lookup.py - Cross-Encoder Reranker
backend/app/retrieval/reranker.py
Answer Generation
- LangGraph Pipeline
backend/app/graph/pipeline.py - Query & Grounding Steps
backend/app/graph/nodes.py - Gemini API
- Query Tracing
backend/app/tracing/tracer.py
Persistence & State
- FAISS Index
backend/app/retrieval/vector_store.py - BM25 Index
backend/app/retrieval/keyword_store.py - SQLite Records
backend/app/db/models.py - Response Cache
backend/app/cache/response_cache.py
Other components
- User
Connections 35
- User → Chat Frontend: asks questions
README.md - Chat Frontend → FastAPI Application: uses API
- FastAPI Application → Ingest Endpoint: registers route
backend/app/main.py - FastAPI Application → Search Endpoint: registers route
backend/app/main.py - FastAPI Application → Query Endpoint: registers route
backend/app/main.py - FastAPI Application → Document Management: registers route
backend/app/main.py - Ingest Endpoint → Background Ingestion: schedules processing
backend/app/api/ingest.py - Ingest Endpoint → File Parsing: detects file type
backend/app/api/ingest.py - Ingest Endpoint → SQLite Records: creates job
backend/app/api/ingest.py - Background Ingestion → File Parsing: parses file
backend/app/ingestion/background.py - Background Ingestion → Text Chunking: chunks text
backend/app/ingestion/background.py - Background Ingestion → Embedding Service: embeds chunks
backend/app/ingestion/background.py - Background Ingestion → SQLite Records: saves records
backend/app/ingestion/background.py - Background Ingestion → FAISS Index: indexes vectors
backend/app/ingestion/background.py - Background Ingestion → BM25 Index: updates index
backend/app/ingestion/background.py - Background Ingestion → Response Cache: invalidates cache
backend/app/ingestion/background.py - Search Endpoint → Hybrid Retrieval: retrieves results
backend/app/api/search.py - Hybrid Retrieval → Embedding Service: embeds query
backend/app/retrieval/hybrid.py - Hybrid Retrieval → FAISS Index: searches vectors
backend/app/retrieval/hybrid.py - Hybrid Retrieval → BM25 Index: searches terms
backend/app/retrieval/hybrid.py - Hybrid Retrieval → Rank Fusion: fuses rankings
backend/app/retrieval/hybrid.py - Hybrid Retrieval → Chunk Lookup: loads chunks
backend/app/retrieval/hybrid.py - Hybrid Retrieval → Cross-Encoder Reranker: reranks results
backend/app/retrieval/hybrid.py - Hybrid Retrieval → SQLite Records: filters documents
backend/app/retrieval/hybrid.py - Query Endpoint → Response Cache: checks and saves
backend/app/api/query.py - Query Endpoint → LangGraph Pipeline: invokes pipeline
backend/app/api/query.py - Query Endpoint → Query Tracing: creates trace
backend/app/api/query.py - LangGraph Pipeline → Query & Grounding Steps: orchestrates steps
backend/app/graph/pipeline.py - Query & Grounding Steps → Hybrid Retrieval: retrieves context
backend/app/graph/nodes.py - Query & Grounding Steps → Gemini API: generates and audits
backend/app/graph/nodes.py - Query & Grounding Steps → SQLite Records: reads records
backend/app/graph/nodes.py - Document Management → SQLite Records: manages records
backend/app/api/documents.py - Document Management → FAISS Index: removes vectors
backend/app/api/documents.py - Document Management → BM25 Index: updates index
backend/app/api/documents.py - Document Management → Response Cache: invalidates cache
backend/app/api/documents.py