How fengzhenfei/carrel works
Carrel is a self-hosted agent knowledge base. Document folders are scanned and scheduled for parsing and chunking; outputs feed vector and keyword indexes and, optionally, a source-aware knowledge graph. Agents query the search API, which combines retrieval channels and returns evidence rather than generated answers. The web console manages knowledge bases, configuration, files, graphs, and tasks. The README documents the overall flow; sampled code supports selected parser, indexing, and search details, so uncertain internal dispatch links are omitted.
Main entry points: parse_job.py Parse jobs · discovery.py File discovery · worker.py Job worker · service.py Retrieval orchestration
Components 26
Ingestion
- Task scheduler
app/kb_pipeline/pipeline/scheduler.py - File discovery
app/kb_pipeline/discovery.py - Job worker
app/kb_pipeline/pipeline/worker.py - Parse jobs
app/kb_pipeline/pipeline/parse_job.py - Format parsers
app/kb_pipeline/parsers/router.py - Chunking
app/kb_pipeline/chunking/chunker.py
Knowledge indexes
- Embedding client
app/kb_pipeline/embedding/client.py - Vector index
app/kb_pipeline/vector/qdrant.py - Keyword index
app/kb_pipeline/search_fts.py - Graph construction
app/kb_pipeline/graph/build.py - Graph model client
app/kb_pipeline/graph/llm.py - Knowledge graph
app/kb_pipeline/graph/neo4j_import.py - Pipeline state
app/kb_pipeline/db.py
Retrieval
- Search API
app/kb_search/api.py - Retrieval orchestration
app/kb_search/service.py - Search channels
app/kb_search/channels.py - Fusion and reranking
app/kb_search/fusion.py - Evidence assembly
app/kb_search/evidence.py
User interfaces
- Web console API
app/kb_server/api.py - Console service
app/kb_server/service.py - Console UI
app/kb_server/static/app.js
Other components
- Document folders
- MinerU service
- Embedding endpoint
- AI agent
- Console user
Connections 31
- Document folders → File discovery: scans files
- Task scheduler → File discovery: schedules scans
- File discovery → Pipeline state: tracks files
- File discovery → Job worker: queues jobs
- Job worker → Parse jobs: runs parse
- Parse jobs → Format parsers: routes files
- Format parsers → MinerU service: parses remotely
- Parse jobs → Chunking: chunks content
- Chunking → Embedding client: embeds chunks
- Embedding client → Embedding endpoint: requests vectors
- Parse jobs → Vector index: writes vectors
- Job worker → Keyword index: syncs keywords
- Keyword index → Vector index: reads active points
- Parse jobs → Graph construction: builds graph
- Graph construction → Graph model client: extracts facts
- Graph construction → Knowledge graph: writes graph
- Job worker → Pipeline state: updates jobs
- AI agent → Search API: submits queries
- Search API → Retrieval orchestration: dispatches search
- Retrieval orchestration → Search channels: runs channels
- Search channels → Vector index: searches vectors
- Search channels → Keyword index: searches keywords
- Search channels → Knowledge graph: follows relations
- Retrieval orchestration → Fusion and reranking: fuses candidates
- Retrieval orchestration → Evidence assembly: assembles evidence
- Evidence assembly → Search API: returns evidence
- Console user → Console UI: manages knowledge
- Console UI → Web console API: calls API
- Web console API → Console service: delegates operations
- Console service → Pipeline state: reads and writes
- Console service → File discovery: configures bases