How kruxai/ragbuilder works
RagBuilder is a toolkit for tuning and assembling production RAG pipelines. A user supplies data and configuration, then optimizes ingestion, retrieval, and generation against a provided or synthetic evaluation dataset; the resulting pipeline can answer queries and expose its components. The README also documents reusable advanced templates, project persistence, and API deployment. The graph combines documented workflow capabilities with sampled pipeline behavior; unsampled subsystem wiring is shown only where the README supports it.
Main entry points: builder.py Builder SDK · pipeline.py Ingestion Pipeline · eval.py Evaluation · results.py Optimization Results
Components 23
User Workflow
- Builder SDK
src/ragbuilder/core/builder.py - Pipeline Configuration
src/ragbuilder/config/data_ingest.py - Source Data
- Optimization Results
src/ragbuilder/core/results.py
Data Ingestion
- Data Sampler
src/ragbuilder/sampler.py - Ingestion Pipeline
src/ragbuilder/data_ingest/pipeline.py - Document Loaders
src/ragbuilder/langchain_module/loader/loader.py - Chunking Strategies
src/ragbuilder/langchain_module/chunkingstrategy/langchain_chunking.py - Embedding Models
src/ragbuilder/langchain_module/embedding_model/embedding.py - Vector Store
src/ragbuilder/langchain_module/vectordb/vectordb.py
Optimization And Evaluation
- Evaluation Dataset
src/ragbuilder/generate_data.py - Ingestion Optimization
src/ragbuilder/data_ingest/optimization.py - Retrieval Optimization
src/ragbuilder/retriever/optimization.py - Generation Optimization
src/ragbuilder/generation/optimization.py - Evaluation
src/ragbuilder/eval.py
RAG Runtime
- Retriever Pipeline
src/ragbuilder/retriever/pipeline.py - RAG Templates
src/ragbuilder/rag_templates/sota - Generation Pipeline
src/ragbuilder/generation/pipeline.py - LLM Provider
Persistence And API
- Project Storage
src/ragbuilder/core/config_store.py - API Service
src/ragbuilder/ragbuilder.py
Other components
- RAG Builder User
- API Consumer
Connections 36
- RAG Builder User → Builder SDK: configures
- RAG Builder User → Source Data: provides
- Builder SDK → Pipeline Configuration: uses
- Builder SDK → Ingestion Optimization: optimizes ingestion
- Builder SDK → Retrieval Optimization: optimizes retrieval
- Builder SDK → Generation Optimization: optimizes generation
- Pipeline Configuration → Ingestion Pipeline: configures
- Source Data → Data Sampler: sampled by
- Data Sampler → Ingestion Pipeline: supplies sample
- Ingestion Pipeline → Document Loaders: loads documents
- Ingestion Pipeline → Chunking Strategies: splits documents
- Ingestion Pipeline → Embedding Models: embeds chunks
- Ingestion Pipeline → Vector Store: populates
- Ingestion Optimization → Ingestion Pipeline: tunes
- Builder SDK → Evaluation Dataset: generates or accepts
- Ingestion Optimization → Evaluation: evaluates candidates
- Retrieval Optimization → Retriever Pipeline: tunes
- Retrieval Optimization → Evaluation: evaluates candidates
- Generation Optimization → Generation Pipeline: tunes
- Generation Optimization → Evaluation: evaluates candidates
- Evaluation Dataset → Evaluation: provides cases
- Evaluation → Ingestion Optimization: scores candidates
- Evaluation → Retrieval Optimization: scores candidates
- Evaluation → Generation Optimization: scores candidates
- RAG Builder User → RAG Templates: selects template
- Vector Store → Retriever Pipeline: searches
- Retriever Pipeline → Generation Pipeline: supplies context
- Generation Pipeline → LLM Provider: generates answer
- Builder SDK → Optimization Results: returns results
- RAG Builder User → Optimization Results: queries
- Optimization Results → Generation Pipeline: invokes pipeline
- Optimization Results → Project Storage: saves or loads
- RAG Builder User → API Service: deploys pipeline
- API Consumer → API Service: sends request
- API Service → Project Storage: reads run data
- API Service → Optimization Results: serves pipeline