# kruxai/ragbuilder architecture

> Architecture diagram of the GitHub repository kruxai/ragbuilder, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kruxai/ragbuilder
- Repository: https://github.com/kruxai/ragbuilder

## Overview

RagBuilder is a toolkit for tuning and assembling production RAG pipelines. A user supplies data and configuration, then optimizes ingestion, retrieval, and generation against a provided or synthetic evaluation dataset; the resulting pipeline can answer queries and expose its components. The README also documents reusable advanced templates, project persistence, and API deployment. The graph combines documented workflow capabilities with sampled pipeline behavior; unsampled subsystem wiring is shown only where the README supports it.

Main entry points: [`src/ragbuilder/core/builder.py`](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/core/builder.py) (Builder SDK), [`src/ragbuilder/data_ingest/pipeline.py`](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/data_ingest/pipeline.py) (Ingestion Pipeline), [`src/ragbuilder/eval.py`](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/eval.py) (Evaluation), [`src/ragbuilder/core/results.py`](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/core/results.py) (Optimization Results)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflow["User Workflow"]
  node_builder["Builder SDK<br/>[builder.py]"]
  node_configuration["Pipeline Configuration<br/>[data_ingest.py]"]
  node_inputdata["Source Data"]
  node_results["Optimization Results<br/>[results.py]"]
end

subgraph group_ingestion["Data Ingestion"]
  node_sampler["Data Sampler<br/>[sampler.py]"]
  node_ingest["Ingestion Pipeline<br/>[pipeline.py]"]
  node_loaders["Document Loaders<br/>[loader.py]"]
  node_chunking["Chunking Strategies"]
  node_embeddings["Embedding Models<br/>[embedding.py]"]
  node_vectorstore[("Vector Store<br/>[vectordb.py]")]
end

subgraph group_tuning["Optimization And Evaluation"]
  node_dataset["Evaluation Dataset<br/>[generate_data.py]"]
  node_ingestopt["Ingestion Optimization<br/>[optimization.py]"]
  node_retrieveropt["Retrieval Optimization<br/>[optimization.py]"]
  node_generationopt["Generation Optimization<br/>[optimization.py]"]
  node_evaluation["Evaluation<br/>[eval.py]"]
end

subgraph group_runtime["RAG Runtime"]
  node_retrieval["Retriever Pipeline<br/>[pipeline.py]"]
  node_templates["RAG Templates"]
  node_generation["Generation Pipeline<br/>[pipeline.py]"]
  node_llm["LLM Provider"]
end

subgraph group_operations["Persistence And API"]
  node_projectstore[("Project Storage<br/>[config_store.py]")]
  node_api["API Service<br/>[ragbuilder.py]"]
end

node_user(("RAG Builder User"))
node_apiuser(("API Consumer"))

node_user -->|"configures"| node_builder
node_user -->|"provides"| node_inputdata
node_builder -->|"uses"| node_configuration
node_builder -->|"optimizes ingestion"| node_ingestopt
node_builder -->|"optimizes retrieval"| node_retrieveropt
node_builder -->|"optimizes generation"| node_generationopt
node_configuration -->|"configures"| node_ingest
node_inputdata -->|"sampled by"| node_sampler
node_sampler -->|"supplies sample"| node_ingest
node_ingest -->|"loads documents"| node_loaders
node_ingest -->|"splits documents"| node_chunking
node_ingest -->|"embeds chunks"| node_embeddings
node_ingest -->|"populates"| node_vectorstore
node_ingestopt -->|"tunes"| node_ingest
node_builder -->|"generates or accepts"| node_dataset
node_ingestopt -->|"evaluates candidates"| node_evaluation
node_retrieveropt -->|"tunes"| node_retrieval
node_retrieveropt -->|"evaluates candidates"| node_evaluation
node_generationopt -->|"tunes"| node_generation
node_generationopt -->|"evaluates candidates"| node_evaluation
node_dataset -->|"provides cases"| node_evaluation
node_evaluation -->|"scores candidates"| node_ingestopt
node_evaluation -->|"scores candidates"| node_retrieveropt
node_evaluation -->|"scores candidates"| node_generationopt
node_user -->|"selects template"| node_templates
node_vectorstore -->|"searches"| node_retrieval
node_retrieval -->|"supplies context"| node_generation
node_generation -->|"generates answer"| node_llm
node_builder -->|"returns results"| node_results
node_user -->|"queries"| node_results
node_results -->|"invokes pipeline"| node_generation
node_results -->|"saves or loads"| node_projectstore
node_user -->|"deploys pipeline"| node_api
node_apiuser -->|"sends request"| node_api
node_api -->|"reads run data"| node_projectstore
node_api -->|"serves pipeline"| node_results

click node_builder "https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/core/builder.py"
click node_configuration "https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/config/data_ingest.py"
click node_sampler "https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/sampler.py"
click node_ingest "https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/data_ingest/pipeline.py"
click node_loaders "https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/langchain_module/loader/loader.py"
click node_chunking "https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/langchain_module/chunkingstrategy/langchain_chunking.py"
click node_embeddings "https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/langchain_module/embedding_model/embedding.py"
click node_vectorstore "https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/langchain_module/vectordb/vectordb.py"
click node_dataset "https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/generate_data.py"
click node_ingestopt "https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/data_ingest/optimization.py"
click node_retrieveropt "https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/retriever/optimization.py"
click node_generationopt "https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/generation/optimization.py"
click node_evaluation "https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/eval.py"
click node_retrieval "https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/retriever/pipeline.py"
click node_templates "https://github.com/kruxai/ragbuilder/tree/main/src/ragbuilder/rag_templates/sota"
click node_generation "https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/generation/pipeline.py"
click node_results "https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/core/results.py"
click node_projectstore "https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/core/config_store.py"
click node_api "https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/ragbuilder.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_builder,node_configuration,node_inputdata,node_results,node_user toneBlue
class node_sampler,node_ingest,node_loaders,node_chunking,node_embeddings,node_vectorstore toneAmber
class node_dataset,node_ingestopt,node_retrieveropt,node_generationopt,node_evaluation,node_apiuser toneMint
class node_retrieval,node_templates,node_generation,node_llm toneRose
class node_projectstore,node_api toneIndigo
```

## Components

### User Workflow

- [Builder SDK](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/core/builder.py): `src/ragbuilder/core/builder.py`
- [Pipeline Configuration](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/config/data_ingest.py): `src/ragbuilder/config/data_ingest.py`
- **Source Data**
- [Optimization Results](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/core/results.py): `src/ragbuilder/core/results.py`

### Data Ingestion

- [Data Sampler](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/sampler.py): `src/ragbuilder/sampler.py`
- [Ingestion Pipeline](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/data_ingest/pipeline.py): `src/ragbuilder/data_ingest/pipeline.py`
- [Document Loaders](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/langchain_module/loader/loader.py): `src/ragbuilder/langchain_module/loader/loader.py`
- [Chunking Strategies](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/langchain_module/chunkingstrategy/langchain_chunking.py): `src/ragbuilder/langchain_module/chunkingstrategy/langchain_chunking.py`
- [Embedding Models](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/langchain_module/embedding_model/embedding.py): `src/ragbuilder/langchain_module/embedding_model/embedding.py`
- [Vector Store](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/langchain_module/vectordb/vectordb.py): `src/ragbuilder/langchain_module/vectordb/vectordb.py`

### Optimization And Evaluation

- [Evaluation Dataset](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/generate_data.py): `src/ragbuilder/generate_data.py`
- [Ingestion Optimization](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/data_ingest/optimization.py): `src/ragbuilder/data_ingest/optimization.py`
- [Retrieval Optimization](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/retriever/optimization.py): `src/ragbuilder/retriever/optimization.py`
- [Generation Optimization](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/generation/optimization.py): `src/ragbuilder/generation/optimization.py`
- [Evaluation](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/eval.py): `src/ragbuilder/eval.py`

### RAG Runtime

- [Retriever Pipeline](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/retriever/pipeline.py): `src/ragbuilder/retriever/pipeline.py`
- [RAG Templates](https://github.com/kruxai/ragbuilder/tree/main/src/ragbuilder/rag_templates/sota): `src/ragbuilder/rag_templates/sota`
- [Generation Pipeline](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/generation/pipeline.py): `src/ragbuilder/generation/pipeline.py`
- **LLM Provider**

### Persistence And API

- [Project Storage](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/core/config_store.py): `src/ragbuilder/core/config_store.py`
- [API Service](https://github.com/kruxai/ragbuilder/blob/main/src/ragbuilder/ragbuilder.py): `src/ragbuilder/ragbuilder.py`

### Other components

- **RAG Builder User**
- **API Consumer**

## Connections

- RAG Builder User → Builder SDK: configures
- RAG Builder User → Source Data: provides
- Builder SDK → Pipeline Configuration: uses
- Builder SDK → Ingestion Optimization: optimizes ingestion
- Builder SDK → Retrieval Optimization: optimizes retrieval
- Builder SDK → Generation Optimization: optimizes generation
- Pipeline Configuration → Ingestion Pipeline: configures
- Source Data → Data Sampler: sampled by
- Data Sampler → Ingestion Pipeline: supplies sample
- Ingestion Pipeline → Document Loaders: loads documents
- Ingestion Pipeline → Chunking Strategies: splits documents
- Ingestion Pipeline → Embedding Models: embeds chunks
- Ingestion Pipeline → Vector Store: populates
- Ingestion Optimization → Ingestion Pipeline: tunes
- Builder SDK → Evaluation Dataset: generates or accepts
- Ingestion Optimization → Evaluation: evaluates candidates
- Retrieval Optimization → Retriever Pipeline: tunes
- Retrieval Optimization → Evaluation: evaluates candidates
- Generation Optimization → Generation Pipeline: tunes
- Generation Optimization → Evaluation: evaluates candidates
- Evaluation Dataset → Evaluation: provides cases
- Evaluation → Ingestion Optimization: scores candidates
- Evaluation → Retrieval Optimization: scores candidates
- Evaluation → Generation Optimization: scores candidates
- RAG Builder User → RAG Templates: selects template
- Vector Store → Retriever Pipeline: searches
- Retriever Pipeline → Generation Pipeline: supplies context
- Generation Pipeline → LLM Provider: generates answer
- Builder SDK → Optimization Results: returns results
- RAG Builder User → Optimization Results: queries
- Optimization Results → Generation Pipeline: invokes pipeline
- Optimization Results → Project Storage: saves or loads
- RAG Builder User → API Service: deploys pipeline
- API Consumer → API Service: sends request
- API Service → Project Storage: reads run data
- API Service → Optimization Results: serves pipeline