# coderonfleek/rag-from-scratch architecture

> Architecture diagram of the GitHub repository coderonfleek/rag-from-scratch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/coderonfleek/rag-from-scratch
- Repository: https://github.com/coderonfleek/rag-from-scratch

## Overview

This repository is a command-line RAG demo. Its build workflow reads supported local files, stores their contents in SQLite, chunks them, embeds them, and persists vectors in Chroma. Its query workflow loads that vector store, retrieves top-k documents for a supplied query, and prints them; it does not generate an LLM answer. The README also documents HuggingFace, OpenAI, and Google embedding options. The workflow is supported by the sampled source; optional provider connections are shown as integrations.

Main entry points: [`rag_pipeline/main.py`](https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/main.py) (CLI orchestrator), [`rag_pipeline/ingest.py`](https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/ingest.py) (File ingestion), [`rag_pipeline/pipeline.py`](https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/pipeline.py) (Persistent Chroma), [`rag_pipeline/db.py`](https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/db.py) (Raw document store)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflow["CLI workflows"]
  node_cli["CLI orchestrator<br/>[main.py]"]
  node_printed_results["Printed documents<br/>[main.py]"]
end

subgraph group_ingestion["Document ingestion"]
  node_ingest["File ingestion<br/>[ingest.py]"]
end

subgraph group_retrieval["Chunking and retrieval"]
  node_chunking["Document chunking<br/>[pipeline.py]"]
  node_retriever["Top-k retrieval<br/>[pipeline.py]"]
end

subgraph group_storage["Persistence and embeddings"]
  node_raw_db[("Raw document store<br/>[db.py]")]
  node_embedding_selection["Embedding selection<br/>[pipeline.py]"]
  node_vector_build["Vector store build<br/>[pipeline.py]"]
  node_chroma[("Persistent Chroma<br/>[pipeline.py]")]
end

node_cli_user(("CLI user"))
node_file_corpus["Local documents"]
node_query_input["Search query"]
node_embedding_provider["Embedding providers"]

node_cli_user -->|"runs"| node_cli
node_file_corpus -->|"reads"| node_ingest
node_query_input -->|"provides query"| node_cli
node_cli -->|"calls build"| node_ingest
node_ingest -->|"initializes and writes"| node_raw_db
node_cli -->|"fetches documents"| node_raw_db
node_cli -->|"chunks documents"| node_chunking
node_cli -->|"selects embeddings"| node_embedding_selection
node_embedding_selection -.->|"constructs provider"| node_embedding_provider
node_cli -->|"builds vectors"| node_vector_build
node_vector_build -->|"persists vectors"| node_chroma
node_cli -->|"loads store"| node_chroma
node_cli -->|"gets retriever and invokes"| node_retriever
node_retriever -->|"searches vectors"| node_chroma
node_cli -->|"prints retrieved documents"| node_printed_results

click node_cli "https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/main.py"
click node_ingest "https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/ingest.py"
click node_raw_db "https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/db.py"
click node_chunking "https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/pipeline.py"
click node_embedding_selection "https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/pipeline.py"
click node_vector_build "https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/pipeline.py"
click node_chroma "https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/pipeline.py"
click node_retriever "https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/pipeline.py"
click node_printed_results "https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/main.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_printed_results,node_cli_user toneBlue
class node_ingest toneAmber
class node_chunking,node_retriever toneMint
class node_raw_db,node_embedding_selection,node_vector_build,node_chroma toneRose
class node_file_corpus,node_query_input,node_embedding_provider toneIndigo
```

## Components

### CLI workflows

- [CLI orchestrator](https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/main.py): `rag_pipeline/main.py`
- [Printed documents](https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/main.py): `rag_pipeline/main.py`

### Document ingestion

- [File ingestion](https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/ingest.py): `rag_pipeline/ingest.py`

### Chunking and retrieval

- [Document chunking](https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/pipeline.py): `rag_pipeline/pipeline.py`
- [Top-k retrieval](https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/pipeline.py): `rag_pipeline/pipeline.py`

### Persistence and embeddings

- [Raw document store](https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/db.py): `rag_pipeline/db.py`
- [Embedding selection](https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/pipeline.py): `rag_pipeline/pipeline.py`
- [Vector store build](https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/pipeline.py): `rag_pipeline/pipeline.py`
- [Persistent Chroma](https://github.com/coderonfleek/rag-from-scratch/blob/main/rag_pipeline/pipeline.py): `rag_pipeline/pipeline.py`

### Other components

- **CLI user**
- **Local documents**
- **Search query**
- **Embedding providers**

## Connections

- CLI user → CLI orchestrator: runs
- Local documents → File ingestion: reads
- Search query → CLI orchestrator: provides query
- CLI orchestrator → File ingestion: calls build
- File ingestion → Raw document store: initializes and writes
- CLI orchestrator → Raw document store: fetches documents
- CLI orchestrator → Document chunking: chunks documents
- CLI orchestrator → Embedding selection: selects embeddings
- Embedding selection → Embedding providers: constructs provider
- CLI orchestrator → Vector store build: builds vectors
- Vector store build → Persistent Chroma: persists vectors
- CLI orchestrator → Persistent Chroma: loads store
- CLI orchestrator → Top-k retrieval: gets retriever and invokes
- Top-k retrieval → Persistent Chroma: searches vectors
- CLI orchestrator → Printed documents: prints retrieved documents