# disgandh/rag_sbc architecture

> Architecture diagram of the GitHub repository disgandh/rag_sbc, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/disgandh/rag_sbc
- Repository: https://github.com/disgandh/rag_sbc

## Overview

##### Project Type and Purpose
- From the name “RAG_sbc” and the solitary README.md, it appears this is a Retrieval-Augmented Generation (RAG) pipeline or toolkit. “sbc” might stand for “semantic-based chunking” or similar.
- Goal: Ingest documents, split into chunks, embed them, store in a vector database, retrieve relevant chunks at query time, then feed those plus the user’s prompt into an LLM to generate answers.

##### High-Level Architectural Components
a. Ingestion & Chunking
– File loader / crawler / uploader
– Text splitter or SBC module
b. Embedding Engine
– Embeddings service (OpenAI, HuggingFace, custom)
c. Vector Store
– Database (e.g., Pinecone, Weaviate, Redis, FAISS)
d. Retriever
– k-NN search module
e. LLM Orchestration
– Prompt constructor, context assembly
– LLM service (OpenAI GPT, local model)
f. API / CLI Interface
– Exposes a REST or CLI entrypoint for user queries
g. Optional:
– Caching layer
– Monitoring & logging
– Authentication / rate limiting

##### Relationships & Data Flow
1. User invokes CLI/API →
2. Ingestion & Chunking →
3. Embedding Engine →
4. Vector Store (indexing)
At query time:
1. User query → Retriever (vector store search) →
2. Retrieved chunks + query → LLM Orchestration →
3. Response returned to user.
– If there is a separate ingestion service vs. query-time service, illustrate these as distinct sub-systems.

##### Architectural Patterns & Design Principles
- Pipeline Pattern: clearly separate “offline” indexing from “online” query flow.
- Modularization: each major capability (chunker, embedder, retriever, generator) is its own service or module.
- Pluggability: show extension points (e.g., swap in a different vector DB or LLM).
- Stateless Query Service: the query node does not hold state; it delegates to the vector store and LLM.

##### Technologies & Frameworks (Placeholders)
– Python modules / packages (e.g., rag_sbc.chunker, rag_sbc.embedder, rag_sbc.retriever)
– Embedding API: OpenAI Embeddings or HuggingFace embeddings
– Vector DB: FAISS or Redis or Pinecone
– LLM API: OpenAI ChatCompletion or local model server
– Web framework (if used): FastAPI or Flask for REST
– CLI framework: Click or argparse

– Validate the actual directory structure as modules become implemented (e.g., src/chunker, src/retriever).
– Identify any missing pieces (e.g., auth, monitoring).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Offline Indexing Workflow
    subgraph "Offline Indexing"
        direction TB
        CLI_Offline["CLI/API"]:::iface
        Chunker["Chunker (Semantic-Based Chunking)"]:::proc
        Embedder["Embedder Module"]:::proc
        EmbeddingsAPI["Embeddings Service (OpenAI/HuggingFace)"]:::ext
        VectorDB[(Vector Store)]:::db
        CLI_Offline -->|"ingest documents"| Chunker
        Chunker -->|"produce text chunks"| Embedder
        Embedder -->|"request embeddings"| EmbeddingsAPI
        EmbeddingsAPI -->|"return embeddings"| Embedder
        Embedder -->|"store embeddings"| VectorDB
    end

    %% Online Serving Workflow
    subgraph "Online Serving"
        direction TB
        CLI_Online["CLI/API"]:::iface
        Retriever["Retriever (k-NN Search)"]:::proc
        PromptConstructor["Prompt Constructor"]:::proc
        LLMService["LLM Service (OpenAI GPT/local)"]:::ext
        CLI_Online -->|"user query"| Retriever
        Retriever -->|"fetch top-k chunks"| VectorDB
        VectorDB --> Retriever
        Retriever --> PromptConstructor
        PromptConstructor -->|"construct prompt"| LLMService
        LLMService -->|"response"| CLI_Online
    end

    %% Class Definitions
    classDef db fill:#AED6F1,stroke:#085A8C,color:#000;
    classDef proc fill:#ABEBC6,stroke:#1D8348,color:#000;
    classDef ext fill:#F5B041,stroke:#B9770E,color:#000;
    classDef iface fill:#D5DBDB,stroke:#7F8C8D,color:#000;
```
