# ubaidur404786/secure-rag-benchmark architecture

> Architecture diagram of the GitHub repository ubaidur404786/secure-rag-benchmark, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ubaidur404786/secure-rag-benchmark
- Repository: https://github.com/ubaidur404786/secure-rag-benchmark

## Overview

Secure RAG is a help-desk chatbot that retrieves company documents and generates answers, with a baseline mode and a guarded mode. The user asks through the Streamlit demo, which builds both RAG graphs over a shared vector store and invokes the selected graph. In guarded mode, input checks can refuse a request; role-filtered retrieval and context cleaning precede generation, then output checks protect the answer. A separate LiteLLM gateway adds shared guardrails, caching, fallback and call logging for applications routed through it. Edges reflect shown calls and README claims; unsampled benchmark and NeMo notebook internals are not expanded.

Main entry points: [`secure_rag.py`](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/secure_rag.py) (Secure RAG graph), [`rag_core.py`](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/rag_core.py) (In-memory vector store), [`app.py`](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/app.py) (Streamlit demo), [`gateway.py`](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/gateway.py) (LLM gateway)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["User interface"]
  node_demo["Streamlit demo<br/>[app.py]"]
end

subgraph group_knowledge["Knowledge base"]
  node_public_docs["Public documents<br/>[fake_docs.py]"]
  node_restricted_docs["Role-restricted documents"]
  node_poisoned_docs["Poisoned wiki document<br/>[poisoned_docs.py]"]
  node_role_policy["Users and role access"]
  node_vector_store[("In-memory vector store<br/>[rag_core.py]")]
end

subgraph group_rag["RAG and security"]
  node_baseline["Baseline RAG graph<br/>[rag_core.py]"]
  node_secure_graph["Secure RAG graph<br/>[secure_rag.py]"]
  node_input_guard["Input checks<br/>[security.py]"]
  node_access_control["Role-filtered retrieval<br/>[security.py]"]
  node_context_guard["Context cleaning<br/>[security.py]"]
  node_output_guard["Output checks<br/>[security.py]"]
  node_model_provider["Model provider<br/>[llm_provider.py]"]
  node_attack_cases["Attack and normal cases<br/>[attacks.py]"]
end

subgraph group_gateway["Model gateway"]
  node_gateway["LLM gateway<br/>[gateway.py]"]
  node_gateway_service["LiteLLM model router<br/>[gateway.py]"]
end

node_user(("Help-desk user"))
node_model_service{{"Ollama or OpenAI"}}

node_user -->|"asks"| node_demo
node_demo -->|"offers role choice"| node_role_policy
node_demo -->|"builds shared store"| node_vector_store
node_demo -->|"builds and invokes"| node_baseline
node_demo -->|"builds and invokes"| node_secure_graph
node_public_docs -->|"indexed with"| node_vector_store
node_restricted_docs -->|"indexed with"| node_vector_store
node_poisoned_docs -->|"indexed with"| node_vector_store
node_role_policy -->|"defines permissions"| node_access_control
node_baseline -->|"searches"| node_vector_store
node_baseline -->|"gets chat model"| node_model_provider
node_secure_graph -->|"checks input"| node_input_guard
node_secure_graph -->|"filters retrieval"| node_access_control
node_secure_graph -->|"searches"| node_vector_store
node_secure_graph -->|"cleans context"| node_context_guard
node_secure_graph -->|"gets chat model"| node_model_provider
node_secure_graph -->|"checks answer"| node_output_guard
node_model_provider -->|"creates model clients"| node_model_service
node_gateway -->|"routes model calls"| node_gateway_service
node_gateway -->|"checks inputs"| node_input_guard
node_gateway -->|"checks outputs"| node_output_guard
node_gateway_service -.->|"calls model"| node_model_service

click node_demo "https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/app.py"
click node_public_docs "https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/fake_docs.py"
click node_restricted_docs "https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/confidential_docs.py"
click node_poisoned_docs "https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/poisoned_docs.py"
click node_role_policy "https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/confidential_docs.py"
click node_vector_store "https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/rag_core.py"
click node_baseline "https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/rag_core.py"
click node_secure_graph "https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/secure_rag.py"
click node_input_guard "https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/security.py"
click node_access_control "https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/security.py"
click node_context_guard "https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/security.py"
click node_output_guard "https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/security.py"
click node_model_provider "https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/llm_provider.py"
click node_attack_cases "https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/attacks.py"
click node_gateway "https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/gateway.py"
click node_gateway_service "https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/gateway.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_demo,node_user toneBlue
class node_public_docs,node_restricted_docs,node_poisoned_docs,node_role_policy,node_vector_store toneAmber
class node_baseline,node_secure_graph,node_input_guard,node_access_control,node_context_guard,node_output_guard,node_model_provider,node_attack_cases toneMint
class node_gateway,node_gateway_service toneRose
class node_model_service toneIndigo
```

## Components

### User interface

- [Streamlit demo](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/app.py): `app.py`

### Knowledge base

- [Public documents](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/fake_docs.py): `fake_docs.py`
- [Role-restricted documents](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/confidential_docs.py): `confidential_docs.py`
- [Poisoned wiki document](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/poisoned_docs.py): `poisoned_docs.py`
- [Users and role access](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/confidential_docs.py): `confidential_docs.py`
- [In-memory vector store](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/rag_core.py): `rag_core.py`

### RAG and security

- [Baseline RAG graph](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/rag_core.py): `rag_core.py`
- [Secure RAG graph](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/secure_rag.py): `secure_rag.py`
- [Input checks](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/security.py): `security.py`
- [Role-filtered retrieval](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/security.py): `security.py`
- [Context cleaning](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/security.py): `security.py`
- [Output checks](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/security.py): `security.py`
- [Model provider](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/llm_provider.py): `llm_provider.py`
- [Attack and normal cases](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/attacks.py): `attacks.py`

### Model gateway

- [LLM gateway](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/gateway.py): `gateway.py`
- [LiteLLM model router](https://github.com/ubaidur404786/secure-rag-benchmark/blob/main/gateway.py): `gateway.py`

### Other components

- **Help-desk user**
- **Ollama or OpenAI**

## Connections

- Help-desk user → Streamlit demo: asks (evidence: `app.py`)
- Streamlit demo → Users and role access: offers role choice (evidence: `app.py`)
- Streamlit demo → In-memory vector store: builds shared store (evidence: `app.py`)
- Streamlit demo → Baseline RAG graph: builds and invokes (evidence: `app.py`)
- Streamlit demo → Secure RAG graph: builds and invokes (evidence: `app.py`)
- Public documents → In-memory vector store: indexed with (evidence: `rag_core.py`)
- Role-restricted documents → In-memory vector store: indexed with (evidence: `rag_core.py`)
- Poisoned wiki document → In-memory vector store: indexed with (evidence: `rag_core.py`)
- Users and role access → Role-filtered retrieval: defines permissions (evidence: `security.py`)
- Baseline RAG graph → In-memory vector store: searches (evidence: `rag_core.py`)
- Baseline RAG graph → Model provider: gets chat model (evidence: `rag_core.py`)
- Secure RAG graph → Input checks: checks input (evidence: `secure_rag.py`)
- Secure RAG graph → Role-filtered retrieval: filters retrieval (evidence: `secure_rag.py`)
- Secure RAG graph → In-memory vector store: searches (evidence: `secure_rag.py`)
- Secure RAG graph → Context cleaning: cleans context (evidence: `secure_rag.py`)
- Secure RAG graph → Model provider: gets chat model (evidence: `secure_rag.py`)
- Secure RAG graph → Output checks: checks answer (evidence: `secure_rag.py`)
- Model provider → Ollama or OpenAI: creates model clients (evidence: `llm_provider.py`)
- LLM gateway → LiteLLM model router: routes model calls (evidence: `gateway.py`)
- LLM gateway → Input checks: checks inputs (evidence: `gateway.py`)
- LLM gateway → Output checks: checks outputs (evidence: `gateway.py`)
- LiteLLM model router → Ollama or OpenAI: calls model
