# ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application architecture

> Architecture diagram of the GitHub repository ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application
- Repository: https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application

## Overview

This repository is a secure, headless RAG API for internal assistants. A client submits a role-scoped chat request; the API creates or reuses a session, and the agent can answer from conversation context or invoke the RBAC-filtered hybrid search tool. Retrieved passages are reranked before the agent responds, then conversation memory is saved and periodically summarized. The README also documents retrieval, tool-calling, and KPI evaluation capabilities. The ingestion workflow files are present, but their runtime connections are not established by the sampled evidence, so they are not linked into the graph.

Main entry points: [`improved_and_optimized_RAG.py`](https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/improved_and_optimized_RAG.py) (RBAC search tool), [`micro_rag_memory.py`](https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/micro_rag_memory.py) (Memory engine), [`reindex_bm25.py`](https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/reindex_bm25.py) (Reindex trigger), [`n8n_workflows/01_main_ingestion_flow.json`](https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/n8n_workflows/01_main_ingestion_flow.json) (Ingestion workflow)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api_agent["API and agent"]
  node_chat_api["Chat endpoint"]
  node_query_flow["Query orchestration"]
  node_react_agent["ReAct agent"]
end

subgraph group_retrieval["Secure retrieval"]
  node_search_tool["RBAC search tool"]
  node_qdrant[("Qdrant collections")]
  node_dense_embed["Dense embeddings"]
  node_sparse_embed["BM25 embeddings"]
  node_reranker["Cross-encoder reranker"]
end

subgraph group_memory["Session memory"]
  node_memory_engine["Memory engine"]
  node_sqlite_memory[("Short-term memory<br/>[chat_memory.sqlite]")]
  node_summary_files["Long-term summaries"]
end

subgraph group_operations["Index operations"]
  node_reindex_api["Reindex trigger<br/>[reindex_bm25.py]"]
  node_n8n_ingestion["Ingestion workflow"]
end

subgraph group_evaluation["Evaluation"]
  node_rag_eval["RAG evaluation<br/>[evaluate_rag.py]"]
  node_tool_eval["Tool-call evaluation"]
  node_kpi_eval["KPI evaluation"]
end

node_client(("Assistant client"))
node_llm["Chat model"]

node_client -->|"submits chat"| node_chat_api
node_chat_api -->|"calls"| node_query_flow
node_query_flow -->|"invokes"| node_react_agent
node_react_agent -->|"reasons with"| node_llm
node_react_agent -->|"can invoke"| node_search_tool
node_search_tool -->|"embeds query"| node_dense_embed
node_search_tool -->|"creates sparse vector"| node_sparse_embed
node_search_tool -->|"queries with RBAC"| node_qdrant
node_search_tool -->|"reranks passages"| node_reranker
node_query_flow -->|"reads and writes"| node_memory_engine
node_memory_engine -->|"stores and recalls"| node_sqlite_memory
node_memory_engine -->|"reads and writes"| node_summary_files
node_query_flow -->|"triggers summary rotation"| node_memory_engine
node_memory_engine -->|"summarizes with"| node_llm
node_reindex_api -->|"upserts BM25 vectors"| node_qdrant
node_rag_eval -->|"evaluates responses"| node_query_flow
node_tool_eval -->|"checks tool behavior"| node_query_flow
node_kpi_eval -.->|"measures retrieval"| node_search_tool
node_n8n_ingestion -.->|"may populate"| node_qdrant

click node_chat_api "https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/improved_and_optimized_RAG.py"
click node_query_flow "https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/improved_and_optimized_RAG.py"
click node_react_agent "https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/improved_and_optimized_RAG.py"
click node_search_tool "https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/improved_and_optimized_RAG.py"
click node_memory_engine "https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/micro_rag_memory.py"
click node_sqlite_memory "https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/chat_memory.sqlite"
click node_reindex_api "https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/reindex_bm25.py"
click node_n8n_ingestion "https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/n8n_workflows/01_main_ingestion_flow.json"
click node_rag_eval "https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/evaluate_rag.py"
click node_tool_eval "https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/evaluate_tool_calling.py"
click node_kpi_eval "https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/final_kpi_evaluation.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_chat_api,node_query_flow,node_react_agent,node_client toneBlue
class node_search_tool,node_qdrant,node_dense_embed,node_sparse_embed,node_reranker toneAmber
class node_memory_engine,node_sqlite_memory,node_summary_files toneMint
class node_reindex_api,node_n8n_ingestion toneRose
class node_rag_eval,node_tool_eval,node_kpi_eval,node_llm toneIndigo
```

## Components

### API and agent

- [Chat endpoint](https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/improved_and_optimized_RAG.py): `improved_and_optimized_RAG.py`
- [Query orchestration](https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/improved_and_optimized_RAG.py): `improved_and_optimized_RAG.py`
- [ReAct agent](https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/improved_and_optimized_RAG.py): `improved_and_optimized_RAG.py`

### Secure retrieval

- [RBAC search tool](https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/improved_and_optimized_RAG.py): `improved_and_optimized_RAG.py`
- **Qdrant collections**
- **Dense embeddings**
- **BM25 embeddings**
- **Cross-encoder reranker**

### Session memory

- [Memory engine](https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/micro_rag_memory.py): `micro_rag_memory.py`
- [Short-term memory](https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/chat_memory.sqlite): `chat_memory.sqlite`
- **Long-term summaries**

### Index operations

- [Reindex trigger](https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/reindex_bm25.py): `reindex_bm25.py`
- [Ingestion workflow](https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/n8n_workflows/01_main_ingestion_flow.json): `n8n_workflows/01_main_ingestion_flow.json`

### Evaluation

- [RAG evaluation](https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/evaluate_rag.py): `evaluate_rag.py`
- [Tool-call evaluation](https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/evaluate_tool_calling.py): `evaluate_tool_calling.py`
- [KPI evaluation](https://github.com/ajayjai30/rag_system_with_n8n_file-ingestion-with-rbac-control-for-kcs-application/blob/main/final_kpi_evaluation.py): `final_kpi_evaluation.py`

### Other components

- **Assistant client**
- **Chat model**

## Connections

- Assistant client → Chat endpoint: submits chat (evidence: `README.md`)
- Chat endpoint → Query orchestration: calls (evidence: `improved_and_optimized_RAG.py`)
- Query orchestration → ReAct agent: invokes (evidence: `improved_and_optimized_RAG.py`)
- ReAct agent → Chat model: reasons with (evidence: `improved_and_optimized_RAG.py`)
- ReAct agent → RBAC search tool: can invoke (evidence: `improved_and_optimized_RAG.py`)
- RBAC search tool → Dense embeddings: embeds query (evidence: `improved_and_optimized_RAG.py`)
- RBAC search tool → BM25 embeddings: creates sparse vector (evidence: `improved_and_optimized_RAG.py`)
- RBAC search tool → Qdrant collections: queries with RBAC (evidence: `improved_and_optimized_RAG.py`)
- RBAC search tool → Cross-encoder reranker: reranks passages (evidence: `improved_and_optimized_RAG.py`)
- Query orchestration → Memory engine: reads and writes (evidence: `improved_and_optimized_RAG.py`)
- Memory engine → Short-term memory: stores and recalls (evidence: `micro_rag_memory.py`)
- Memory engine → Long-term summaries: reads and writes (evidence: `micro_rag_memory.py`)
- Query orchestration → Memory engine: triggers summary rotation (evidence: `improved_and_optimized_RAG.py`)
- Memory engine → Chat model: summarizes with (evidence: `micro_rag_memory.py`)
- Reindex trigger → Qdrant collections: upserts BM25 vectors (evidence: `reindex_bm25.py`)
- RAG evaluation → Query orchestration: evaluates responses (evidence: `evaluate_rag.py`)
- Tool-call evaluation → Query orchestration: checks tool behavior (evidence: `evaluate_tool_calling.py`)
- KPI evaluation → RBAC search tool: measures retrieval (evidence: `final_kpi_evaluation.py`)
- Ingestion workflow → Qdrant collections: may populate
