# campusx-official/rag-eval-deepeval architecture

> Architecture diagram of the GitHub repository campusx-official/rag-eval-deepeval, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/campusx-official/rag-eval-deepeval
- Repository: https://github.com/campusx-official/rag-eval-deepeval

## Overview

This repository is a transcript-grounded RAG course assistant accompanied by a broad evaluation toolkit. The user workflow is chat question → retrieval and reranking → answer generation, with retrieved context shown alongside the answer. A separate evaluation workflow runs quality, safety, and operational checks against golden datasets or live pipeline output. The README is empty, and several evaluation and pipeline modules were not sampled; their presence is clear from the tree, but uncertain internal wiring is omitted.

Main entry points: [`evals/run_suite.py`](https://github.com/campusx-official/rag-eval-deepeval/blob/main/evals/run_suite.py) (Suite runner), [`evals/eval_rag_pipeline.py`](https://github.com/campusx-official/rag-eval-deepeval/blob/main/evals/eval_rag_pipeline.py) (RAG quality), [`evals/eval_retriever.py`](https://github.com/campusx-official/rag-eval-deepeval/blob/main/evals/eval_retriever.py) (Retrieval quality), [`src/rag_pipeline.py`](https://github.com/campusx-official/rag-eval-deepeval/blob/main/src/rag_pipeline.py) (RAG pipeline)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_assistant["Course assistant"]
  node_chat_ui["Chat UI<br/>[app.py]"]
  node_rag_pipeline["RAG pipeline<br/>[rag_pipeline.py]"]
  node_generator["Answer generator<br/>[generator.py]"]
end

subgraph group_retrieval["Knowledge retrieval"]
  node_reranker["Reranking retriever<br/>[reranker.py]"]
  node_retriever["Vector retriever<br/>[retriever.py]"]
  node_transcripts["Course transcripts"]
  node_vector_store[("Chroma store<br/>[retriever.py]")]
end

subgraph group_evaluation["Evaluation suite"]
  node_eval_suite["Suite runner<br/>[run_suite.py]"]
  node_eval_harness["Eval harness<br/>[harness.py]"]
  node_pipeline_eval["RAG quality"]
  node_retriever_eval["Retrieval quality<br/>[eval_retriever.py]"]
  node_reranker_eval["Reranker quality"]
  node_safety_eval["Safety checks"]
  node_ops_eval["Operational checks<br/>[eval_ops.py]"]
  node_other_evals["Other evals"]
end

subgraph group_evaluation_data["Evaluation data"]
  node_goldens["Golden datasets"]
end

node_user(("Chat user"))
node_llm_service{{"LLM service"}}
node_judge{{"LLM judge"}}
node_embeddings{{"Embedding service"}}

node_user -->|"asks"| node_chat_ui
node_chat_ui -->|"invokes"| node_rag_pipeline
node_rag_pipeline -->|"retrieves"| node_reranker
node_rag_pipeline -->|"generates"| node_generator
node_reranker -->|"fetches candidates"| node_retriever
node_retriever -->|"searches"| node_vector_store
node_transcripts -->|"loads and chunks"| node_retriever
node_retriever -.->|"embeds queries"| node_embeddings
node_generator -.->|"requests completion"| node_llm_service
node_eval_suite -->|"dispatches"| node_pipeline_eval
node_eval_suite -->|"dispatches"| node_retriever_eval
node_eval_suite -->|"dispatches"| node_reranker_eval
node_eval_suite -->|"dispatches"| node_safety_eval
node_eval_suite -->|"dispatches"| node_ops_eval
node_eval_suite -->|"dispatches"| node_other_evals
node_pipeline_eval -->|"evaluates"| node_rag_pipeline
node_retriever_eval -->|"evaluates"| node_reranker
node_reranker_eval -->|"evaluates"| node_reranker
node_pipeline_eval -->|"loads queries"| node_goldens
node_retriever_eval -->|"loads cases"| node_goldens
node_reranker_eval -->|"loads cases"| node_goldens
node_pipeline_eval -->|"summarizes"| node_eval_harness
node_retriever_eval -->|"summarizes"| node_eval_harness
node_pipeline_eval -.->|"scores outputs"| node_judge
node_retriever_eval -.->|"scores retrieval"| node_judge
node_reranker_eval -.->|"scores retrieval"| node_judge
node_ops_eval -->|"measures"| node_rag_pipeline

click node_chat_ui "https://github.com/campusx-official/rag-eval-deepeval/blob/main/src/app.py"
click node_rag_pipeline "https://github.com/campusx-official/rag-eval-deepeval/blob/main/src/rag_pipeline.py"
click node_generator "https://github.com/campusx-official/rag-eval-deepeval/blob/main/src/generator.py"
click node_reranker "https://github.com/campusx-official/rag-eval-deepeval/blob/main/src/reranker.py"
click node_retriever "https://github.com/campusx-official/rag-eval-deepeval/blob/main/src/retriever.py"
click node_transcripts "https://github.com/campusx-official/rag-eval-deepeval/tree/main/data"
click node_vector_store "https://github.com/campusx-official/rag-eval-deepeval/blob/main/src/retriever.py"
click node_eval_suite "https://github.com/campusx-official/rag-eval-deepeval/blob/main/evals/run_suite.py"
click node_eval_harness "https://github.com/campusx-official/rag-eval-deepeval/blob/main/evals/harness.py"
click node_pipeline_eval "https://github.com/campusx-official/rag-eval-deepeval/blob/main/evals/eval_rag_pipeline.py"
click node_retriever_eval "https://github.com/campusx-official/rag-eval-deepeval/blob/main/evals/eval_retriever.py"
click node_reranker_eval "https://github.com/campusx-official/rag-eval-deepeval/blob/main/evals/eval_retriever_with_reranker.py"
click node_safety_eval "https://github.com/campusx-official/rag-eval-deepeval/tree/main/evals"
click node_ops_eval "https://github.com/campusx-official/rag-eval-deepeval/blob/main/evals/eval_ops.py"
click node_other_evals "https://github.com/campusx-official/rag-eval-deepeval/tree/main/evals"
click node_goldens "https://github.com/campusx-official/rag-eval-deepeval/tree/main/goldens"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_chat_ui,node_rag_pipeline,node_generator,node_user toneBlue
class node_reranker,node_retriever,node_transcripts,node_vector_store toneAmber
class node_eval_suite,node_eval_harness,node_pipeline_eval,node_retriever_eval,node_reranker_eval,node_safety_eval,node_ops_eval,node_other_evals toneMint
class node_goldens toneRose
class node_llm_service,node_judge,node_embeddings toneIndigo
```

## Components

### Course assistant

- [Chat UI](https://github.com/campusx-official/rag-eval-deepeval/blob/main/src/app.py): `src/app.py`
- [RAG pipeline](https://github.com/campusx-official/rag-eval-deepeval/blob/main/src/rag_pipeline.py): `src/rag_pipeline.py`
- [Answer generator](https://github.com/campusx-official/rag-eval-deepeval/blob/main/src/generator.py): `src/generator.py`

### Knowledge retrieval

- [Reranking retriever](https://github.com/campusx-official/rag-eval-deepeval/blob/main/src/reranker.py): `src/reranker.py`
- [Vector retriever](https://github.com/campusx-official/rag-eval-deepeval/blob/main/src/retriever.py): `src/retriever.py`
- [Course transcripts](https://github.com/campusx-official/rag-eval-deepeval/tree/main/data): `data`
- [Chroma store](https://github.com/campusx-official/rag-eval-deepeval/blob/main/src/retriever.py): `src/retriever.py`

### Evaluation suite

- [Suite runner](https://github.com/campusx-official/rag-eval-deepeval/blob/main/evals/run_suite.py): `evals/run_suite.py`
- [Eval harness](https://github.com/campusx-official/rag-eval-deepeval/blob/main/evals/harness.py): `evals/harness.py`
- [RAG quality](https://github.com/campusx-official/rag-eval-deepeval/blob/main/evals/eval_rag_pipeline.py): `evals/eval_rag_pipeline.py`
- [Retrieval quality](https://github.com/campusx-official/rag-eval-deepeval/blob/main/evals/eval_retriever.py): `evals/eval_retriever.py`
- [Reranker quality](https://github.com/campusx-official/rag-eval-deepeval/blob/main/evals/eval_retriever_with_reranker.py): `evals/eval_retriever_with_reranker.py`
- [Safety checks](https://github.com/campusx-official/rag-eval-deepeval/tree/main/evals): `evals`
- [Operational checks](https://github.com/campusx-official/rag-eval-deepeval/blob/main/evals/eval_ops.py): `evals/eval_ops.py`
- [Other evals](https://github.com/campusx-official/rag-eval-deepeval/tree/main/evals): `evals`

### Evaluation data

- [Golden datasets](https://github.com/campusx-official/rag-eval-deepeval/tree/main/goldens): `goldens`

### Other components

- **Chat user**
- **LLM service**
- **LLM judge**
- **Embedding service**

## Connections

- Chat user → Chat UI: asks
- Chat UI → RAG pipeline: invokes
- RAG pipeline → Reranking retriever: retrieves
- RAG pipeline → Answer generator: generates
- Reranking retriever → Vector retriever: fetches candidates
- Vector retriever → Chroma store: searches
- Course transcripts → Vector retriever: loads and chunks
- Vector retriever → Embedding service: embeds queries
- Answer generator → LLM service: requests completion
- Suite runner → RAG quality: dispatches
- Suite runner → Retrieval quality: dispatches
- Suite runner → Reranker quality: dispatches
- Suite runner → Safety checks: dispatches
- Suite runner → Operational checks: dispatches
- Suite runner → Other evals: dispatches
- RAG quality → RAG pipeline: evaluates
- Retrieval quality → Reranking retriever: evaluates
- Reranker quality → Reranking retriever: evaluates
- RAG quality → Golden datasets: loads queries
- Retrieval quality → Golden datasets: loads cases
- Reranker quality → Golden datasets: loads cases
- RAG quality → Eval harness: summarizes
- Retrieval quality → Eval harness: summarizes
- RAG quality → LLM judge: scores outputs
- Retrieval quality → LLM judge: scores retrieval
- Reranker quality → LLM judge: scores retrieval
- Operational checks → RAG pipeline: measures