# sdkdeepa/ai-quality-gate architecture

> Architecture diagram of the GitHub repository sdkdeepa/ai-quality-gate, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sdkdeepa/ai-quality-gate
- Repository: https://github.com/sdkdeepa/ai-quality-gate

## Overview

AI Quality Gate is a release-readiness platform, not a chatbot: engineers submit golden-dataset evaluation runs, inspect normalized results, then apply release policy and optional baseline comparison to produce an auditable PASS/WARN/BLOCK decision. The graph includes the separate RAG system-under-test, framework/provider signal sources, persistence, reports, and the read-oriented dashboard. Documented framework and tracing capabilities are included; links without direct source evidence are omitted. Unsampled API and integration internals are represented only where README or indexed references support them.

Main entry points: [`backend/app/services/policy_service.py`](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/services/policy_service.py) (Policy service), [`backend/app/main.py`](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/main.py) (FastAPI app), [`backend/app/evaluation/runner.py`](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/evaluation/runner.py) (Evaluation runner), [`backend/app/services/evaluation_service.py`](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/services/evaluation_service.py) (Evaluation service)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ui["Engineering dashboard"]
  node_dashboard["Dashboard pages"]
  node_api_client["API client<br/>[client.ts]"]
end

subgraph group_api["API and orchestration"]
  node_app["FastAPI app<br/>[main.py]"]
  node_evaluation_api["Evaluation endpoints<br/>[evaluations.py]"]
  node_dataset_api["Dataset endpoints<br/>[datasets.py]"]
  node_gate_api["Gate endpoints<br/>[gate.py]"]
  node_report_api["Report endpoints<br/>[reports.py]"]
  node_dataset_service["Dataset service<br/>[dataset_service.py]"]
  node_evaluation_service["Evaluation service"]
end

subgraph group_eval["Evaluation signals"]
  node_runner["Evaluation runner<br/>[runner.py]"]
  node_deterministic_eval["Deterministic evaluators<br/>[deterministic.py]"]
  node_framework_evals["Framework evaluators"]
  node_providers["Model providers"]
  node_case_results["Case results<br/>[case_result.py]"]
end

subgraph group_gate["Release decisions"]
  node_policy_service["Policy service<br/>[policy_service.py]"]
  node_policy_engine["Policy engine<br/>[engine.py]"]
  node_release_policy["Release policies<br/>[release_policy.py]"]
  node_baseline["Baselines<br/>[baseline.py]"]
  node_decision["Gate decision<br/>[gate_decision.py]"]
  node_report_service["Report generation<br/>[report_service.py]"]
end

subgraph group_rag["RAG system under test"]
  node_rag_service["RAG service<br/>[rag_service.py]"]
  node_rag_pipeline["RAG pipeline<br/>[pipeline.py]"]
  node_retriever["Retriever<br/>[retriever.py]"]
  node_corpus_service["Corpus ingestion<br/>[corpus_service.py]"]
  node_vector_store[("Vector store<br/>[vector_store.py]")]
end

subgraph group_state["Data and observability"]
  node_run_store[("Run and case storage<br/>[in_memory.py]")]
  node_policy_store[("Policy and decision storage<br/>[sqlite.py]")]
  node_tracing["Tracing<br/>[tracing.py]"]
end

node_engineer(("Engineer"))

node_engineer -->|"uses"| node_dashboard
node_dashboard -->|"requests data"| node_api_client
node_api_client -->|"calls API"| node_app
node_app -->|"registers routes"| node_evaluation_api
node_app -->|"registers routes"| node_dataset_api
node_app -->|"registers routes"| node_gate_api
node_app -->|"registers routes"| node_report_api
node_evaluation_api -.->|"dispatches"| node_evaluation_service
node_dataset_api -.->|"dispatches"| node_dataset_service
node_evaluation_service -.->|"loads datasets"| node_dataset_service
node_evaluation_service -.->|"runs evaluation"| node_runner
node_runner -->|"requests responses"| node_providers
node_runner -->|"executes metrics"| node_deterministic_eval
node_runner -->|"executes optional metrics"| node_framework_evals
node_runner -->|"produces results"| node_case_results
node_evaluation_service -.->|"stores runs"| node_run_store
node_evaluation_service -.->|"stores results"| node_case_results
node_gate_api -.->|"dispatches"| node_policy_service
node_policy_service -->|"requests decision"| node_policy_engine
node_policy_service -->|"reads run results"| node_run_store
node_policy_service -->|"resolves policy"| node_release_policy
node_policy_service -->|"loads baseline"| node_baseline
node_policy_service -->|"persists decision"| node_policy_store
node_policy_engine -->|"evaluates metrics"| node_case_results
node_policy_engine -->|"computes outcome"| node_decision
node_report_api -.->|"dispatches"| node_report_service
node_report_service -->|"loads decision data"| node_policy_service
node_rag_service -->|"answers queries"| node_rag_pipeline
node_rag_pipeline -->|"retrieves context"| node_retriever
node_rag_pipeline -->|"generates answer"| node_providers
node_rag_service -->|"grades case"| node_runner
node_corpus_service -->|"writes chunks"| node_vector_store
node_retriever -->|"searches chunks"| node_vector_store
node_app -->|"configures tracing"| node_tracing

click node_dashboard "https://github.com/sdkdeepa/ai-quality-gate/tree/main/frontend/src/pages"
click node_api_client "https://github.com/sdkdeepa/ai-quality-gate/blob/main/frontend/src/api/client.ts"
click node_app "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/main.py"
click node_evaluation_api "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/api/evaluations.py"
click node_dataset_api "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/api/datasets.py"
click node_gate_api "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/api/gate.py"
click node_report_api "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/api/reports.py"
click node_dataset_service "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/services/dataset_service.py"
click node_evaluation_service "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/services/evaluation_service.py"
click node_runner "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/evaluation/runner.py"
click node_deterministic_eval "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/evaluation/deterministic.py"
click node_framework_evals "https://github.com/sdkdeepa/ai-quality-gate/tree/main/backend/app/evaluation"
click node_providers "https://github.com/sdkdeepa/ai-quality-gate/tree/main/backend/app/providers"
click node_case_results "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/domain/case_result.py"
click node_policy_service "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/services/policy_service.py"
click node_policy_engine "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/policy/engine.py"
click node_release_policy "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/domain/release_policy.py"
click node_baseline "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/domain/baseline.py"
click node_decision "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/domain/gate_decision.py"
click node_report_service "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/services/report_service.py"
click node_rag_service "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/services/rag_service.py"
click node_rag_pipeline "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/rag/pipeline.py"
click node_retriever "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/rag/retriever.py"
click node_corpus_service "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/rag/corpus_service.py"
click node_vector_store "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/rag/vector_store.py"
click node_run_store "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/repositories/in_memory.py"
click node_policy_store "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/repositories/sqlite.py"
click node_tracing "https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/observability/tracing.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dashboard,node_api_client toneBlue
class node_app,node_evaluation_api,node_dataset_api,node_gate_api,node_report_api,node_dataset_service,node_evaluation_service toneAmber
class node_runner,node_deterministic_eval,node_framework_evals,node_providers,node_case_results toneMint
class node_policy_service,node_policy_engine,node_release_policy,node_baseline,node_decision,node_report_service toneRose
class node_rag_service,node_rag_pipeline,node_retriever,node_corpus_service,node_vector_store,node_engineer toneIndigo
class node_run_store,node_policy_store,node_tracing toneTeal
```

## Components

### Engineering dashboard

- [Dashboard pages](https://github.com/sdkdeepa/ai-quality-gate/tree/main/frontend/src/pages): `frontend/src/pages`
- [API client](https://github.com/sdkdeepa/ai-quality-gate/blob/main/frontend/src/api/client.ts): `frontend/src/api/client.ts`

### API and orchestration

- [FastAPI app](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/main.py): `backend/app/main.py`
- [Evaluation endpoints](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/api/evaluations.py): `backend/app/api/evaluations.py`
- [Dataset endpoints](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/api/datasets.py): `backend/app/api/datasets.py`
- [Gate endpoints](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/api/gate.py): `backend/app/api/gate.py`
- [Report endpoints](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/api/reports.py): `backend/app/api/reports.py`
- [Dataset service](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/services/dataset_service.py): `backend/app/services/dataset_service.py`
- [Evaluation service](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/services/evaluation_service.py): `backend/app/services/evaluation_service.py`

### Evaluation signals

- [Evaluation runner](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/evaluation/runner.py): `backend/app/evaluation/runner.py`
- [Deterministic evaluators](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/evaluation/deterministic.py): `backend/app/evaluation/deterministic.py`
- [Framework evaluators](https://github.com/sdkdeepa/ai-quality-gate/tree/main/backend/app/evaluation): `backend/app/evaluation`
- [Model providers](https://github.com/sdkdeepa/ai-quality-gate/tree/main/backend/app/providers): `backend/app/providers`
- [Case results](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/domain/case_result.py): `backend/app/domain/case_result.py`

### Release decisions

- [Policy service](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/services/policy_service.py): `backend/app/services/policy_service.py`
- [Policy engine](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/policy/engine.py): `backend/app/policy/engine.py`
- [Release policies](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/domain/release_policy.py): `backend/app/domain/release_policy.py`
- [Baselines](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/domain/baseline.py): `backend/app/domain/baseline.py`
- [Gate decision](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/domain/gate_decision.py): `backend/app/domain/gate_decision.py`
- [Report generation](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/services/report_service.py): `backend/app/services/report_service.py`

### RAG system under test

- [RAG service](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/services/rag_service.py): `backend/app/services/rag_service.py`
- [RAG pipeline](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/rag/pipeline.py): `backend/app/rag/pipeline.py`
- [Retriever](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/rag/retriever.py): `backend/app/rag/retriever.py`
- [Corpus ingestion](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/rag/corpus_service.py): `backend/app/rag/corpus_service.py`
- [Vector store](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/rag/vector_store.py): `backend/app/rag/vector_store.py`

### Data and observability

- [Run and case storage](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/repositories/in_memory.py): `backend/app/repositories/in_memory.py`
- [Policy and decision storage](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/repositories/sqlite.py): `backend/app/repositories/sqlite.py`
- [Tracing](https://github.com/sdkdeepa/ai-quality-gate/blob/main/backend/app/observability/tracing.py): `backend/app/observability/tracing.py`

### Other components

- **Engineer**

## Connections

- Engineer → Dashboard pages: uses
- Dashboard pages → API client: requests data (evidence: `frontend/src/pages/RunDetail.tsx`)
- API client → FastAPI app: calls API (evidence: `frontend/src/api/client.ts`)
- FastAPI app → Evaluation endpoints: registers routes (evidence: `backend/app/main.py`)
- FastAPI app → Dataset endpoints: registers routes (evidence: `backend/app/main.py`)
- FastAPI app → Gate endpoints: registers routes (evidence: `backend/app/main.py`)
- FastAPI app → Report endpoints: registers routes (evidence: `backend/app/main.py`)
- Evaluation endpoints → Evaluation service: dispatches
- Dataset endpoints → Dataset service: dispatches
- Evaluation service → Dataset service: loads datasets
- Evaluation service → Evaluation runner: runs evaluation
- Evaluation runner → Model providers: requests responses (evidence: `backend/app/evaluation/runner.py`)
- Evaluation runner → Deterministic evaluators: executes metrics (evidence: `backend/app/main.py`)
- Evaluation runner → Framework evaluators: executes optional metrics (evidence: `backend/app/main.py`)
- Evaluation runner → Case results: produces results (evidence: `backend/app/evaluation/runner.py`)
- Evaluation service → Run and case storage: stores runs
- Evaluation service → Case results: stores results
- Gate endpoints → Policy service: dispatches
- Policy service → Policy engine: requests decision (evidence: `backend/app/services/policy_service.py`)
- Policy service → Run and case storage: reads run results (evidence: `backend/app/services/policy_service.py`)
- Policy service → Release policies: resolves policy (evidence: `backend/app/services/policy_service.py`)
- Policy service → Baselines: loads baseline (evidence: `backend/app/services/policy_service.py`)
- Policy service → Policy and decision storage: persists decision (evidence: `backend/app/services/policy_service.py`)
- Policy engine → Case results: evaluates metrics (evidence: `backend/app/policy/engine.py`)
- Policy engine → Gate decision: computes outcome (evidence: `backend/app/policy/engine.py`)
- Report endpoints → Report generation: dispatches
- Report generation → Policy service: loads decision data (evidence: `backend/app/main.py`)
- RAG service → RAG pipeline: answers queries (evidence: `backend/app/services/rag_service.py`)
- RAG pipeline → Retriever: retrieves context (evidence: `backend/app/rag/pipeline.py`)
- RAG pipeline → Model providers: generates answer (evidence: `backend/app/rag/pipeline.py`)
- RAG service → Evaluation runner: grades case (evidence: `backend/app/services/rag_service.py`)
- Corpus ingestion → Vector store: writes chunks (evidence: `backend/app/rag/corpus_service.py`)
- Retriever → Vector store: searches chunks (evidence: `backend/app/rag/retriever.py`)
- FastAPI app → Tracing: configures tracing (evidence: `backend/app/main.py`)
