# sumeet-giri/ai-response-quality-evaluator-agent architecture

> Architecture diagram of the GitHub repository sumeet-giri/ai-response-quality-evaluator-agent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sumeet-giri/ai-response-quality-evaluator-agent
- Repository: https://github.com/sumeet-giri/ai-response-quality-evaluator-agent

## Overview

This is a multi-agent AI response-quality evaluator with a Streamlit interface and FastAPI backend. Users submit a question, response, and optional reference; evaluation scores relevance, accuracy, hallucination risk, and completeness, then produces a verdict and report. The app also supports CSV batch evaluation and historical dashboards. The graph includes the documented retrieval fallback and validation capabilities. Edges rely on excerpted code or SOURCE INDEX references; unsampled implementation details are not inferred.

Main entry points: [`backend/app/orchestration/orchestrator.py`](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/orchestration/orchestrator.py) (Evaluation Pipeline), [`backend/app/api/evaluate.py`](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/api/evaluate.py) (Evaluation Routes), [`frontend/pages_content/single_evaluation.py`](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/pages_content/single_evaluation.py) (Single Evaluation), [`frontend/pages_content/benchmark_validation.py`](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/pages_content/benchmark_validation.py) (Batch Evaluation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["Streamlit Interface"]
  node_frontend_app["Page Router<br/>[app.py]"]
  node_single_page["Single Evaluation"]
  node_batch_page["Batch Evaluation"]
  node_dashboard["History Dashboard<br/>[dashboard.py]"]
  node_api_client["Backend Client<br/>[api_client.py]"]
  node_batch_processor["Batch Processor<br/>[batch_processor.py]"]
  node_csv_parser["CSV Parser<br/>[csv_parser.py]"]
  node_pdf_export["PDF Reports<br/>[pdf_export.py]"]
  node_result_cards["Score Cards<br/>[result_cards.py]"]
  node_charts["Score Charts<br/>[charts.py]"]
  node_batch_charts["Batch Charts<br/>[batch_charts.py]"]
  node_badges["UI Metrics<br/>[badges.py]"]
  node_pipeline_viz["Pipeline View<br/>[pipeline_viz.py]"]
end

subgraph group_backend["Evaluation Backend"]
  node_api_main["FastAPI App<br/>[main.py]"]
  node_evaluate_api["Evaluation Routes<br/>[evaluate.py]"]
  node_orchestrator["Evaluation Pipeline<br/>[orchestrator.py]"]
end

subgraph group_agents["Scoring Agents"]
  node_relevance["Relevance Scorer<br/>[relevance_agent.py]"]
  node_accuracy["Accuracy Scorer<br/>[accuracy_agent.py]"]
  node_hallucination["Hallucination Scorer"]
  node_completeness["Completeness Scorer"]
  node_verdict["Verdict Scorer<br/>[verdict_agent.py]"]
end

subgraph group_knowledge["Reference Retrieval"]
  node_embedder["Text Embedder<br/>[embedder.py]"]
  node_similarity["Similarity<br/>[similarity.py]"]
  node_retriever["Reference Retriever<br/>[retriever.py]"]
  node_chroma[("Knowledge Collection<br/>[chroma_manager.py]")]
end

subgraph group_history["History and Validation"]
  node_history_api["History Routes<br/>[history.py]"]
  node_history_store[("Evaluation History<br/>[history_store.py]")]
  node_validation_api["Validation Route<br/>[validation.py]"]
  node_validator["Benchmark Validator<br/>[validator.py]"]
  node_benchmark_cases["Benchmark Cases<br/>[benchmark_cases.py]"]
end

node_user(("Evaluator"))
node_external_models(("Embedding Model"))

node_user -->|"uses"| node_frontend_app
node_frontend_app -->|"routes to"| node_single_page
node_frontend_app -->|"routes to"| node_batch_page
node_frontend_app -->|"routes to"| node_dashboard
node_single_page -->|"requests scores"| node_api_client
node_single_page -->|"renders"| node_result_cards
node_single_page -->|"renders"| node_charts
node_single_page -->|"renders"| node_pipeline_viz
node_single_page -->|"exports report"| node_pdf_export
node_batch_page -->|"parses upload"| node_csv_parser
node_batch_page -->|"runs rows"| node_batch_processor
node_batch_page -->|"exports report"| node_pdf_export
node_batch_page -->|"renders analytics"| node_batch_charts
node_batch_page -->|"renders metrics"| node_badges
node_dashboard -->|"loads history"| node_api_client
node_dashboard -->|"renders charts"| node_charts
node_dashboard -->|"renders charts"| node_batch_charts
node_dashboard -->|"renders metrics"| node_badges
node_api_client -->|"calls evaluation"| node_evaluate_api
node_api_client -->|"fetches history"| node_history_api
node_api_main -->|"mounts routes"| node_evaluate_api
node_api_main -->|"mounts routes"| node_history_api
node_api_main -->|"mounts routes"| node_validation_api
node_evaluate_api -->|"runs pipeline"| node_orchestrator
node_evaluate_api -->|"scores relevance"| node_relevance
node_evaluate_api -->|"scores accuracy"| node_accuracy
node_evaluate_api -->|"scores hallucination"| node_hallucination
node_evaluate_api -->|"scores completeness"| node_completeness
node_orchestrator -->|"retrieves evidence"| node_retriever
node_orchestrator -->|"scores relevance"| node_relevance
node_orchestrator -->|"scores accuracy"| node_accuracy
node_orchestrator -->|"scores claims"| node_hallucination
node_orchestrator -->|"scores coverage"| node_completeness
node_orchestrator -->|"builds verdict"| node_verdict
node_orchestrator -->|"records results"| node_history_store
node_relevance -->|"embeds text"| node_embedder
node_relevance -->|"compares vectors"| node_similarity
node_accuracy -->|"embeds text"| node_embedder
node_accuracy -->|"compares vectors"| node_similarity
node_hallucination -->|"embeds claims"| node_embedder
node_hallucination -->|"compares claims"| node_similarity
node_retriever -->|"embeds query"| node_embedder
node_retriever -->|"queries collection"| node_chroma
node_embedder -.->|"uses"| node_external_models
node_history_api -->|"reads history"| node_history_store
node_validation_api -->|"runs validation"| node_validator
node_validator -->|"evaluates relevance"| node_relevance
node_validator -->|"evaluates accuracy"| node_accuracy

click node_frontend_app "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/app.py"
click node_single_page "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/pages_content/single_evaluation.py"
click node_batch_page "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/pages_content/benchmark_validation.py"
click node_dashboard "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/pages_content/dashboard.py"
click node_api_client "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/utils/api_client.py"
click node_batch_processor "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/utils/batch_processor.py"
click node_csv_parser "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/utils/csv_parser.py"
click node_pdf_export "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/utils/pdf_export.py"
click node_result_cards "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/components/result_cards.py"
click node_charts "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/components/charts.py"
click node_batch_charts "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/components/batch_charts.py"
click node_badges "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/components/badges.py"
click node_pipeline_viz "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/components/pipeline_viz.py"
click node_api_main "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/main.py"
click node_evaluate_api "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/api/evaluate.py"
click node_orchestrator "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/orchestration/orchestrator.py"
click node_relevance "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/agents/relevance_agent.py"
click node_accuracy "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/agents/accuracy_agent.py"
click node_hallucination "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/agents/hallucination_agent.py"
click node_completeness "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/agents/completeness_agent.py"
click node_verdict "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/agents/verdict_agent.py"
click node_embedder "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/services/embedder.py"
click node_similarity "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/services/similarity.py"
click node_retriever "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/services/retriever.py"
click node_chroma "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/services/chroma_manager.py"
click node_history_api "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/api/history.py"
click node_history_store "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/services/history_store.py"
click node_validation_api "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/api/validation.py"
click node_validator "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/validation/validator.py"
click node_benchmark_cases "https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/validation/benchmark_cases.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_frontend_app,node_single_page,node_batch_page,node_dashboard,node_api_client,node_batch_processor,node_csv_parser,node_pdf_export,node_result_cards,node_charts,node_batch_charts,node_badges,node_pipeline_viz toneBlue
class node_api_main,node_evaluate_api,node_orchestrator toneAmber
class node_relevance,node_accuracy,node_hallucination,node_completeness,node_verdict toneMint
class node_embedder,node_similarity,node_retriever,node_chroma toneRose
class node_history_api,node_history_store,node_validation_api,node_validator,node_benchmark_cases,node_user,node_external_models toneIndigo
```

## Components

### Streamlit Interface

- [Page Router](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/app.py): `frontend/app.py`
- [Single Evaluation](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/pages_content/single_evaluation.py): `frontend/pages_content/single_evaluation.py`
- [Batch Evaluation](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/pages_content/benchmark_validation.py): `frontend/pages_content/benchmark_validation.py`
- [History Dashboard](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/pages_content/dashboard.py): `frontend/pages_content/dashboard.py`
- [Backend Client](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/utils/api_client.py): `frontend/utils/api_client.py`
- [Batch Processor](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/utils/batch_processor.py): `frontend/utils/batch_processor.py`
- [CSV Parser](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/utils/csv_parser.py): `frontend/utils/csv_parser.py`
- [PDF Reports](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/utils/pdf_export.py): `frontend/utils/pdf_export.py`
- [Score Cards](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/components/result_cards.py): `frontend/components/result_cards.py`
- [Score Charts](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/components/charts.py): `frontend/components/charts.py`
- [Batch Charts](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/components/batch_charts.py): `frontend/components/batch_charts.py`
- [UI Metrics](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/components/badges.py): `frontend/components/badges.py`
- [Pipeline View](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/frontend/components/pipeline_viz.py): `frontend/components/pipeline_viz.py`

### Evaluation Backend

- [FastAPI App](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/main.py): `backend/app/main.py`
- [Evaluation Routes](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/api/evaluate.py): `backend/app/api/evaluate.py`
- [Evaluation Pipeline](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/orchestration/orchestrator.py): `backend/app/orchestration/orchestrator.py`

### Scoring Agents

- [Relevance Scorer](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/agents/relevance_agent.py): `backend/app/agents/relevance_agent.py`
- [Accuracy Scorer](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/agents/accuracy_agent.py): `backend/app/agents/accuracy_agent.py`
- [Hallucination Scorer](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/agents/hallucination_agent.py): `backend/app/agents/hallucination_agent.py`
- [Completeness Scorer](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/agents/completeness_agent.py): `backend/app/agents/completeness_agent.py`
- [Verdict Scorer](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/agents/verdict_agent.py): `backend/app/agents/verdict_agent.py`

### Reference Retrieval

- [Text Embedder](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/services/embedder.py): `backend/app/services/embedder.py`
- [Similarity](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/services/similarity.py): `backend/app/services/similarity.py`
- [Reference Retriever](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/services/retriever.py): `backend/app/services/retriever.py`
- [Knowledge Collection](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/services/chroma_manager.py): `backend/app/services/chroma_manager.py`

### History and Validation

- [History Routes](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/api/history.py): `backend/app/api/history.py`
- [Evaluation History](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/services/history_store.py): `backend/app/services/history_store.py`
- [Validation Route](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/api/validation.py): `backend/app/api/validation.py`
- [Benchmark Validator](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/validation/validator.py): `backend/app/validation/validator.py`
- [Benchmark Cases](https://github.com/sumeet-giri/ai-response-quality-evaluator-agent/blob/main/backend/app/validation/benchmark_cases.py): `backend/app/validation/benchmark_cases.py`

### Other components

- **Evaluator**
- **Embedding Model**

## Connections

- Evaluator → Page Router: uses (evidence: `frontend/app.py`)
- Page Router → Single Evaluation: routes to (evidence: `frontend/app.py`)
- Page Router → Batch Evaluation: routes to (evidence: `frontend/app.py`)
- Page Router → History Dashboard: routes to (evidence: `frontend/app.py`)
- Single Evaluation → Backend Client: requests scores (evidence: `frontend/pages_content/single_evaluation.py`)
- Single Evaluation → Score Cards: renders (evidence: `frontend/pages_content/single_evaluation.py`)
- Single Evaluation → Score Charts: renders (evidence: `frontend/pages_content/single_evaluation.py`)
- Single Evaluation → Pipeline View: renders (evidence: `frontend/pages_content/single_evaluation.py`)
- Single Evaluation → PDF Reports: exports report (evidence: `frontend/pages_content/single_evaluation.py`)
- Batch Evaluation → CSV Parser: parses upload (evidence: `frontend/pages_content/benchmark_validation.py`)
- Batch Evaluation → Batch Processor: runs rows (evidence: `frontend/pages_content/benchmark_validation.py`)
- Batch Evaluation → PDF Reports: exports report (evidence: `frontend/pages_content/benchmark_validation.py`)
- Batch Evaluation → Batch Charts: renders analytics (evidence: `frontend/pages_content/benchmark_validation.py`)
- Batch Evaluation → UI Metrics: renders metrics (evidence: `frontend/pages_content/benchmark_validation.py`)
- History Dashboard → Backend Client: loads history (evidence: `frontend/pages_content/dashboard.py`)
- History Dashboard → Score Charts: renders charts (evidence: `frontend/pages_content/dashboard.py`)
- History Dashboard → Batch Charts: renders charts (evidence: `frontend/pages_content/dashboard.py`)
- History Dashboard → UI Metrics: renders metrics (evidence: `frontend/pages_content/dashboard.py`)
- Backend Client → Evaluation Routes: calls evaluation (evidence: `frontend/utils/api_client.py`)
- Backend Client → History Routes: fetches history (evidence: `frontend/utils/api_client.py`)
- FastAPI App → Evaluation Routes: mounts routes (evidence: `backend/app/main.py`)
- FastAPI App → History Routes: mounts routes (evidence: `backend/app/main.py`)
- FastAPI App → Validation Route: mounts routes (evidence: `backend/app/main.py`)
- Evaluation Routes → Evaluation Pipeline: runs pipeline (evidence: `backend/app/api/evaluate.py`)
- Evaluation Routes → Relevance Scorer: scores relevance (evidence: `backend/app/api/evaluate.py`)
- Evaluation Routes → Accuracy Scorer: scores accuracy (evidence: `backend/app/api/evaluate.py`)
- Evaluation Routes → Hallucination Scorer: scores hallucination (evidence: `backend/app/api/evaluate.py`)
- Evaluation Routes → Completeness Scorer: scores completeness (evidence: `backend/app/api/evaluate.py`)
- Evaluation Pipeline → Reference Retriever: retrieves evidence (evidence: `backend/app/orchestration/orchestrator.py`)
- Evaluation Pipeline → Relevance Scorer: scores relevance (evidence: `backend/app/orchestration/orchestrator.py`)
- Evaluation Pipeline → Accuracy Scorer: scores accuracy (evidence: `backend/app/orchestration/orchestrator.py`)
- Evaluation Pipeline → Hallucination Scorer: scores claims (evidence: `backend/app/orchestration/orchestrator.py`)
- Evaluation Pipeline → Completeness Scorer: scores coverage (evidence: `backend/app/orchestration/orchestrator.py`)
- Evaluation Pipeline → Verdict Scorer: builds verdict (evidence: `backend/app/orchestration/orchestrator.py`)
- Evaluation Pipeline → Evaluation History: records results (evidence: `backend/app/orchestration/orchestrator.py`)
- Relevance Scorer → Text Embedder: embeds text (evidence: `backend/app/agents/relevance_agent.py`)
- Relevance Scorer → Similarity: compares vectors (evidence: `backend/app/agents/relevance_agent.py`)
- Accuracy Scorer → Text Embedder: embeds text (evidence: `backend/app/agents/accuracy_agent.py`)
- Accuracy Scorer → Similarity: compares vectors (evidence: `backend/app/agents/accuracy_agent.py`)
- Hallucination Scorer → Text Embedder: embeds claims (evidence: `backend/app/agents/hallucination_agent.py`)
- Hallucination Scorer → Similarity: compares claims (evidence: `backend/app/agents/hallucination_agent.py`)
- Reference Retriever → Text Embedder: embeds query (evidence: `backend/app/services/retriever.py`)
- Reference Retriever → Knowledge Collection: queries collection (evidence: `backend/app/services/retriever.py`)
- Text Embedder → Embedding Model: uses
- History Routes → Evaluation History: reads history (evidence: `backend/app/api/history.py`)
- Validation Route → Benchmark Validator: runs validation (evidence: `backend/app/api/validation.py`)
- Benchmark Validator → Relevance Scorer: evaluates relevance (evidence: `backend/app/validation/validator.py`)
- Benchmark Validator → Accuracy Scorer: evaluates accuracy (evidence: `backend/app/validation/validator.py`)
