# mayureshmohnalkar22/llm-output-arbitrator architecture

> Architecture diagram of the GitHub repository mayureshmohnalkar22/llm-output-arbitrator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mayureshmohnalkar22/llm-output-arbitrator
- Repository: https://github.com/mayureshmohnalkar22/llm-output-arbitrator

## Overview

This is a local service for evaluating LLM outputs through three independent critics, routing disagreements or missing critiques to adjudication, and returning an auditable verdict. The main user path runs from the explorer or API through validation, parallel criticism, disagreement detection, synthesis or adjudication, and persistent history. The README documents batch evaluation, analytics, and live providers; those capabilities are included. Source excerpts show API, workflow, provider, storage, and explorer wiring; other README-described behavior is represented without claiming unsampled implementation details.

Main entry points: [`app/workflow.py`](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/workflow.py) (Workflow engine), [`app/main.py`](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/main.py) (Arbitration API), [`dashboard/explorer.py`](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/dashboard/explorer.py) (Verdict explorer), [`app/providers.py`](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/providers.py) (Provider routing)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["User interfaces"]
  node_explorer["Verdict explorer<br/>[explorer.py]"]
  node_api["Arbitration API<br/>[main.py]"]
  node_annotations["Evidence annotations<br/>[annotations.py]"]
end

subgraph group_orchestration["Arbitration workflow"]
  node_request["Request models<br/>[models.py]"]
  node_workflow["Workflow engine<br/>[workflow.py]"]
  node_collection["Collect disagreements<br/>[workflow.py]"]
  node_adjudication["Adjudicator<br/>[workflow.py]"]
  node_synthesis["Direct synthesis<br/>[workflow.py]"]
  node_settings["Runtime settings<br/>[config.py]"]
end

subgraph group_evaluation["Evaluation providers"]
  node_accuracy["Accuracy critic<br/>[workflow.py]"]
  node_logic["Logic critic<br/>[workflow.py]"]
  node_completeness["Completeness critic<br/>[workflow.py]"]
  node_providers["Provider routing<br/>[providers.py]"]
end

subgraph group_records["Verdicts and history"]
  node_result["Verdict and telemetry<br/>[models.py]"]
  node_sqlite[("SQLite audit store<br/>[storage.py]")]
end

node_user(("User"))
node_models{{"Live model services"}}

node_user -->|"submits request"| node_explorer
node_explorer -->|"calls API"| node_api
node_user -->|"submits request"| node_api
node_api -->|"runs arbitration"| node_workflow
node_api -->|"validates input"| node_request
node_workflow -->|"parses request"| node_request
node_workflow -->|"dispatches"| node_accuracy
node_workflow -->|"dispatches"| node_logic
node_workflow -->|"dispatches"| node_completeness
node_accuracy -->|"evaluates via"| node_providers
node_logic -->|"evaluates via"| node_providers
node_completeness -->|"evaluates via"| node_providers
node_providers -.->|"calls live models"| node_models
node_workflow -->|"collects critiques"| node_collection
node_collection -->|"routes conflicts"| node_adjudication
node_collection -->|"routes agreement"| node_synthesis
node_adjudication -->|"requests decision"| node_providers
node_adjudication -->|"produces verdict"| node_result
node_synthesis -->|"produces verdict"| node_result
node_workflow -->|"uses settings"| node_settings
node_workflow -->|"returns result"| node_result
node_api -->|"saves verdict"| node_sqlite
node_api -->|"reads history and analytics"| node_sqlite
node_sqlite -->|"stores verdict"| node_result
node_api -.->|"serves API responses"| node_explorer
node_explorer -->|"renders evidence"| node_annotations
node_explorer -->|"displays verdict"| node_result

click node_explorer "https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/dashboard/explorer.py"
click node_api "https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/main.py"
click node_request "https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/models.py"
click node_workflow "https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/workflow.py"
click node_accuracy "https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/workflow.py"
click node_logic "https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/workflow.py"
click node_completeness "https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/workflow.py"
click node_providers "https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/providers.py"
click node_collection "https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/workflow.py"
click node_adjudication "https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/workflow.py"
click node_synthesis "https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/workflow.py"
click node_result "https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/models.py"
click node_annotations "https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/annotations.py"
click node_sqlite "https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/storage.py"
click node_settings "https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/config.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_explorer,node_api,node_annotations,node_user toneBlue
class node_request,node_workflow,node_collection,node_adjudication,node_synthesis,node_settings toneAmber
class node_accuracy,node_logic,node_completeness,node_providers toneMint
class node_result,node_sqlite toneRose
class node_models toneIndigo
```

## Components

### User interfaces

- [Verdict explorer](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/dashboard/explorer.py): `dashboard/explorer.py`
- [Arbitration API](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/main.py): `app/main.py`
- [Evidence annotations](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/annotations.py): `app/annotations.py`

### Arbitration workflow

- [Request models](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/models.py): `app/models.py`
- [Workflow engine](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/workflow.py): `app/workflow.py`
- [Collect disagreements](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/workflow.py): `app/workflow.py`
- [Adjudicator](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/workflow.py): `app/workflow.py`
- [Direct synthesis](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/workflow.py): `app/workflow.py`
- [Runtime settings](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/config.py): `app/config.py`

### Evaluation providers

- [Accuracy critic](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/workflow.py): `app/workflow.py`
- [Logic critic](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/workflow.py): `app/workflow.py`
- [Completeness critic](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/workflow.py): `app/workflow.py`
- [Provider routing](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/providers.py): `app/providers.py`

### Verdicts and history

- [Verdict and telemetry](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/models.py): `app/models.py`
- [SQLite audit store](https://github.com/mayureshmohnalkar22/llm-output-arbitrator/blob/main/app/storage.py): `app/storage.py`

### Other components

- **User**
- **Live model services**

## Connections

- User → Verdict explorer: submits request (evidence: `README.md`)
- Verdict explorer → Arbitration API: calls API (evidence: `dashboard/explorer.py`)
- User → Arbitration API: submits request (evidence: `README.md`)
- Arbitration API → Workflow engine: runs arbitration (evidence: `app/main.py`)
- Arbitration API → Request models: validates input (evidence: `app/main.py`)
- Workflow engine → Request models: parses request (evidence: `app/workflow.py`)
- Workflow engine → Accuracy critic: dispatches (evidence: `app/workflow.py`)
- Workflow engine → Logic critic: dispatches (evidence: `app/workflow.py`)
- Workflow engine → Completeness critic: dispatches (evidence: `app/workflow.py`)
- Accuracy critic → Provider routing: evaluates via (evidence: `app/workflow.py`)
- Logic critic → Provider routing: evaluates via (evidence: `app/workflow.py`)
- Completeness critic → Provider routing: evaluates via (evidence: `app/workflow.py`)
- Provider routing → Live model services: calls live models (evidence: `README.md`)
- Workflow engine → Collect disagreements: collects critiques (evidence: `app/workflow.py`)
- Collect disagreements → Adjudicator: routes conflicts (evidence: `app/workflow.py`)
- Collect disagreements → Direct synthesis: routes agreement (evidence: `app/workflow.py`)
- Adjudicator → Provider routing: requests decision (evidence: `app/workflow.py`)
- Adjudicator → Verdict and telemetry: produces verdict (evidence: `app/workflow.py`)
- Direct synthesis → Verdict and telemetry: produces verdict (evidence: `app/workflow.py`)
- Workflow engine → Runtime settings: uses settings (evidence: `app/workflow.py`)
- Workflow engine → Verdict and telemetry: returns result (evidence: `app/workflow.py`)
- Arbitration API → SQLite audit store: saves verdict (evidence: `app/main.py`)
- Arbitration API → SQLite audit store: reads history and analytics (evidence: `app/main.py`)
- SQLite audit store → Verdict and telemetry: stores verdict (evidence: `app/storage.py`)
- Arbitration API → Verdict explorer: serves API responses (evidence: `README.md`)
- Verdict explorer → Evidence annotations: renders evidence (evidence: `dashboard/explorer.py`)
- Verdict explorer → Verdict and telemetry: displays verdict (evidence: `dashboard/explorer.py`)
