# almolsiaboelm-tech/medical-ai-evaluation-lab architecture

> Architecture diagram of the GitHub repository almolsiaboelm-tech/medical-ai-evaluation-lab, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/almolsiaboelm-tech/medical-ai-evaluation-lab
- Repository: https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab

## Overview

Medical AI Evaluation Lab is an experimental workflow for generating model responses to synthetic medical cases and evaluating them independently with a structured rubric. Its main path runs Day 4 cases through a MedGemma client, saves raw outputs, combines them with human-scored cases, and writes safety-aware evaluation results. A separate synthetic-case runner uses the same scoring engine. Scope and rubric documents define the evaluation boundaries and criteria. The graph follows documented workflows and sampled calls; unsampled details are omitted.

Main entry points: [`src/run_day4_evaluation.py`](https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/src/run_day4_evaluation.py) (Day 4 evaluation runner), [`src/run_medgemma_day4.py`](https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/src/run_medgemma_day4.py) (Generation runner), [`src/run_evaluation.py`](https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/src/run_evaluation.py) (Synthetic evaluation runner), [`medgemma_client.py`](https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/medgemma_client.py) (MedGemma client)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_definition["Evaluation definition"]
  node_scope["Evaluation scope"]
  node_rubric["Evaluation rubric"]
end

subgraph group_generation["Model generation"]
  node_day4_cases["Day 4 cases<br/>[day4_cases.json]"]
  node_generation_runner["Generation runner"]
  node_medgemma_client["MedGemma client<br/>[medgemma_client.py]"]
  node_raw_outputs["Raw model outputs"]
end

subgraph group_assessment["Scoring and results"]
  node_scored_cases["Scored Day 4 cases"]
  node_day4_runner["Day 4 evaluation runner"]
  node_synthetic_cases["Synthetic cases"]
  node_synthetic_runner["Synthetic evaluation runner<br/>[run_evaluation.py]"]
  node_scoring_engine["Scoring engine<br/>[evaluator.py]"]
  node_day4_results["Day 4 results"]
  node_synthetic_results["Synthetic results"]
end

node_remote_endpoint{{"Remote model endpoint"}}
node_human_reviewer(("Human reviewer"))

node_generation_runner -->|"reads cases"| node_day4_cases
node_generation_runner -->|"requests generation"| node_medgemma_client
node_medgemma_client -->|"sends prompt"| node_remote_endpoint
node_generation_runner -->|"writes outputs"| node_raw_outputs
node_human_reviewer -->|"reviews against"| node_rubric
node_human_reviewer -->|"records scores"| node_scored_cases
node_day4_runner -->|"reads scores"| node_scored_cases
node_day4_runner -->|"reads responses"| node_raw_outputs
node_day4_runner -->|"evaluates scores"| node_scoring_engine
node_day4_runner -->|"writes results"| node_day4_results
node_synthetic_runner -->|"reads cases"| node_synthetic_cases
node_synthetic_runner -->|"evaluates scores"| node_scoring_engine
node_synthetic_runner -->|"writes results"| node_synthetic_results

click node_scope "https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/scope/evaluation_scope_v1.md"
click node_rubric "https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/rubric/evaluation_rubric_v1.md"
click node_day4_cases "https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/cases/day4_cases.json"
click node_generation_runner "https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/src/run_medgemma_day4.py"
click node_medgemma_client "https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/medgemma_client.py"
click node_raw_outputs "https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/results/model_outputs_day4.json"
click node_scored_cases "https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/cases/day4_scored_cases.json"
click node_day4_runner "https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/src/run_day4_evaluation.py"
click node_synthetic_cases "https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/cases/synthetic_cases_v1.json"
click node_synthetic_runner "https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/src/run_evaluation.py"
click node_scoring_engine "https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/src/evaluator.py"
click node_day4_results "https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/results/evaluation_results_day4.json"
click node_synthetic_results "https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/results/evaluation_results_v1.json"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scope,node_rubric,node_human_reviewer toneBlue
class node_day4_cases,node_generation_runner,node_medgemma_client,node_raw_outputs toneAmber
class node_scored_cases,node_day4_runner,node_synthetic_cases,node_synthetic_runner,node_scoring_engine,node_day4_results,node_synthetic_results toneMint
class node_remote_endpoint toneIndigo
```

## Components

### Evaluation definition

- [Evaluation scope](https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/scope/evaluation_scope_v1.md): `scope/evaluation_scope_v1.md`
- [Evaluation rubric](https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/rubric/evaluation_rubric_v1.md): `rubric/evaluation_rubric_v1.md`

### Model generation

- [Day 4 cases](https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/cases/day4_cases.json): `cases/day4_cases.json`
- [Generation runner](https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/src/run_medgemma_day4.py): `src/run_medgemma_day4.py`
- [MedGemma client](https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/medgemma_client.py): `medgemma_client.py`
- [Raw model outputs](https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/results/model_outputs_day4.json): `results/model_outputs_day4.json`

### Scoring and results

- [Scored Day 4 cases](https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/cases/day4_scored_cases.json): `cases/day4_scored_cases.json`
- [Day 4 evaluation runner](https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/src/run_day4_evaluation.py): `src/run_day4_evaluation.py`
- [Synthetic cases](https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/cases/synthetic_cases_v1.json): `cases/synthetic_cases_v1.json`
- [Synthetic evaluation runner](https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/src/run_evaluation.py): `src/run_evaluation.py`
- [Scoring engine](https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/src/evaluator.py): `src/evaluator.py`
- [Day 4 results](https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/results/evaluation_results_day4.json): `results/evaluation_results_day4.json`
- [Synthetic results](https://github.com/almolsiaboelm-tech/medical-ai-evaluation-lab/blob/main/results/evaluation_results_v1.json): `results/evaluation_results_v1.json`

### Other components

- **Remote model endpoint**
- **Human reviewer**

## Connections

- Generation runner → Day 4 cases: reads cases (evidence: `src/run_medgemma_day4.py`)
- Generation runner → MedGemma client: requests generation (evidence: `src/run_medgemma_day4.py`)
- MedGemma client → Remote model endpoint: sends prompt (evidence: `medgemma_client.py`)
- Generation runner → Raw model outputs: writes outputs (evidence: `src/run_medgemma_day4.py`)
- Human reviewer → Evaluation rubric: reviews against (evidence: `README.md`)
- Human reviewer → Scored Day 4 cases: records scores (evidence: `README.md`)
- Day 4 evaluation runner → Scored Day 4 cases: reads scores (evidence: `src/run_day4_evaluation.py`)
- Day 4 evaluation runner → Raw model outputs: reads responses (evidence: `src/run_day4_evaluation.py`)
- Day 4 evaluation runner → Scoring engine: evaluates scores (evidence: `src/run_day4_evaluation.py`)
- Day 4 evaluation runner → Day 4 results: writes results (evidence: `src/run_day4_evaluation.py`)
- Synthetic evaluation runner → Synthetic cases: reads cases (evidence: `src/run_evaluation.py`)
- Synthetic evaluation runner → Scoring engine: evaluates scores (evidence: `src/run_evaluation.py`)
- Synthetic evaluation runner → Synthetic results: writes results (evidence: `src/run_evaluation.py`)