# jamal-20/genai-hta-consistency architecture

> Architecture diagram of the GitHub repository jamal-20/genai-hta-consistency, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jamal-20/genai-hta-consistency
- Repository: https://github.com/jamal-20/genai-hta-consistency

## Overview

This proof-of-concept extracts five PICO fields from a NICE appraisal report in five Gemini runs, checks field-level consistency, then simulates review of flagged disagreements and evaluates single-run and reviewed results against a gold standard. The graph follows the documented workflow and sampled source; evaluation’s gold standard is embedded in its script, and the review is simulated rather than an interactive researcher step.

Main entry points: [`src/evaluate.py`](https://github.com/jamal-20/genai-hta-consistency/blob/main/src/evaluate.py) (Evaluation and review), [`src/extract_multiple.py`](https://github.com/jamal-20/genai-hta-consistency/blob/main/src/extract_multiple.py) (Repeated extraction), [`src/utils.py`](https://github.com/jamal-20/genai-hta-consistency/blob/main/src/utils.py) (LLM and data helpers), [`src/check_consistency.py`](https://github.com/jamal-20/genai-hta-consistency/blob/main/src/check_consistency.py) (Consistency check)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_inputs["Extraction Inputs"]
  node_report["NICE report text"]
  node_prompt["Extraction prompt"]
end

subgraph group_pipeline["Extraction Pipeline"]
  node_extract["Repeated extraction"]
  node_helpers["LLM and data helpers<br/>[utils.py]"]
  node_runs["Five extraction runs<br/>[multiple_runs.json]"]
end

subgraph group_evaluation["Consistency and Evaluation"]
  node_consistency["Consistency check"]
  node_consistency_results["Consistency table"]
  node_evaluator["Evaluation and review<br/>[evaluate.py]"]
  node_review["Simulated review<br/>[evaluate.py]"]
  node_gold["Embedded gold standard<br/>[evaluate.py]"]
  node_comparison["Comparison table"]
  node_review_results["Review detail"]
  node_error_analysis["Error analysis<br/>[error_analysis.md]"]
  node_hta_reflection["HTA reflection"]
end

node_researcher(("Project user"))
node_gemini["Google Gemini"]

node_researcher -->|"starts extraction"| node_extract
node_extract -->|"reads report"| node_report
node_extract -->|"reads prompt"| node_prompt
node_extract -->|"calls helpers"| node_helpers
node_helpers -->|"sends prompt"| node_gemini
node_helpers -->|"returns response"| node_extract
node_extract -->|"writes runs"| node_runs
node_researcher -->|"starts check"| node_consistency
node_consistency -->|"reads runs"| node_runs
node_consistency -->|"calls matcher"| node_helpers
node_consistency -->|"writes table"| node_consistency_results
node_researcher -->|"starts evaluation"| node_evaluator
node_evaluator -->|"reads runs"| node_runs
node_evaluator -->|"reads flags"| node_consistency_results
node_evaluator -->|"simulates review"| node_review
node_evaluator -->|"compares against"| node_gold
node_evaluator -->|"calls matcher"| node_helpers
node_evaluator -->|"writes comparison"| node_comparison
node_evaluator -->|"writes review detail"| node_review_results

click node_report "https://github.com/jamal-20/genai-hta-consistency/blob/main/data/ta1074_clinical_effectiveness.txt"
click node_prompt "https://github.com/jamal-20/genai-hta-consistency/blob/main/prompts/extraction_prompt.txt"
click node_extract "https://github.com/jamal-20/genai-hta-consistency/blob/main/src/extract_multiple.py"
click node_helpers "https://github.com/jamal-20/genai-hta-consistency/blob/main/src/utils.py"
click node_runs "https://github.com/jamal-20/genai-hta-consistency/blob/main/results/multiple_runs.json"
click node_consistency "https://github.com/jamal-20/genai-hta-consistency/blob/main/src/check_consistency.py"
click node_consistency_results "https://github.com/jamal-20/genai-hta-consistency/blob/main/results/consistency_table.csv"
click node_evaluator "https://github.com/jamal-20/genai-hta-consistency/blob/main/src/evaluate.py"
click node_review "https://github.com/jamal-20/genai-hta-consistency/blob/main/src/evaluate.py"
click node_gold "https://github.com/jamal-20/genai-hta-consistency/blob/main/src/evaluate.py"
click node_comparison "https://github.com/jamal-20/genai-hta-consistency/blob/main/results/comparison_table.csv"
click node_review_results "https://github.com/jamal-20/genai-hta-consistency/blob/main/results/researcher_in_loop.csv"
click node_error_analysis "https://github.com/jamal-20/genai-hta-consistency/blob/main/analysis/error_analysis.md"
click node_hta_reflection "https://github.com/jamal-20/genai-hta-consistency/blob/main/analysis/hta_context_reflection.md"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_report,node_prompt,node_researcher toneBlue
class node_extract,node_helpers,node_runs toneAmber
class node_consistency,node_consistency_results,node_evaluator,node_review,node_gold,node_comparison,node_review_results,node_error_analysis,node_hta_reflection toneMint
class node_gemini toneIndigo
```

## Components

### Extraction Inputs

- [NICE report text](https://github.com/jamal-20/genai-hta-consistency/blob/main/data/ta1074_clinical_effectiveness.txt): `data/ta1074_clinical_effectiveness.txt`
- [Extraction prompt](https://github.com/jamal-20/genai-hta-consistency/blob/main/prompts/extraction_prompt.txt): `prompts/extraction_prompt.txt`

### Extraction Pipeline

- [Repeated extraction](https://github.com/jamal-20/genai-hta-consistency/blob/main/src/extract_multiple.py): `src/extract_multiple.py`
- [LLM and data helpers](https://github.com/jamal-20/genai-hta-consistency/blob/main/src/utils.py): `src/utils.py`
- [Five extraction runs](https://github.com/jamal-20/genai-hta-consistency/blob/main/results/multiple_runs.json): `results/multiple_runs.json`

### Consistency and Evaluation

- [Consistency check](https://github.com/jamal-20/genai-hta-consistency/blob/main/src/check_consistency.py): `src/check_consistency.py`
- [Consistency table](https://github.com/jamal-20/genai-hta-consistency/blob/main/results/consistency_table.csv): `results/consistency_table.csv`
- [Evaluation and review](https://github.com/jamal-20/genai-hta-consistency/blob/main/src/evaluate.py): `src/evaluate.py`
- [Simulated review](https://github.com/jamal-20/genai-hta-consistency/blob/main/src/evaluate.py): `src/evaluate.py`
- [Embedded gold standard](https://github.com/jamal-20/genai-hta-consistency/blob/main/src/evaluate.py): `src/evaluate.py`
- [Comparison table](https://github.com/jamal-20/genai-hta-consistency/blob/main/results/comparison_table.csv): `results/comparison_table.csv`
- [Review detail](https://github.com/jamal-20/genai-hta-consistency/blob/main/results/researcher_in_loop.csv): `results/researcher_in_loop.csv`
- [Error analysis](https://github.com/jamal-20/genai-hta-consistency/blob/main/analysis/error_analysis.md): `analysis/error_analysis.md`
- [HTA reflection](https://github.com/jamal-20/genai-hta-consistency/blob/main/analysis/hta_context_reflection.md): `analysis/hta_context_reflection.md`

### Other components

- **Project user**
- **Google Gemini**

## Connections

- Project user → Repeated extraction: starts extraction
- Repeated extraction → NICE report text: reads report
- Repeated extraction → Extraction prompt: reads prompt
- Repeated extraction → LLM and data helpers: calls helpers
- LLM and data helpers → Google Gemini: sends prompt
- LLM and data helpers → Repeated extraction: returns response
- Repeated extraction → Five extraction runs: writes runs
- Project user → Consistency check: starts check
- Consistency check → Five extraction runs: reads runs
- Consistency check → LLM and data helpers: calls matcher
- Consistency check → Consistency table: writes table
- Project user → Evaluation and review: starts evaluation
- Evaluation and review → Five extraction runs: reads runs
- Evaluation and review → Consistency table: reads flags
- Evaluation and review → Simulated review: simulates review
- Evaluation and review → Embedded gold standard: compares against
- Evaluation and review → LLM and data helpers: calls matcher
- Evaluation and review → Comparison table: writes comparison
- Evaluation and review → Review detail: writes review detail