# ibtraore/mie-confidence-estimation architecture

> Architecture diagram of the GitHub repository ibtraore/mie-confidence-estimation, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ibtraore/mie-confidence-estimation
- Repository: https://github.com/ibtraore/mie-confidence-estimation

## Overview

This repository is a research study codebase for clinical information extraction and LLM confidence evaluation. Its user workflow is to provide task-specific annotated CSV data and model configuration, run one of four task scripts, obtain structured model predictions with confidence estimates, and evaluate correctness and uncertainty. The task scripts and shared confidence utilities are excerpted; the broader evaluation metrics are documented in the README, but their full implementation is not shown. The README also notes that the clinical dataset and processing code are not included.

Main entry points: [`src/utils.py`](https://github.com/ibtraore/mie-confidence-estimation/blob/main/src/utils.py) (Confidence utilities), [`src/clinical_sense_disambiguation.py`](https://github.com/ibtraore/mie-confidence-estimation/blob/main/src/clinical_sense_disambiguation.py) (Acronym disambiguation), [`src/coreference_resolution.py`](https://github.com/ibtraore/mie-confidence-estimation/blob/main/src/coreference_resolution.py) (Coreference resolution), [`src/medication_attribute_extraction.py`](https://github.com/ibtraore/mie-confidence-estimation/blob/main/src/medication_attribute_extraction.py) (Medication attributes)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_clinical_tasks["Clinical extraction tasks"]
  node_casi["Acronym disambiguation"]
  node_coreference["Coreference resolution"]
  node_med_attribute["Medication attributes"]
  node_med_status["Medication status"]
end

subgraph group_shared_runtime["Shared runtime"]
  node_configuration["Model configuration<br/>[config.py]"]
  node_confidence_utils["Confidence utilities<br/>[utils.py]"]
end

subgraph group_evaluation["Confidence evaluation"]
  node_correctness["Task correctness"]
  node_confidence_metrics["Confidence metrics<br/>[README.md]"]
end

node_researcher(("Researcher"))
node_clinical_input["Clinical task data"]
node_llm{{"Configured LLM"}}
node_results["Prediction and evaluation files"]

node_researcher -->|"provides data"| node_clinical_input
node_clinical_input -->|"supplies examples"| node_casi
node_clinical_input -->|"supplies examples"| node_coreference
node_clinical_input -->|"supplies examples"| node_med_attribute
node_clinical_input -->|"supplies examples"| node_med_status
node_researcher -->|"sets model options"| node_configuration
node_casi -->|"reads settings"| node_configuration
node_coreference -->|"reads settings"| node_configuration
node_med_attribute -->|"reads settings"| node_configuration
node_med_status -->|"reads settings"| node_configuration
node_casi -->|"invokes model"| node_llm
node_coreference -->|"invokes model"| node_llm
node_med_attribute -->|"invokes model"| node_llm
node_med_status -->|"invokes model"| node_llm
node_casi -->|"scores token uncertainty"| node_confidence_utils
node_coreference -->|"scores token uncertainty"| node_confidence_utils
node_med_attribute -->|"scores token uncertainty"| node_confidence_utils
node_med_status -->|"scores token uncertainty"| node_confidence_utils
node_casi -->|"computes correctness"| node_correctness
node_coreference -->|"checks answer F1"| node_confidence_utils
node_med_attribute -->|"scores answer matches"| node_confidence_utils
node_med_status -->|"checks answer F1"| node_confidence_utils
node_confidence_utils -->|"supports confidence scores"| node_confidence_metrics
node_correctness -.->|"evaluates calibration"| node_confidence_metrics
node_casi -->|"writes predictions"| node_results
node_coreference -->|"writes results"| node_results
node_med_attribute -->|"writes results"| node_results
node_med_status -->|"writes results"| node_results

click node_casi "https://github.com/ibtraore/mie-confidence-estimation/blob/main/src/clinical_sense_disambiguation.py"
click node_coreference "https://github.com/ibtraore/mie-confidence-estimation/blob/main/src/coreference_resolution.py"
click node_med_attribute "https://github.com/ibtraore/mie-confidence-estimation/blob/main/src/medication_attribute_extraction.py"
click node_med_status "https://github.com/ibtraore/mie-confidence-estimation/blob/main/src/medication_status_extraction.py"
click node_configuration "https://github.com/ibtraore/mie-confidence-estimation/blob/main/src/config.py"
click node_confidence_utils "https://github.com/ibtraore/mie-confidence-estimation/blob/main/src/utils.py"
click node_correctness "https://github.com/ibtraore/mie-confidence-estimation/blob/main/src/clinical_sense_disambiguation.py"
click node_confidence_metrics "https://github.com/ibtraore/mie-confidence-estimation/blob/main/README.md"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_casi,node_coreference,node_med_attribute,node_med_status toneBlue
class node_configuration,node_confidence_utils toneAmber
class node_correctness,node_confidence_metrics toneMint
class node_researcher,node_llm,node_results toneIndigo
class node_clinical_input toneRose
```

## Components

### Clinical extraction tasks

- [Acronym disambiguation](https://github.com/ibtraore/mie-confidence-estimation/blob/main/src/clinical_sense_disambiguation.py): `src/clinical_sense_disambiguation.py`
- [Coreference resolution](https://github.com/ibtraore/mie-confidence-estimation/blob/main/src/coreference_resolution.py): `src/coreference_resolution.py`
- [Medication attributes](https://github.com/ibtraore/mie-confidence-estimation/blob/main/src/medication_attribute_extraction.py): `src/medication_attribute_extraction.py`
- [Medication status](https://github.com/ibtraore/mie-confidence-estimation/blob/main/src/medication_status_extraction.py): `src/medication_status_extraction.py`

### Shared runtime

- [Model configuration](https://github.com/ibtraore/mie-confidence-estimation/blob/main/src/config.py): `src/config.py`
- [Confidence utilities](https://github.com/ibtraore/mie-confidence-estimation/blob/main/src/utils.py): `src/utils.py`

### Confidence evaluation

- [Task correctness](https://github.com/ibtraore/mie-confidence-estimation/blob/main/src/clinical_sense_disambiguation.py): `src/clinical_sense_disambiguation.py`
- [Confidence metrics](https://github.com/ibtraore/mie-confidence-estimation/blob/main/README.md): `README.md`

### Other components

- **Researcher**
- **Clinical task data**
- **Configured LLM**
- **Prediction and evaluation files**

## Connections

- Researcher → Clinical task data: provides data (evidence: `README.md`)
- Clinical task data → Acronym disambiguation: supplies examples (evidence: `README.md`)
- Clinical task data → Coreference resolution: supplies examples (evidence: `README.md`)
- Clinical task data → Medication attributes: supplies examples (evidence: `README.md`)
- Clinical task data → Medication status: supplies examples (evidence: `README.md`)
- Researcher → Model configuration: sets model options (evidence: `README.md`)
- Acronym disambiguation → Model configuration: reads settings (evidence: `src/clinical_sense_disambiguation.py`)
- Coreference resolution → Model configuration: reads settings (evidence: `src/coreference_resolution.py`)
- Medication attributes → Model configuration: reads settings (evidence: `src/medication_attribute_extraction.py`)
- Medication status → Model configuration: reads settings (evidence: `src/medication_status_extraction.py`)
- Acronym disambiguation → Configured LLM: invokes model (evidence: `src/clinical_sense_disambiguation.py`)
- Coreference resolution → Configured LLM: invokes model (evidence: `src/coreference_resolution.py`)
- Medication attributes → Configured LLM: invokes model (evidence: `src/medication_attribute_extraction.py`)
- Medication status → Configured LLM: invokes model (evidence: `src/medication_status_extraction.py`)
- Acronym disambiguation → Confidence utilities: scores token uncertainty (evidence: `src/clinical_sense_disambiguation.py`)
- Coreference resolution → Confidence utilities: scores token uncertainty (evidence: `src/coreference_resolution.py`)
- Medication attributes → Confidence utilities: scores token uncertainty (evidence: `src/medication_attribute_extraction.py`)
- Medication status → Confidence utilities: scores token uncertainty (evidence: `src/medication_status_extraction.py`)
- Acronym disambiguation → Task correctness: computes correctness (evidence: `src/clinical_sense_disambiguation.py`)
- Coreference resolution → Confidence utilities: checks answer F1 (evidence: `src/coreference_resolution.py`)
- Medication attributes → Confidence utilities: scores answer matches (evidence: `src/medication_attribute_extraction.py`)
- Medication status → Confidence utilities: checks answer F1 (evidence: `src/medication_status_extraction.py`)
- Confidence utilities → Confidence metrics: supports confidence scores (evidence: `README.md`)
- Task correctness → Confidence metrics: evaluates calibration
- Acronym disambiguation → Prediction and evaluation files: writes predictions (evidence: `src/clinical_sense_disambiguation.py`)
- Coreference resolution → Prediction and evaluation files: writes results (evidence: `src/coreference_resolution.py`)
- Medication attributes → Prediction and evaluation files: writes results (evidence: `README.md`)
- Medication status → Prediction and evaluation files: writes results (evidence: `README.md`)
