# mli0118/ontology-llm architecture

> Architecture diagram of the GitHub repository mli0118/ontology-llm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mli0118/ontology-llm
- Repository: https://github.com/mli0118/ontology-llm

## Overview

This repository implements a benchmark for detecting taxonomic hallucinations in a language model, comparing an Algebraic Ontology Projection (AOP) detector with confidence and consistency baselines. The intended workflow is to derive labeled probes from an ontology, obtain model answers and hidden-state representations, evaluate detectors, and report results. The sampled modules establish ontology-to-probe generation, model I/O, AOP training/evaluation, and baseline calls. `benchmark.py` and the result artifact are present in the tree but were not sampled, so their orchestration and data links are not asserted.

Main entry points: [`files/model_io.py`](https://github.com/mli0118/ontology-llm/blob/main/files/model_io.py) (Model I/O), [`files/probes.py`](https://github.com/mli0118/ontology-llm/blob/main/files/probes.py) (Probe generation), [`files/aop.py`](https://github.com/mli0118/ontology-llm/blob/main/files/aop.py) (AOP projection), [`files/baselines.py`](https://github.com/mli0118/ontology-llm/blob/main/files/baselines.py) (Baseline detectors)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Benchmark data"]
  node_ontology["Taxonomic ontology<br/>[ontology.py]"]
  node_probe_generator["Probe generation<br/>[probes.py]"]
  node_probe_set["Labeled probes"]
end

subgraph group_model["Model access"]
  node_model_io["Model I/O<br/>[model_io.py]"]
  node_language_model["Causal language model"]
  node_hidden_states["Concept representations"]
  node_text_judgments["Yes/no judgments"]
end

subgraph group_detectors["Detection methods"]
  node_aop["AOP projection<br/>[aop.py]"]
  node_projection_training["Projection training<br/>[aop.py]"]
  node_code_evaluation["Code evaluation<br/>[aop.py]"]
  node_baselines["Baseline detectors<br/>[baselines.py]"]
  node_confidence_baseline["Confidence baseline<br/>[baselines.py]"]
  node_consistency_baseline["Consistency baseline<br/>[baselines.py]"]
end

subgraph group_evaluation["Benchmark evaluation"]
  node_runner["Benchmark runner<br/>[benchmark.py]"]
  node_report["Hallucination report"]
end

node_researcher(("Researcher"))

node_researcher -->|"initiates"| node_runner
node_probe_generator -->|"queries relations"| node_ontology
node_probe_generator -->|"generates"| node_probe_set
node_model_io -->|"loads and queries"| node_language_model
node_model_io -->|"extracts"| node_hidden_states
node_model_io -->|"computes"| node_text_judgments
node_aop -->|"trains"| node_projection_training
node_projection_training -->|"reads"| node_hidden_states
node_aop -->|"evaluates codes"| node_code_evaluation
node_baselines -->|"defines"| node_confidence_baseline
node_baselines -->|"defines"| node_consistency_baseline
node_confidence_baseline -->|"requests confidence"| node_model_io
node_consistency_baseline -->|"requests judgments"| node_model_io

click node_runner "https://github.com/mli0118/ontology-llm/blob/main/files/benchmark.py"
click node_ontology "https://github.com/mli0118/ontology-llm/blob/main/files/ontology.py"
click node_probe_generator "https://github.com/mli0118/ontology-llm/blob/main/files/probes.py"
click node_model_io "https://github.com/mli0118/ontology-llm/blob/main/files/model_io.py"
click node_aop "https://github.com/mli0118/ontology-llm/blob/main/files/aop.py"
click node_projection_training "https://github.com/mli0118/ontology-llm/blob/main/files/aop.py"
click node_code_evaluation "https://github.com/mli0118/ontology-llm/blob/main/files/aop.py"
click node_baselines "https://github.com/mli0118/ontology-llm/blob/main/files/baselines.py"
click node_confidence_baseline "https://github.com/mli0118/ontology-llm/blob/main/files/baselines.py"
click node_consistency_baseline "https://github.com/mli0118/ontology-llm/blob/main/files/baselines.py"
click node_report "https://github.com/mli0118/ontology-llm/blob/main/files/results/hallucination_report.json"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ontology,node_probe_generator,node_probe_set toneBlue
class node_model_io,node_language_model,node_hidden_states,node_text_judgments toneAmber
class node_aop,node_projection_training,node_code_evaluation,node_baselines,node_confidence_baseline,node_consistency_baseline toneMint
class node_runner,node_report toneRose
class node_researcher toneIndigo
```

## Components

### Benchmark data

- [Taxonomic ontology](https://github.com/mli0118/ontology-llm/blob/main/files/ontology.py): `files/ontology.py`
- [Probe generation](https://github.com/mli0118/ontology-llm/blob/main/files/probes.py): `files/probes.py`
- **Labeled probes**

### Model access

- [Model I/O](https://github.com/mli0118/ontology-llm/blob/main/files/model_io.py): `files/model_io.py`
- **Causal language model**
- **Concept representations**
- **Yes/no judgments**

### Detection methods

- [AOP projection](https://github.com/mli0118/ontology-llm/blob/main/files/aop.py): `files/aop.py`
- [Projection training](https://github.com/mli0118/ontology-llm/blob/main/files/aop.py): `files/aop.py`
- [Code evaluation](https://github.com/mli0118/ontology-llm/blob/main/files/aop.py): `files/aop.py`
- [Baseline detectors](https://github.com/mli0118/ontology-llm/blob/main/files/baselines.py): `files/baselines.py`
- [Confidence baseline](https://github.com/mli0118/ontology-llm/blob/main/files/baselines.py): `files/baselines.py`
- [Consistency baseline](https://github.com/mli0118/ontology-llm/blob/main/files/baselines.py): `files/baselines.py`

### Benchmark evaluation

- [Benchmark runner](https://github.com/mli0118/ontology-llm/blob/main/files/benchmark.py): `files/benchmark.py`
- [Hallucination report](https://github.com/mli0118/ontology-llm/blob/main/files/results/hallucination_report.json): `files/results/hallucination_report.json`

### Other components

- **Researcher**

## Connections

- Researcher → Benchmark runner: initiates
- Probe generation → Taxonomic ontology: queries relations
- Probe generation → Labeled probes: generates
- Model I/O → Causal language model: loads and queries
- Model I/O → Concept representations: extracts
- Model I/O → Yes/no judgments: computes
- AOP projection → Projection training: trains
- Projection training → Concept representations: reads
- AOP projection → Code evaluation: evaluates codes
- Baseline detectors → Confidence baseline: defines
- Baseline detectors → Consistency baseline: defines
- Confidence baseline → Model I/O: requests confidence
- Consistency baseline → Model I/O: requests judgments