# ivicanikolicsg/model_provenance_testing architecture

> Architecture diagram of the GitHub repository ivicanikolicsg/model_provenance_testing, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ivicanikolicsg/model_provenance_testing
- Repository: https://github.com/ivicanikolicsg/model_provenance_testing

## Overview

This repository implements statistical model-provenance testing: it compares a candidate language model’s prompt responses with responses from possible parent models and reports a parent guess, optionally evaluating it against supplied ground truth. The main workflow is to load parent and candidate lists, reuse or generate parent prompt-output caches, query each candidate on those prompts, and statistically compare response-match rates. The graph includes the documented benchmark inputs and sampling options; relationships involving unsampled implementation details are omitted.

Main entry points: [`tester.py`](https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/tester.py) (Test runner), [`cache_parent_prompts.py`](https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/cache_parent_prompts.py) (Parent cache builder), [`llm_functions.py`](https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/llm_functions.py) (LLM inference), [`sampler_classes.py`](https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/sampler_classes.py) (Prompt samplers)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflow["Test workflow"]
  node_cli["Test runner<br/>[tester.py]"]
  node_model_lists["Model lists"]
end

subgraph group_prompts["Prompt generation"]
  node_sampler["Prompt samplers<br/>[sampler_classes.py]"]
  node_sentence_data["Random sentences"]
  node_cache_builder["Parent cache builder"]
  node_cached_outputs[("Prompt output cache")]
end

subgraph group_inference["Model inference"]
  node_inference["LLM inference<br/>[llm_functions.py]"]
  node_hub["Hugging Face Hub"]
end

subgraph group_results["Comparison and results"]
  node_stats["Statistical tests<br/>[misc.py]"]
  node_decision["Parent decision<br/>[tester.py]"]
  node_evaluation["Ground-truth evaluation<br/>[tester.py]"]
  node_report["Test report<br/>[tester.py]"]
end

node_operator(("Researcher"))

node_operator -->|"runs"| node_cli
node_model_lists -->|"provides models"| node_cli
node_cli -->|"selects sampler"| node_sampler
node_sentence_data -->|"provides prompts"| node_sampler
node_cli -->|"loads cache"| node_cache_builder
node_cache_builder -->|"samples prompts"| node_sampler
node_cache_builder -->|"queries parents"| node_inference
node_cache_builder -->|"writes outputs"| node_cached_outputs
node_cached_outputs -->|"supplies prompts"| node_cli
node_cli -->|"queries candidates"| node_inference
node_inference -->|"loads models"| node_hub
node_cli -->|"compares rates"| node_stats
node_cli -->|"selects parent"| node_decision
node_cli -->|"scores guesses"| node_evaluation
node_cli -->|"prints results"| node_report

click node_cli "https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/tester.py"
click node_model_lists "https://github.com/ivicanikolicsg/model_provenance_testing/tree/main/data"
click node_sampler "https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/sampler_classes.py"
click node_sentence_data "https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/data/random_sentences.txt"
click node_cache_builder "https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/cache_parent_prompts.py"
click node_cached_outputs "https://github.com/ivicanikolicsg/model_provenance_testing/tree/main/cached_prompts"
click node_inference "https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/llm_functions.py"
click node_stats "https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/misc.py"
click node_decision "https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/tester.py"
click node_evaluation "https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/tester.py"
click node_report "https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/tester.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_model_lists toneBlue
class node_sampler,node_sentence_data,node_cache_builder,node_cached_outputs toneAmber
class node_inference,node_hub toneMint
class node_stats,node_decision,node_evaluation,node_report toneRose
class node_operator toneIndigo
```

## Components

### Test workflow

- [Test runner](https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/tester.py): `tester.py`
- [Model lists](https://github.com/ivicanikolicsg/model_provenance_testing/tree/main/data): `data`

### Prompt generation

- [Prompt samplers](https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/sampler_classes.py): `sampler_classes.py`
- [Random sentences](https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/data/random_sentences.txt): `data/random_sentences.txt`
- [Parent cache builder](https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/cache_parent_prompts.py): `cache_parent_prompts.py`
- [Prompt output cache](https://github.com/ivicanikolicsg/model_provenance_testing/tree/main/cached_prompts): `cached_prompts`

### Model inference

- [LLM inference](https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/llm_functions.py): `llm_functions.py`
- **Hugging Face Hub**

### Comparison and results

- [Statistical tests](https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/misc.py): `misc.py`
- [Parent decision](https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/tester.py): `tester.py`
- [Ground-truth evaluation](https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/tester.py): `tester.py`
- [Test report](https://github.com/ivicanikolicsg/model_provenance_testing/blob/main/tester.py): `tester.py`

### Other components

- **Researcher**

## Connections

- Researcher → Test runner: runs (evidence: `README.md`)
- Model lists → Test runner: provides models (evidence: `README.md`)
- Test runner → Prompt samplers: selects sampler (evidence: `tester.py`)
- Random sentences → Prompt samplers: provides prompts (evidence: `README.md`)
- Test runner → Parent cache builder: loads cache (evidence: `tester.py`)
- Parent cache builder → Prompt samplers: samples prompts (evidence: `cache_parent_prompts.py`)
- Parent cache builder → LLM inference: queries parents (evidence: `cache_parent_prompts.py`)
- Parent cache builder → Prompt output cache: writes outputs (evidence: `cache_parent_prompts.py`)
- Prompt output cache → Test runner: supplies prompts
- Test runner → LLM inference: queries candidates (evidence: `tester.py`)
- LLM inference → Hugging Face Hub: loads models (evidence: `llm_functions.py`)
- Test runner → Statistical tests: compares rates (evidence: `tester.py`)
- Test runner → Parent decision: selects parent (evidence: `tester.py`)
- Test runner → Ground-truth evaluation: scores guesses (evidence: `tester.py`)
- Test runner → Test report: prints results (evidence: `tester.py`)
