# jeff-2000/kreyolbench architecture

> Architecture diagram of the GitHub repository jeff-2000/kreyolbench, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jeff-2000/kreyolbench
- Repository: https://github.com/jeff-2000/kreyolbench

## Overview

KreyolBench is a Haitian Creole NLP benchmark ecosystem, not just a scoring library. Its main implemented workflow is CLI-driven: load JSONL predictions and references, extract task-specific values, compute metrics, and save reproducible result records. Separate capabilities cover dataset schemas and provenance, source planning and governance audits, preprocessing, baselines, annotation, and Hugging Face dataset compatibility. The README documents several ecosystem branches whose runtime wiring was not sampled; those are shown without asserted call paths. Source excerpts establish the CLI, evaluation, and governance links.

Main entry points: [`src/kreyolbench/cli.py`](https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/cli.py) (CLI commands), [`src/kreyolbench/governance.py`](https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/governance.py) (Governance audit), [`src/kreyolbench/evaluation/runner.py`](https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/evaluation/runner.py) (Evaluation runner), [`src/kreyolbench/io.py`](https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/io.py) (JSONL and hashes)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["User entry"]
  node_cli["CLI commands<br/>[cli.py]"]
end

subgraph group_data["Data and governance"]
  node_jsonl["JSONL and hashes<br/>[io.py]"]
  node_schemas["Dataset schemas<br/>[schemas.py]"]
  node_config_registry["Config registry<br/>[registry.py]"]
  node_source_plans["Source plans<br/>[sources.py]"]
  node_governance["Governance audit<br/>[governance.py]"]
  node_preprocessing["Text preprocessing"]
end

subgraph group_research["Benchmark research"]
  node_dataset_loader["Dataset loader"]
  node_task_configs["Task and source configs"]
  node_annotation["Annotation resources"]
  node_baselines["Model baselines"]
end

subgraph group_evaluation["Evaluation"]
  node_evaluation_runner["Evaluation runner<br/>[runner.py]"]
  node_metrics["Task metrics<br/>[metrics.py]"]
  node_reporting["Result reporting<br/>[reporting.py]"]
  node_result_files[("Evaluation results")]
end

subgraph group_ecosystem["Results ecosystem"]
  node_leaderboard["Leaderboard"]
  node_hf["Hugging Face"]
end

node_researcher(("Researcher"))

node_researcher -->|"runs commands"| node_cli
node_cli -->|"runs evaluation"| node_evaluation_runner
node_cli -->|"reads rows"| node_jsonl
node_cli -->|"validates rows"| node_schemas
node_cli -->|"loads YAML"| node_config_registry
node_cli -->|"builds plans"| node_source_plans
node_cli -->|"audits repository"| node_governance
node_evaluation_runner -->|"reads data"| node_jsonl
node_evaluation_runner -->|"computes scores"| node_metrics
node_evaluation_runner -->|"builds results"| node_reporting
node_reporting -->|"writes JSON"| node_result_files
node_cli -->|"exports leaderboard"| node_reporting
node_source_plans -->|"validates records"| node_governance
node_governance -->|"uses helpers"| node_jsonl
node_governance -->|"uses registry"| node_config_registry
node_governance -->|"uses records"| node_schemas
node_preprocessing -->|"hashes text"| node_jsonl
node_dataset_loader -.->|"supports compatibility"| node_hf
node_task_configs -.->|"defines task data"| node_schemas
node_annotation -.->|"annotates examples"| node_dataset_loader
node_baselines -.->|"provides model outputs"| node_evaluation_runner
node_leaderboard -.->|"records submissions"| node_result_files

click node_cli "https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/cli.py"
click node_jsonl "https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/io.py"
click node_schemas "https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/schemas.py"
click node_config_registry "https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/registry.py"
click node_source_plans "https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/sources.py"
click node_governance "https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/governance.py"
click node_preprocessing "https://github.com/jeff-2000/kreyolbench/tree/main/src/kreyolbench/preprocessing"
click node_dataset_loader "https://github.com/jeff-2000/kreyolbench/blob/main/datasets/kreyolbench_dataset.py"
click node_task_configs "https://github.com/jeff-2000/kreyolbench/tree/main/configs"
click node_annotation "https://github.com/jeff-2000/kreyolbench/tree/main/annotation"
click node_baselines "https://github.com/jeff-2000/kreyolbench/tree/main/src/kreyolbench/baselines"
click node_evaluation_runner "https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/evaluation/runner.py"
click node_metrics "https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/evaluation/metrics.py"
click node_reporting "https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/evaluation/reporting.py"
click node_result_files "https://github.com/jeff-2000/kreyolbench/tree/main/eval/results"
click node_leaderboard "https://github.com/jeff-2000/kreyolbench/tree/main/leaderboard"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli toneBlue
class node_jsonl,node_schemas,node_config_registry,node_source_plans,node_governance,node_preprocessing toneAmber
class node_dataset_loader,node_task_configs,node_annotation,node_baselines toneMint
class node_evaluation_runner,node_metrics,node_reporting,node_result_files toneRose
class node_leaderboard,node_hf,node_researcher toneIndigo
```

## Components

### User entry

- [CLI commands](https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/cli.py): `src/kreyolbench/cli.py`

### Data and governance

- [JSONL and hashes](https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/io.py): `src/kreyolbench/io.py`
- [Dataset schemas](https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/schemas.py): `src/kreyolbench/schemas.py`
- [Config registry](https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/registry.py): `src/kreyolbench/registry.py`
- [Source plans](https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/sources.py): `src/kreyolbench/sources.py`
- [Governance audit](https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/governance.py): `src/kreyolbench/governance.py`
- [Text preprocessing](https://github.com/jeff-2000/kreyolbench/tree/main/src/kreyolbench/preprocessing): `src/kreyolbench/preprocessing`

### Benchmark research

- [Dataset loader](https://github.com/jeff-2000/kreyolbench/blob/main/datasets/kreyolbench_dataset.py): `datasets/kreyolbench_dataset.py`
- [Task and source configs](https://github.com/jeff-2000/kreyolbench/tree/main/configs): `configs`
- [Annotation resources](https://github.com/jeff-2000/kreyolbench/tree/main/annotation): `annotation`
- [Model baselines](https://github.com/jeff-2000/kreyolbench/tree/main/src/kreyolbench/baselines): `src/kreyolbench/baselines`

### Evaluation

- [Evaluation runner](https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/evaluation/runner.py): `src/kreyolbench/evaluation/runner.py`
- [Task metrics](https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/evaluation/metrics.py): `src/kreyolbench/evaluation/metrics.py`
- [Result reporting](https://github.com/jeff-2000/kreyolbench/blob/main/src/kreyolbench/evaluation/reporting.py): `src/kreyolbench/evaluation/reporting.py`
- [Evaluation results](https://github.com/jeff-2000/kreyolbench/tree/main/eval/results): `eval/results`

### Results ecosystem

- [Leaderboard](https://github.com/jeff-2000/kreyolbench/tree/main/leaderboard): `leaderboard`
- **Hugging Face**

### Other components

- **Researcher**

## Connections

- Researcher → CLI commands: runs commands (evidence: `README.md`)
- CLI commands → Evaluation runner: runs evaluation (evidence: `src/kreyolbench/cli.py`)
- CLI commands → JSONL and hashes: reads rows (evidence: `src/kreyolbench/cli.py`)
- CLI commands → Dataset schemas: validates rows (evidence: `src/kreyolbench/cli.py`)
- CLI commands → Config registry: loads YAML (evidence: `src/kreyolbench/cli.py`)
- CLI commands → Source plans: builds plans (evidence: `src/kreyolbench/cli.py`)
- CLI commands → Governance audit: audits repository (evidence: `src/kreyolbench/cli.py`)
- Evaluation runner → JSONL and hashes: reads data (evidence: `src/kreyolbench/evaluation/runner.py`)
- Evaluation runner → Task metrics: computes scores (evidence: `src/kreyolbench/evaluation/runner.py`)
- Evaluation runner → Result reporting: builds results (evidence: `src/kreyolbench/evaluation/runner.py`)
- Result reporting → Evaluation results: writes JSON (evidence: `src/kreyolbench/evaluation/reporting.py`)
- CLI commands → Result reporting: exports leaderboard (evidence: `src/kreyolbench/cli.py`)
- Source plans → Governance audit: validates records (evidence: `src/kreyolbench/sources.py`)
- Governance audit → JSONL and hashes: uses helpers (evidence: `src/kreyolbench/governance.py`)
- Governance audit → Config registry: uses registry (evidence: `src/kreyolbench/governance.py`)
- Governance audit → Dataset schemas: uses records (evidence: `src/kreyolbench/governance.py`)
- Text preprocessing → JSONL and hashes: hashes text (evidence: `src/kreyolbench/preprocessing/dedupe.py`)
- Dataset loader → Hugging Face: supports compatibility (evidence: `README.md`)
- Task and source configs → Dataset schemas: defines task data
- Annotation resources → Dataset loader: annotates examples
- Model baselines → Evaluation runner: provides model outputs
- Leaderboard → Evaluation results: records submissions