# astrozoid2604/llm_leaderboard_crawler architecture

> Architecture diagram of the GitHub repository astrozoid2604/llm_leaderboard_crawler, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/astrozoid2604/llm_leaderboard_crawler
- Repository: https://github.com/astrozoid2604/llm_leaderboard_crawler

## Overview

This repository is a research pipeline for comparing full-dataset and sampled evaluation scores of Hugging Face leaderboard models. Its documented workflow selects top models (or accepts a chosen list), collects benchmark data, computes full and subset scores, ranks the models, and visualizes sampling behavior. The sampled Python source confirms cloning leaderboard result data, loading parquet data, and computing comparison statistics; the README documents additional sampling and ranking capabilities. Some detailed stage-to-stage calls are documented rather than visible in the excerpts.

Main entry points: [`sampling_pipeline.py`](https://github.com/astrozoid2604/llm_leaderboard_crawler/blob/main/sampling_pipeline.py) (Sampling pipeline), [`npy_file`](https://github.com/astrozoid2604/llm_leaderboard_crawler/tree/main/npy_file) (Saved sample data), [`visualization_results`](https://github.com/astrozoid2604/llm_leaderboard_crawler/tree/main/visualization_results) (Plot outputs)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_discovery["Model discovery"]
  node_leaderboard["Leaderboard results"]
  node_topmodels["Top-model selection"]
end

subgraph group_evaluation["Evaluation pipeline"]
  node_pipeline["Sampling pipeline"]
  node_benchmarkdata["Benchmark datasets"]
  node_scoring["Full and subset scoring"]
  node_sampling["Sampling strategies"]
  node_qualityfeatures["Quality features"]
end

subgraph group_analysis["Comparison and analysis"]
  node_rankings["Model rankings"]
  node_similarity["Similarity measures"]
  node_plots["Sampling visualizations"]
end

subgraph group_artifacts["Saved artifacts"]
  node_sampledata["Saved sample data"]
  node_visualoutputs["Plot outputs"]
end

node_researcher(("Researcher"))
node_huggingface(("Hugging Face datasets"))

node_researcher -->|"runs"| node_pipeline
node_pipeline -->|"selects models"| node_topmodels
node_topmodels -->|"clones results"| node_leaderboard
node_leaderboard -.->|"fetched from"| node_huggingface
node_pipeline -->|"tabulates scores"| node_scoring
node_scoring -->|"loads parquet"| node_benchmarkdata
node_pipeline -->|"chooses strategy"| node_sampling
node_sampling -.->|"uses features"| node_qualityfeatures
node_scoring -->|"ranks scores"| node_rankings
node_scoring -.->|"saves samples"| node_sampledata
node_pipeline -->|"compares distributions"| node_similarity
node_pipeline -->|"plots results"| node_plots
node_plots -.->|"writes plots"| node_visualoutputs

click node_pipeline "https://github.com/astrozoid2604/llm_leaderboard_crawler/blob/main/sampling_pipeline.py"
click node_topmodels "https://github.com/astrozoid2604/llm_leaderboard_crawler/blob/main/sampling_pipeline.py"
click node_scoring "https://github.com/astrozoid2604/llm_leaderboard_crawler/blob/main/sampling_pipeline.py"
click node_sampling "https://github.com/astrozoid2604/llm_leaderboard_crawler/blob/main/sampling_pipeline.py"
click node_qualityfeatures "https://github.com/astrozoid2604/llm_leaderboard_crawler/blob/main/sampling_pipeline.py"
click node_rankings "https://github.com/astrozoid2604/llm_leaderboard_crawler/blob/main/sampling_pipeline.py"
click node_similarity "https://github.com/astrozoid2604/llm_leaderboard_crawler/blob/main/sampling_pipeline.py"
click node_plots "https://github.com/astrozoid2604/llm_leaderboard_crawler/blob/main/sampling_pipeline.py"
click node_sampledata "https://github.com/astrozoid2604/llm_leaderboard_crawler/tree/main/npy_file"
click node_visualoutputs "https://github.com/astrozoid2604/llm_leaderboard_crawler/tree/main/visualization_results"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_leaderboard,node_topmodels toneBlue
class node_pipeline,node_benchmarkdata,node_scoring,node_sampling,node_qualityfeatures toneAmber
class node_rankings,node_similarity,node_plots toneMint
class node_sampledata,node_visualoutputs toneRose
class node_researcher,node_huggingface toneIndigo
```

## Components

### Model discovery

- **Leaderboard results**
- [Top-model selection](https://github.com/astrozoid2604/llm_leaderboard_crawler/blob/main/sampling_pipeline.py): `sampling_pipeline.py`

### Evaluation pipeline

- [Sampling pipeline](https://github.com/astrozoid2604/llm_leaderboard_crawler/blob/main/sampling_pipeline.py): `sampling_pipeline.py`
- **Benchmark datasets**
- [Full and subset scoring](https://github.com/astrozoid2604/llm_leaderboard_crawler/blob/main/sampling_pipeline.py): `sampling_pipeline.py`
- [Sampling strategies](https://github.com/astrozoid2604/llm_leaderboard_crawler/blob/main/sampling_pipeline.py): `sampling_pipeline.py`
- [Quality features](https://github.com/astrozoid2604/llm_leaderboard_crawler/blob/main/sampling_pipeline.py): `sampling_pipeline.py`

### Comparison and analysis

- [Model rankings](https://github.com/astrozoid2604/llm_leaderboard_crawler/blob/main/sampling_pipeline.py): `sampling_pipeline.py`
- [Similarity measures](https://github.com/astrozoid2604/llm_leaderboard_crawler/blob/main/sampling_pipeline.py): `sampling_pipeline.py`
- [Sampling visualizations](https://github.com/astrozoid2604/llm_leaderboard_crawler/blob/main/sampling_pipeline.py): `sampling_pipeline.py`

### Saved artifacts

- [Saved sample data](https://github.com/astrozoid2604/llm_leaderboard_crawler/tree/main/npy_file): `npy_file`
- [Plot outputs](https://github.com/astrozoid2604/llm_leaderboard_crawler/tree/main/visualization_results): `visualization_results`

### Other components

- **Researcher**
- **Hugging Face datasets**

## Connections

- Researcher → Sampling pipeline: runs (evidence: `README.md`)
- Sampling pipeline → Top-model selection: selects models (evidence: `README.md`)
- Top-model selection → Leaderboard results: clones results (evidence: `sampling_pipeline.py`)
- Leaderboard results → Hugging Face datasets: fetched from (evidence: `README.md`)
- Sampling pipeline → Full and subset scoring: tabulates scores (evidence: `README.md`)
- Full and subset scoring → Benchmark datasets: loads parquet (evidence: `sampling_pipeline.py`)
- Sampling pipeline → Sampling strategies: chooses strategy (evidence: `README.md`)
- Sampling strategies → Quality features: uses features (evidence: `sampling_pipeline.py`)
- Full and subset scoring → Model rankings: ranks scores (evidence: `README.md`)
- Full and subset scoring → Saved sample data: saves samples
- Sampling pipeline → Similarity measures: compares distributions (evidence: `sampling_pipeline.py`)
- Sampling pipeline → Sampling visualizations: plots results (evidence: `README.md`)
- Sampling visualizations → Plot outputs: writes plots
