How astrozoid2604/llm_leaderboard_crawler works
This repository is a research pipeline for comparing full-dataset and sampled evaluation scores of Hugging Face leaderboard models. Its documented workflow selects top models (or accepts a chosen list), collects benchmark data, computes full and subset scores, ranks the models, and visualizes sampling behavior. The sampled Python source confirms cloning leaderboard result data, loading parquet data, and computing comparison statistics; the README documents additional sampling and ranking capabilities. Some detailed stage-to-stage calls are documented rather than visible in the excerpts.
Main entry points: sampling_pipeline.py Sampling pipeline · npy_file Saved sample data · visualization_results Plot outputs
Components 14
Model discovery
- Leaderboard results
- Top-model selection
sampling_pipeline.py
Evaluation pipeline
- Sampling pipeline
sampling_pipeline.py - Benchmark datasets
- Full and subset scoring
sampling_pipeline.py - Sampling strategies
sampling_pipeline.py - Quality features
sampling_pipeline.py
Comparison and analysis
- Model rankings
sampling_pipeline.py - Similarity measures
sampling_pipeline.py - Sampling visualizations
sampling_pipeline.py
Saved artifacts
- Saved sample data
npy_file - Plot outputs
visualization_results
Other components
- Researcher
- Hugging Face datasets
Connections 13
- Researcher → Sampling pipeline: runs
README.md - Sampling pipeline → Top-model selection: selects models
README.md - Top-model selection → Leaderboard results: clones results
sampling_pipeline.py - Leaderboard results → Hugging Face datasets: fetched from
README.md - Sampling pipeline → Full and subset scoring: tabulates scores
README.md - Full and subset scoring → Benchmark datasets: loads parquet
sampling_pipeline.py - Sampling pipeline → Sampling strategies: chooses strategy
README.md - Sampling strategies → Quality features: uses features
sampling_pipeline.py - Full and subset scoring → Model rankings: ranks scores
README.md - Full and subset scoring → Saved sample data: saves samples
- Sampling pipeline → Similarity measures: compares distributions
sampling_pipeline.py - Sampling pipeline → Sampling visualizations: plots results
README.md - Sampling visualizations → Plot outputs: writes plots