How aicrowd/whest-starterkit works
This is a participant starter kit for the ARC WhiteBox Estimation Challenge, not the scoring harness itself. The main workflow is to implement an estimator, iterate against a local Monte Carlo comparison, then use the documented whestbench CLI stages to validate, score on a dataset, package, and submit. The graph also preserves the bundled estimator examples and the FLOP-accounting dependency. CLI and submission behavior is documented but implemented outside this repository; source excerpts do not show its internals.
Main entry points: estimator.py Estimator · local_engine.py Local comparison engine · examples Estimator examples
Components 11
Local iteration
- Estimator
estimator.py - Estimator examples
examples - Local comparison engine
local_engine.py - FLOP accounting
- Whestbench API types
Validation and scoring
- Whestbench CLI
- Public evaluation dataset
- Evaluation results
Packaging and submission
- Packaged submission
- AIcrowd challenge
Other components
- Participant
Connections 12
- Participant → Estimator: implements
README.md - Participant → Local comparison engine: runs locally
README.md - Estimator → Local comparison engine: evaluated by
estimator.py - Estimator → Estimator examples: loads baseline
estimator.py - Local comparison engine → FLOP accounting: meters operations
local_engine.py - Local comparison engine → Whestbench API types: uses API types
local_engine.py - Participant → Whestbench CLI: runs CLI stages
README.md - Whestbench CLI → Estimator: validates and runs
README.md - Whestbench CLI → Public evaluation dataset: scores against
README.md - Whestbench CLI → Evaluation results: reports results
README.md - Whestbench CLI → Packaged submission: packages estimator
README.md - Whestbench CLI → AIcrowd challenge: submits package
README.md