How eaminc/nips2027 works
abhack is a Python toolkit for simulating A/B experiments where sticky versus freshly randomized traffic assignments interact with honest versus score-peeking agents. Its main workflow configures a study, generates a stable user population and scores, simulates rounds, and reports AB/AA gains and theory bounds. A separate run workflow writes and checks user, bucket, and score CSVs. The README documents additional score processes and experiment modes; their implementation details are represented at subsystem level where source excerpts are limited.
Main entry points: run.py Run workflow · core.py Study simulation · study.py Study workflow · distributions.py Score distributions
Components 19
Configuration and runs
- Run config
exp/spec.py - Study config
exp/spec.py - Run workflow
exp/run.py - Study workflow
exp/study.py
Population and artifacts
- User IDs
src/abhack/users/core.py - User CSV I/O
src/abhack/users/io.py - Bucket CSV I/O
src/abhack/layer/io.py - Bucket assignment
src/abhack/layer/core.py - Stable hashing
src/abhack/utils/hash.py - Score CSV I/O
src/abhack/metric/io.py - Experiment outputs
Experiment simulation
- Score distributions
src/abhack/utils/distributions.py - Score processes
src/abhack/process/core.py - System and agents
src/abhack/strategies/core.py - Study simulation
src/abhack/study/core.py
Metrics and reporting
- Score checks
src/abhack/metric/core.py - Gain and bounds
src/abhack/evaluate/core.py - Terminal reports
src/abhack/utils/term.py
Other components
- Experimenter
Connections 29
- Experimenter → Run workflow: runs config
README.md - Experimenter → Study workflow: runs study
README.md - Run workflow → Run config: loads config
exp/run.py - Study workflow → Study config: loads config
exp/study.py - Run workflow → User CSV I/O: writes users
exp/run.py - User CSV I/O → User IDs: generates IDs
src/abhack/users/io.py - Run workflow → Bucket CSV I/O: writes and checks
exp/run.py - Bucket CSV I/O → User CSV I/O: reads IDs
src/abhack/layer/io.py - Bucket CSV I/O → Bucket assignment: assigns buckets
src/abhack/layer/io.py - Bucket assignment → Stable hashing: hashes IDs
src/abhack/layer/core.py - Run workflow → Score CSV I/O: writes and checks
exp/run.py - Score CSV I/O → User CSV I/O: reads IDs
src/abhack/metric/io.py - Score CSV I/O → Score distributions: draws scores
src/abhack/metric/io.py - Score CSV I/O → Score checks: checks scores
src/abhack/metric/io.py - Run config → Score distributions: builds distributions
exp/spec.py - Study config → Score distributions: builds distribution
exp/spec.py - Run workflow → Experiment outputs: writes artifacts
exp/run.py - Run workflow → Terminal reports: renders summaries
exp/run.py - Study workflow → Study simulation: runs simulation
exp/study.py - Study workflow → Terminal reports: renders study
exp/study.py - Study simulation → User IDs: creates population
src/abhack/study/core.py - Study simulation → Score distributions: samples scores
src/abhack/study/core.py - Study simulation → Score processes: advances scores
src/abhack/study/core.py - Study simulation → System and agents: assigns systems and arms
src/abhack/study/core.py - Study simulation → Gain and bounds: computes gains and bounds
src/abhack/study/core.py - Study simulation → Experiment outputs: returns round rows
README.md - Score processes → Score distributions: draws score values
src/abhack/process/core.py - System and agents → Stable hashing: hashes buckets
src/abhack/strategies/core.py - Study workflow → Experiment outputs: writes study rows
README.md