How factslab/bead works
bead is a Python framework for building and running large linguistic judgment experiments. Its principal workflow composes configuration and annotation protocols, creates stimuli from lexical resources and templates, constructs items, partitions balanced lists, deploys jsPsych/JATOS studies, and uses collected judgments for evaluation and active-learning training. Separate branches support corpus-to-item processing, simulated annotation, and behavioral analysis. The README documents these capabilities; several subsystem connections below are based on that documented workflow because their implementation was not sampled.
Main entry points: loop.py Active-learning loop · items Experimental items · pipeline.py Corpus processing · lists Balanced lists
Components 23
Experiment design
- Config composition
bead/config/loader.py - Annotation protocols
bead/protocol
Stimulus construction
- Lexical resources
bead/resources - Template realization
bead/templates - Corpus processing
bead/corpus/pipeline.py - Dependency parsing
bead/tokenization/parsers.py - Experimental items
bead/items - Balanced lists
bead/lists
Study delivery
- jsPsych deployment
bead/deployment/jspsych - JATOS integration
bead/deployment/jatos
Judgments and analysis
- Response collection
bead/data_collection - Judgment evaluation
bead/evaluation - Behavioral analysis
bead/behavioral
Model learning
- Active-learning loop
bead/active_learning/loop.py - Uncertainty selection
bead/active_learning/selection.py - Task models
bead/active_learning/models - Model trainers
bead/active_learning/trainers - Judgment simulation
bead/simulation
Shared runtime
- Constraint DSL
bead/dsl - JSONL repositories
bead/data/repository.py
Other components
- Researcher
- Study annotator
- Model APIs
Connections 30
- Researcher → Config composition: configures
- Researcher → Annotation protocols: defines questions
- Config composition → Annotation protocols: configures
- Lexical resources → Template realization: supplies lexicon
- Template realization → Experimental items: realizes stimuli
- Annotation protocols → Experimental items: defines templates
- Corpus processing → Dependency parsing: parses records
- Dependency parsing → Corpus processing: provides parses
- Corpus processing → Experimental items: constructs items
- Corpus processing → Constraint DSL: filters structure
- Balanced lists → Constraint DSL: checks constraints
- Experimental items → Balanced lists: partitions items
- Balanced lists → jsPsych deployment: provides lists
- Annotation protocols → jsPsych deployment: provides task definitions
- jsPsych deployment → JATOS integration: exports study
- Study annotator → JATOS integration: completes study
- JATOS integration → Response collection: returns responses
- Response collection → Judgment evaluation: supplies judgments
- Response collection → Active-learning loop: provides ratings
- Judgment evaluation → Active-learning loop: provides agreement target
- Active-learning loop → Uncertainty selection: requests items
- Active-learning loop → Task models: uses model
- Active-learning loop → Model trainers: trains model
- Active-learning loop → Judgment evaluation: checks convergence
- Judgment simulation → Active-learning loop: demonstrates loop
- Response collection → Behavioral analysis: supplies behavior data
- Lexical resources → Model APIs: queries providers
- Task models → Model APIs: calls model APIs
- Experimental items → JSONL repositories: persists records
- Balanced lists → JSONL repositories: persists lists