How github/scientist works
Project Type and Purpose
• This is a Ruby library (a gem) named “Scientist” that instruments your application code to run two behaviors (“control” and “candidate”) side-by-side, compare their results, measure timing, and publish mismatches or metrics.
• It’s not a web app or microservice; it’s a pluggable library that your own Ruby code includes to wrap critical paths during refactoring.
File Structure Highlights
• Top-level:
• .github/, CI config
• Gemfile, scientist.gemspec – gem packaging
• lib/scientist.rb – entry point requiring sub-modules
• lib/scientist/experiment.rb, default.rb, observation.rb, result.rb – core components
• test/ – unit tests for all core behaviors
• script/ – helper scripts for bootstrapping, testing, releasing
• There are no frontend/backend folders or external services built-in; the gem defers publishing to user-provided publish implementations (e.g., StatsD, Redis).
Key Architecture from the README
• Public API:
• include Scientist → adds science helper to your class
• Scientist.run → alternative entry for non-module usage
• Core Modules:
• Scientist::Default – default experiment implementation
• Scientist::Experiment – mix-in that defines enabled?, publish, compare, etc.
• Scientist::Observation – encapsulates one run of control/candidate blocks
• Scientist::Result – aggregates observations, provides metrics (matched, mismatched, durations)
• Extensibility: Users supply their own Experiment classes (override enabled?, publish, raised, etc.).
a. Main Components
• User Application Code (your classes/modules)
• Scientist Public API (science helper, Scientist.run)
• Experiment Factory (default or custom): instantiates Experiment objects
• Scientist::Experiment module (mixin): defines lifecycle hooks
• Control & Candidate Blocks (closures provided by user code)
• Observation & Result objects
• Publishing Targets (StatsD, Redis, custom sink)
• Test Harness (scripts and test suite with raise_on_mismatches)
b. Component Interactions
1. User calls science("name") { |e| e.use{...}; e.try{...} }
2. The API loads the default Experiment implementation (or a custom one)
3. Experiment#run:
• Checks run_if → enabled?
• Randomizes execution order of control & candidate
• Wraps each block in an Observation → records durations, exceptions
• Compares results via compare / compare_errors
• Aggregates Observations into a Result
• Calls publish(result)
4. publish method pushes metrics to external systems (e.g., StatsD, Redis)
5. Test suite toggles raise_on_mismatches to convert mismatches into test failures
c. Architectural Patterns & Principles
• Mixins (Scientist module and Scientist::Experiment module) for clean API extension
• Dependency Injection for custom Experiment classes
• Immutable value objects (Observation, Result) to model run data
• Randomized sequencing to avoid bias in timing measurements
• Callback hooks (before_run, compare, compare_errors, ignore, clean)
d. Relevant Technologies
• Ruby 2.3+
• Bundler / Gemfile / gems
• Rake & script tasks for CI, testing, releasing