How stan-dev/stan works
Project Type & Purpose
• Stan is a C++ probabilistic programming library that provides full Bayesian inference (via the No-U-Turn Sampler, a variant of Hamiltonian Monte Carlo), approximate Bayesian inference using automatic differentiation variational inference (ADVI), and penalized maximum likelihood optimization (via L-BFGS).
• Its design is centered on high-performance statistical computation and automatic differentiation, with a clear separation between the modeling language, mathematical back-end (Stan Math), and various inference engines.
Main Architectural Components
a. Core Libraries & External Dependencies:
– Stan Math Library (providing autodiff, linear algebra, probability functions)
– RapidJSON (for JSON parsing and serialization in the IO components)
– External libraries like Intel TBB (for parallelism; note the licensing implications)
b. Subsystems within the Stan Code Base:
– Model & Language Parsing: The files in src/stan/lang indicate a domain-specific language parser that processes Stan models.
– Inference & Sampling Engines: Multiple directories under src/stan/mcmc (including hmc, nuts, static, etc.) represent the implementation of various sampling methods like HMC/NUTS.
– Variational Inference: The src/stan/variational directory houses the ADVI implementation.
– Optimization: The src/stan/optimization directory provides methods (e.g., L-BFGS, Newton) for penalized MLE.
– IO & Data Handling: Components in src/stan/io for reading, writing, and serializing data (including CSV and JSON support) are key to interfacing with models and external tools.
– Services Layer: Located in src/stan/services, these modules wrap specific tasks (diagnose, optimize, sample, pathfinder) that use the underlying model/sampler implementations.
– Callbacks & Logging: Support for handling output and logging is implemented in src/stan/callbacks.
c. Testing & Documentation:
– Extensive unit tests, integration tests, and Doxygen-based documentation (under src/doxygen and src/test) ensure robustness.
– The make and Jenkinsfile build configurations point to an organized CI/CD and documentation generation process.
Relationships and Data Flow
• The input (model definition in the Stan DSL) is parsed in the language module (src/stan/lang), which then constructs an internal model representation.
• The model is passed on to the inference subsystems, where different engines handle sampling (MCMC), variational inference (ADVI), or optimization.
• Underneath, the Stan Math library provides the gradient and autodiff capabilities needed by these engines.
• Data IO components manage input/output transfers, supporting JSON, CSV, and other formats.
• Callbacks and logging are integrated to capture the execution diagnostics and trace outputs through the pipeline.
• Finally, testing and documentation (including Doxygen) form supporting layers around the core components.