How infer-actively/pymdp works
Project type and purpose
• This is an open-source Python library (package) named “pymdp” for simulating Active Inference agents in discrete Markov Decision Process (MDP) environments.
• It provides core algorithms (inference, control, learning), environment models (grid worlds, T-maze, visual foraging), utility/math functions, and both NumPy and JAX implementations. It ships documentation, examples, notebooks, tests, and CI workflows.
File structure
• Top-level package:
– pymdp/ contains the library’s core modules:
• agent.py – high-level Agent API
• algos/ – inference/control algorithms (FPI, MMP, old variants)
• control.py, inference.py, learning.py – generic pipelines
• default_models.py – built-in A/B matrices, C vectors
• envs/ – environment definitions (gridworlds, T-maze, visual foraging)
• utils.py, maths.py – basic array utilities, math operations
– pymdp/jax/ mirrors the same structure for JAX-based acceleration.
• Documentation: docs/ (Sphinx/ReadTheDocs), tutorial notebooks.
• Examples: examples/ – Jupyter notebooks and scripts showing usage.
• Tests: test/ – unit/integration tests, MATLAB crossvalidation.
• CI/config: GitHub Actions workflow, .readthedocs.yml, setup.py, requirements.txt, LICENSE.
README insights
• Stack: Python, NumPy, optional JAX.
• Key components: Agent class, utility to define A/B/C matrices, policy inference, action sampling, environment stepping.
• Emphasis on algorithm validation against MATLAB SPM functions.
Main components and their relationships
a. User / Client
• Interacts via Python import (import pymdp) or CLI (pip installation).
• Defines or loads environment parameters (A, B, C arrays).
• Instantiates Agent and calls methods (infer_states, infer_policies, sample_action).
b. Core Library
• Agent API – orchestrates inference, policy evaluation, control, and learning loops.
• Inference module – variational updates over hidden states given observations.
• Control module – computes expected free energies, policy posterior.
• Learning module – updates A/B matrices or model parameters online.
• Algos subpackage – specific implementations (FPI, message passing MMP).
• Utils/Maths – random matrix generators, array operations, Dirichlet functions.
c. Environment Models
• Provide observation and state transition functions.
• Used by examples and tests to simulate agent-environment loop.
d. JAX Extensions
• Parallel versions of Agent, inference, control, learning, maths to leverage JIT and GPUs.
e. External Libraries & Tools
• NumPy (core computations), JAX (optional acceleration), pytest (testing), Sphinx (docs).
• GitHub Actions for CI, ReadTheDocs for documentation hosting.
Architectural patterns and design principles
• Modular layering: separation of high-level API (Agent) from low-level math/inference functions.
• Dual implementation pattern: mirrored NumPy and JAX modules under pymdp/jax/.
• Test-driven development: comprehensive unit tests, MATLAB crossvalidation suite.
• Pluggable environments: abstract Env interface in pymdp/envs/env.py.
• Declarative model representation: use of A (likelihood), B (transition), C (preferences) arrays.
Essential elements
• Arrowheads showing call direction and data flow (e.g., observations → inference, actions → environment)
• Annotations for major data objects (A-matrix, B-matrix, C-vector, q(s), π distributions)