How dso-org/deep-symbolic-optimization works
1. What this repository is
- dso is a Python package for Deep Symbolic Optimization: it learns symbolic expressions or symbolic control policies by sampling programs from a learned policy and scoring them on a task.
- The repo is split into a core optimization engine plus two first-class task families: regression and control.
2. Primary runtime entry points
- dso/run.py is the command-line launcher described in the README.
- dso/core.py contains the main DeepSymbolicOptimizer API used from Python.
- dso/train.py likely owns the training loop orchestration, with run.py and the Python API both feeding into the same core path.
3. Configuration is the main control surface
- Runs are driven by JSON configs under dso/config/, with shared defaults in config_common.json and task-specific defaults in config_regression.json and config_control.json.
- Example configs in config/examples/ show the two main task shapes: datasets for regression and Gym-style environments for control.
- This means the architecture is configuration-first: the task, search space, priors, optimizer, logging, and optional GP/LM components are all selected before execution.
4. Core symbolic representation and execution
- program.py, library.py, functions.py, and execute.py form the symbolic expression layer.
- The library defines tokens/operators available to the search process; program represents a candidate expression; execute evaluates a program on task inputs.
- cyfunc.pyx is a native acceleration path for function execution, indicating that expression evaluation is performance-sensitive.
5. Search-space shaping: priors and constraints
- prior.py and const.py define search priors, constraints, and special tokens such as floating constants.
- memory.py, variance.py, and subroutines.py support the optimizer’s statistical bookkeeping and helper operations.
- In architecture terms, these components sit between the raw token library and the policy sampler, biasing which expressions get explored.
6. Learning loop and state management
- The optimizer loop is centered around sampling candidate programs, evaluating reward, and updating the policy.
- checkpoint.py, train_stats.py, logeval.py, and tf_state_manager.py handle persistence, metrics, and TensorFlow state restoration.
- The data flow is: config → task + library/prior → policy samples programs → task evaluates reward → optimizer updates parameters → logs/checkpoints written.
7. Policy model subsystem
- policy/ contains the learned sequence model used to generate symbolic programs.
- policy/rnn_policy.py suggests an RNN-based autoregressive policy, with policy/policy.py providing the shared policy interface.
- This policy is the central generator in the system: it produces token sequences that become candidate expressions or policies.
8. Policy optimization strategies
- policy_optimizer/ contains interchangeable optimizers: pg_policy_optimizer.py, ppo_policy_optimizer.py, and pqt_policy_optimizer.py.
- These correspond to different training objectives over sampled programs, including risk-seeking policy gradient, PPO, and priority-queue training.
- Architecturally, this is a pluggable optimization boundary: the same program sampler and task evaluator can be paired with different update rules.
9. Task abstraction
- task/task.py defines the task interface shared by all problem types.
- The README and tree show a simple extension point for custom tasks, so tasks are not hard-wired into the optimizer; they supply reward evaluation, input/output structure, and any task-specific token additions.
10. Regression subsystem
- task/regression/ contains dataset loaders, benchmark definitions, and regression-specific utilities.
- dataset.py, regression.py, polyfit.py, sklearn.py, and mat_mult_benchmark.py indicate support for CSV datasets, benchmark suites, and polynomial/linear sub-fitting inside expressions.
- The regression task is centered on recovering mathematical expressions from data, with optional constant optimization and the poly token as a structured inner optimization.
11. Control subsystem
- task/control/ wraps symbolic policy search over reinforcement-learning environments.
- control.py, envs/, utils.py, and data/ show that this task includes custom environment adapters and pretrained anchor policies.
- Important architectural boundary: for multi-action environments, the system can mix symbolic actions, fixed symbolic expressions, and anchor-policy outputs within one action specification.
12. Optional hybrid search: genetic programming
- gp/ contains a separate GP controller (gp_controller.py, base.py, utils.py) used for neural-guided genetic programming population seeding.
- This is an inner-loop optimizer that can be enabled from config and runs alongside or inside the main neural-guided search process.
- In diagrams, it should be shown as an optional branch that consumes task evaluations and produces candidate populations.
13. Optional language-model prior
- language_model/ contains a learned prior over token sequences, including a saved TensorFlow model checkpoint.
- language_model_prior.py integrates that model into the search distribution, biasing program sampling toward patterns learned offline.
- This is a distinct prior source, separate from handcrafted priors in prior.py.
14. Testing, artifacts, and packaged assets
- test/ contains unit tests for core program behavior, priors, checkpointing, and custom-task extension points.
- The repo also ships benchmark data, pretrained control policies, and a saved language-model checkpoint, meaning the package is not purely code-driven; it depends on bundled artifacts for reproducible experiments.
- The setup.py packaging and the TensorFlow-era saved models indicate the project is designed as an installable research framework with both library and experiment assets.
Main entry points: train.py Training loop · config_common.json Configs · task.py Task API · program.py Program
Components 25
Core engine
Top-level orchestration, symbolic representation, and execution.
- CLI run · Command-line launcher for experiments.
dso/dso/run.py - Optimizer API · Main DeepSymbolicOptimizer API used programmatically.
dso/dso/core.py - Training loop · Coordinates sampling, evaluation, updates, and logging.
dso/dso/train.py - Program · Candidate symbolic expression or policy representation.
dso/dso/program.py - Library · Defines available symbols and operators.
dso/dso/library.py - Execute · Runs programs on task inputs to produce outputs or rewards.
dso/dso/execute.py - Functions · Primitive math and functional building blocks.
dso/dso/functions.py - Native funcs · Cython fast path for function execution.
dso/dso/cyfunc.pyx - Priors · Handcrafted priors and constraints over programs.
dso/dso/prior.py - Constants · Floating constants and related token handling.
dso/dso/const.py - Configs · JSON configuration layer for common, regression, control, and GP runs.
dso/dso/config/config_common.json - State & logs · Checkpoints, TensorFlow state, metrics, and evaluation logging.
dso/dso/checkpoint.py
Policy search
Learned sampler and interchangeable training objectives.
- Policy model · Shared interface for the learned program sampler.
dso/dso/policy/policy.py - RNN policy · RNN-based sampler that emits token sequences.
dso/dso/policy/rnn_policy.py - PG optimizer · Risk-seeking policy-gradient training.
dso/dso/policy_optimizer/pg_policy_optimizer.py - PPO optimizer · PPO-based policy optimization.
dso/dso/policy_optimizer/ppo_policy_optimizer.py - PQT optimizer · Priority-queue training over sampled programs.
dso/dso/policy_optimizer/pqt_policy_optimizer.py
Tasks
Regression and control task families.
- Task API · Common task contract for evaluation and extensions.
dso/dso/task/task.py - Regression · Symbolic regression on datasets and benchmarks.
dso/dso/task/regression/regression.py - Control · Symbolic control policies for RL environments.
dso/dso/task/control/control.py
Optional branches
Alternative priors and inner-loop search components.
- Genetic search · Optional GP controller used for population seeding or hybrid search.
dso/dso/gp/gp_controller.py - LM prior · Learned token prior from an offline language model.
dso/dso/language_model/language_model_prior.py
Packaged assets
Bundled models, datasets, and experiment artifacts.
- Regression data · Bundled regression datasets and benchmark definitions.
dso/dso/task/regression/data/Nguyen-1.csv - Control models · Pretrained anchor policies and environment-specific assets.
dso/dso/task/control/data/Pendulum-v0/model-td3.pkl - LM checkpoint · Saved language-model checkpoint used by the learned prior.
dso/dso/language_model/model/saved_model/dynrnn.meta
Connections 29
- CLI run → Configs: loads · CLI experiments are driven by JSON configuration.
- Optimizer API → Training loop: invokes · Python API enters the shared training path.
- CLI run → Training loop: starts · Command-line entrypoint launches training.
- Training loop → Task API: uses · Training depends on the task interface for evaluation.
- Training loop → Policy model: updates · Training coordinates policy optimization.
- Policy model → RNN policy: implemented by · Primary learned sampler is RNN-based.
- Training loop → PG optimizer: selects · Config may choose a policy-gradient optimizer.
- Training loop → PPO optimizer: selects · Config may choose PPO.
- Training loop → PQT optimizer: selects · Config may choose priority-queue training.
- RNN policy → Program: samples · Policy emits token sequences that become programs.
- Program → Library: constrains · Program structure is defined over library tokens.
- Program → Execute: evaluated by · Candidate programs are executed on task inputs.
- Execute → Functions: uses · Execution relies on primitive functions.
- Functions → Native funcs: accelerated by · Native Cython path speeds up evaluation.
- Priors → Library: biases · Priors shape the search over available tokens.
- Constants → Library: extends · Constants add special token support.
- Task API → Regression: specializes to · Regression implements the shared task contract.
- Task API → Control: specializes to · Control implements the shared task contract.
- Regression → Regression data: reads · Regression experiments use bundled datasets.
- Control → Control models: reads · Control experiments use bundled environment models.
- Training loop → State & logs: persists · Training writes checkpoints and metrics.
- Training loop → Genetic search: optional branch · Hybrid GP search can feed candidate populations.
- Training loop → LM prior: optional branch · Learned language-model prior can bias sampling.
- LM prior → LM checkpoint: loads · The LM prior depends on saved checkpoint artifacts.
- Configs → Priors: configures · Config chooses priors and constraints.
- Configs → Task API: configures · Config selects task family and task-specific settings.
- Configs → Policy model: configures · Config selects sampler and model parameters.
- Configs → Genetic search: enables · GP can be turned on from configuration.
- Configs → LM prior: enables · Language-model prior can be turned on from configuration.