How swagnercarena/paltas works
paltas is a Python package with two major architectural halves:
1. Simulation / dataset generation
2. Analysis / simulation-based inference
The repository is organized around a modular “configuration → sampling → physical components → image generation → saved dataset” pipeline, with a separate “dataset → neural network / posterior estimation → hierarchical inference” analysis pipeline.
1) Core purpose and top-level flow
The package generates realistic strong-lensing image datasets, primarily for HST-like observations, using lenstronomy as the lensing/image-simulation engine. A user supplies a Python config file, and paltas/generate.py turns that config into a batch of synthetic lenses. Those outputs are then consumed either as standalone datasets or by the analysis code in paltas/Analysis for training posterior estimators and doing population-level inference.
The important architectural boundary is:
• Configuration layer decides what distributions and component models to use.
• Physical component layer builds the lensing system pieces.
• Generation layer combines those pieces into simulated outputs.
• Analysis layer treats generated datasets as training/inference inputs.
2) Simulation architecture
A. Configuration subsystem
Located under paltas/Configs.
This is the user-facing definition layer for simulations. Config files are plain Python modules that describe which distributions and component classes to use, and with what parameters.
Key files and roles:
• paltas/Configs/config_handler.py: the central loader/dispatcher for config files.
• paltas/Configs/Examples/*: example configs for common simulation setups.
• paltas/Configs/Horton/*: configs tied to specific HST/COSMOS-style setups.
• paltas/Configs/paper_2203_00690/*: experiment-specific configs used for a published study.
Architecturally, configs are not just constants; they are the glue that binds together the model subsystems. A config defines which:
• source model to use
• main deflector model to use
• substructure model to use
• line-of-sight halo model to use
• point source model to use
• sampling distributions to draw parameter values from
This makes the system highly modular and lets the same generation pipeline produce very different lens populations.
B. Sampling subsystem
Located under paltas/Sampling.
This subsystem provides the distribution machinery used by configs and component classes.
Key files:
• distributions.py: parameter distribution definitions
• sampler.py: sampling orchestration
Its role is to translate config-specified priors/distributions into concrete parameter draws for one simulated lens realization. It is the stochastic boundary of the simulation stack: everything below it consumes sampled parameters, not abstract priors.
C. Physical component subsystems
These are the model families that represent pieces of a strong-lensing system.
1. Main deflector
Located in paltas/MainDeflector.
Files:
• main_deflector_base.py
• simple_deflectors.py
This subsystem represents the primary lens galaxy mass model, which bends light from the background source. It abstracts the main lens into a base interface plus concrete deflector implementations.
2. Source galaxies
Located in paltas/Sources.
Files:
• source_base.py
• sersic.py
• cosmos.py
• cosmos_sersic.py
• galaxy_catalog.py
This subsystem defines the background source light profile. It supports both analytic models, such as Sersic-based sources, and catalog-driven galaxy models based on COSMOS-like data.
Important data assets here:
• bad_galaxies.csv
• val_galaxies.csv
Those catalogs indicate that source selection is partly data-driven and that there is a curated catalog boundary between generic source modeling and real-galaxy image material.
3. Point sources
Located in paltas/PointSource.
Files:
• point_source_base.py
• single_point_source.py
This subsystem models compact emission components, such as a single point source. It is structurally separate from extended galaxy sources because its rendering and parameterization differ.
4. Substructure
Located in paltas/Substructure.
Files:
• los_base.py
• los_dg19.py
• subhalos_base.py
• subhalos_dg19.py
• nfw_functions.py
This subsystem adds small-scale structure to the lensing mass distribution:
• subhalos internal to the main lens halo
• line-of-sight halos between observer and source
The base/implementation split shows that the repo is built for multiple halo population models, with dg19 variants providing one concrete prescription. nfw_functions.py contains shared NFW-related computations used by these models.
5. Utility functions
Located in paltas/Utils.
Files:
• cosmology_utils.py
• hubble_utils.py
• lenstronomy_utils.py
• power_law.py
These are cross-cutting helpers. They are not standalone subsystems but support the physical models and the integration with lenstronomy. lenstronomy_utils.py is the strongest architectural hint that the package delegates core ray-tracing / lens rendering to the external lenstronomy engine.
D. Generation entry point
Located in paltas/generate.py.
This is the main execution path for dataset creation. The README’s primary workflow is:
• load a config file
• sample a lens system
• assemble source/lens/substructure/point-source components
• render the final image
• write outputs to disk
This is the central orchestrator for the simulation half of the repository.
Although the file tree does not show its internals, the surrounding structure implies that it coordinates the config handler, sampler, component models, and rendering utilities.
3) Analysis architecture
The analysis half lives in paltas/Analysis and is separated cleanly from simulation. Its role is to take generated datasets and run machine-learning-based inference on them.
A. Dataset preparation
Files:
• dataset_generation.py
This module likely converts generated images and metadata into model-ready training datasets. It forms the bridge between simulator output and ML training input.
B. Neural network models
Files:
• conv_models.py
• transformer_models.py
This subsystem contains alternative posterior-estimation architectures:
• convolutional models
• transformer-based models
The presence of both suggests the analysis layer is designed as a model zoo, not a single network implementation.
C. Training and loss
Files:
• train_model.py
• loss_functions.py
These provide the training loop and objective functions for neural posterior estimation. The training code consumes simulated data and model definitions, then optimizes posterior estimators.
D. Posterior and PDF utilities
Files:
• posterior_functions.py
• pdf_functions.py
These are the inference-side utilities for turning model outputs into usable probability distributions and posteriors.
E. Hierarchical inference
File:
• hierarchical_inference.py
This is a higher-level inference subsystem that operates on populations of lenses rather than individual systems. It is the repository’s second major analysis boundary:
• first-level inference: infer parameters per lens
• second-level inference: infer population-level hyperparameters from collections of inferred lenses
F. Analysis configs
Located in paltas/Analysis/AnalysisConfigs/*
These are experiment-specific training configurations for the analysis pipeline, analogous to the simulation configs in paltas/Configs. They define how a particular network or inference experiment should be run.
The split between Configs and AnalysisConfigs is important:
• Configs define synthetic universe generation
• AnalysisConfigs define learning/inference experiments on those synthetic universes
4) Data flow across the repository
A useful way to diagram the system is as two connected pipelines.
Simulation data flow
1. User config file
2. config_handler loads and resolves configuration
3. Sampling draws parameter values from configured distributions
4. Component subsystems build:
- main deflector mass model
- source light profile
- substructure population
- point sources
5. Utils and lenstronomy integration turn physical parameters into images
6. generate.py writes simulated data and metadata to disk
This flow is modular: each component can be swapped independently through configuration.
Analysis data flow
1. Generated datasets or saved lens images
2. dataset_generation.py prepares training/inference data
3. Neural model from conv_models.py or transformer_models.py
4. train_model.py optimizes the network with loss_functions.py
5. posterior_functions.py and pdf_functions.py convert outputs to distributions
6. hierarchical_inference.py aggregates inference across a population
This flow is a consumer of the simulation pipeline and does not directly generate the physical lens images itself.
5) External dependencies that matter architecturally
lenstronomy
This is the critical simulation backend. paltas wraps and extends it rather than replacing it. That means the physics/rendering boundary is externalized: paltas primarily defines the lens population, component models, and sampling logic, while lenstronomy handles the underlying lensing computations and image generation.
tensorflow
The analysis module depends on TensorFlow for neural posterior estimation. This is not part of the simulation runtime by default, only the ML/inference runtime.
Data files
The repo includes real-world assets that materially affect the architecture:
• COSMOS-related FITS files in datasets/hst_psf/
• HST PSF files
• source catalogs and galaxy images under test/test_data/cosmos/
• fake training data under test/test_data/fake_train/
These indicate that the system is not purely synthetic in isolation; it relies on real observational artifacts and curated catalog data to produce more realistic simulations and to support tests.
6) Packaging, validation, and documentation boundaries
Tests
test/ is structured by subsystem:
• analysis_tests.py
• config_tests.py
• generate_tests.py
• main_deflector_tests.py
• point_source_tests.py
• sampling_tests.py
• source_tests.py
• substructure_tests.py
• utils_tests.py
This mirrors the architecture and confirms the package is intentionally decomposed by domain boundary. test/test_data/ contains fixture data used to validate catalog handling, config parsing, and generated dataset behavior.
Documentation and notebooks
• docs/ contains the Sphinx documentation structure.
• notebooks/ provides tutorials for config creation, pipeline understanding, training, and population analysis.
These are not core runtime subsystems, but they reflect the two main user journeys:
• simulation authoring
• inference/training workflow
7) Suggested architecture diagram framing
If you are drawing the architecture, the clearest layout is:
• Left side: configuration and sampling
- config files
- config handler
- distributions/sampler
• Center: simulation components
- main deflector
- source models
- point sources
- substructure
- utilities / lenstronomy integration
• Output boundary
- generated images
- metadata / catalogs / saved arrays
• Right side: analysis pipeline
- dataset preparation
- neural network models
- training/loss
- posterior/PDF utilities
- hierarchical inference
• Supporting layer
- data assets: PSFs, COSMOS catalogs, test fixtures
- docs, notebooks, tests, CI
The key architectural idea is that paltas is a modular lens-simulation framework with a separate ML inference stack, connected through generated datasets and experiment-specific configs.
Main entry points: generate.py Generator · config_handler.py Config loader · train_model.py Training · dataset_generation.py Dataset prep
Components 22
Simulation
Configuration, sampling, physical components, and dataset generation.
- Config loader · Loads and resolves simulation config modules into a runnable setup.
paltas/Configs/config_handler.py - Sim configs · Example, Horton, and paper-specific configuration families for synthetic lens generation.
paltas/Configs - Sampler · Draws concrete parameter values from config-defined distributions.
paltas/Sampling/sampler.py - Distributions · Distribution definitions used to parameterize lens population sampling.
paltas/Sampling/distributions.py - Main deflector · Primary lens galaxy mass model and concrete deflector implementations.
paltas/MainDeflector - Sources · Extended background source models including analytic and catalog-driven galaxies.
paltas/Sources - Point source · Models compact emission components such as single point sources.
paltas/PointSource - Substructure · Subhalos and line-of-sight halo models added to the main lens mass distribution.
paltas/Substructure - Generator · Main execution entry point that assembles components and writes simulated outputs.
paltas/generate.py
Analysis
Neural posterior estimation and hierarchical inference on generated datasets.
- Dataset prep · Converts generated images and metadata into training-ready analysis inputs.
paltas/Analysis/dataset_generation.py - Conv nets · Convolutional neural network architectures for inference.
paltas/Analysis/conv_models.py - Transformers · Transformer-based inference architectures.
paltas/Analysis/transformer_models.py - Training · Trains inference models on simulated datasets using loss functions.
paltas/Analysis/train_model.py - Losses · Training objectives for neural posterior estimation.
paltas/Analysis/loss_functions.py - Posteriors · Utilities for converting model outputs into posterior estimates.
paltas/Analysis/posterior_functions.py - PDF utils · Probability-density helpers used by the analysis pipeline.
paltas/Analysis/pdf_functions.py - Hierarchical inf. · Aggregates per-lens inference into population-level hyperparameter inference.
paltas/Analysis/hierarchical_inference.py - Analysis configs · Training and inference experiment definitions for the analysis stack.
paltas/Analysis/AnalysisConfigs
Support
Shared utilities, reference data, docs, notebooks, and tests.
- Utilities · Cross-cutting cosmology, Hubble, power-law, and lenstronomy integration helpers.
paltas/Utils - Lenstronomy · External lensing and image-simulation engine used by the package.
- Data assets · PSFs, COSMOS catalogs, and fixture datasets used by simulation and tests.
datasets - Docs & tests · Documentation, notebooks, and subsystem-aligned tests covering the package architecture.
docs
Connections 26
- Sim configs → Config loader: loaded by · Configuration modules are resolved through the config handler.
- Config loader → Distributions: binds priors · Config resolution connects model choices to sampling distributions.
- Config loader → Main deflector: selects · Configs choose the main lens mass model.
- Config loader → Sources: selects · Configs choose the source light model.
- Config loader → Point source: selects · Configs may include compact source components.
- Config loader → Substructure: selects · Configs choose halo and line-of-sight structure models.
- Sampler → Distributions: draws from · Sampler turns priors into concrete parameter values.
- Generator → Config loader: loads config · Generation begins by reading the user-supplied config.
- Generator → Sampler: samples system · Generator requests a parameter draw for each realization.
- Generator → Main deflector: assembles lens · Generator composes the primary lens model into the scene.
- Generator → Sources: assembles source · Generator composes the background source model into the scene.
- Generator → Point source: assembles point light · Generator adds compact emission where configured.
- Generator → Substructure: adds halos · Generator includes substructure and line-of-sight perturbations.
- Generator → Utilities: uses helpers · Generator relies on shared physics and integration utilities.
- Utilities → Lenstronomy: bridges to · Utilities adapt repository models to the external simulation backend.
- Generator → Data assets: reads refs · Simulation can depend on PSFs and catalog assets.
- Dataset prep → Generator: consumes outputs · Analysis preparation starts from saved simulated datasets.
- Dataset prep → Analysis configs: driven by · Analysis experiments are defined by separate config modules.
- Training → Conv nets: trains · Training loop can optimize convolutional inference models.
- Training → Transformers: trains · Training loop can optimize transformer inference models.
- Training → Losses: optimizes with · Training uses configured objective functions.
- Posteriors → PDF utils: formats as · Posterior utilities build density-like outputs.
- Hierarchical inf. → Posteriors: aggregates · Population inference consumes per-lens posterior products.
- Hierarchical inf. → Dataset prep: consumes data · Hierarchical inference operates on prepared datasets and inference outputs.
- Docs & tests → Sim configs: documents · Documentation and tests mirror the simulation configuration layer.
- Docs & tests → Analysis configs: documents · Documentation and tests mirror the analysis configuration layer.