How dmey/synthia works
1. Project shape
- Synthia is a Python library, not an application service.
- The main architectural boundary is a small public API over statistical model implementations for synthetic data generation.
2. Core workflow
- The central flow is: fit on labeled data → parameterize marginals/latent structure → generate new samples → return data in the original xarray-shaped structure.
- The library is designed around xarray datasets/data arrays, so labeled dimensions and coordinates are preserved through generation.
3. Main public subsystems
- synthia.generators: high-level orchestration for synthetic sample generation.
- synthia.copulas: dependency models for multivariate continuous/discrete/categorical data.
- synthia.parameterizers: marginal/distribution fitting and transformation logic.
- synthia.transform and synthia.distribution: supporting statistical transformation and distribution abstractions.
- synthia.util: shared helpers and dataset-loading utilities used by examples/tests.
4. Generator layer
- generators/copula.py and generators/fpca.py are the user-facing model builders.
- The generator layer coordinates fitting and sampling, hiding the lower-level details of copula fitting, quantile transforms, and inverse transforms.
- This is the main entry point for users, e.g. “fit then generate”.
5. Copula modeling subsystem
- copulas/copula.py defines the copula abstraction.
- copulas/gaussian.py provides a pure-Python Gaussian copula implementation.
- copulas/independence.py represents a baseline no-dependence model.
- copulas/vine.py supports vine copulas and is the one that depends on the external pyvinecopulib/vinecopulib stack for efficient C++-backed computation.
6. Marginal parameterization subsystem
- parameterizers/parameterizer.py appears to define the common parameterizer interface.
- parameterizers/distribution.py, quantile.py, and const.py cover different ways of modeling marginals:
- fitted distributions,
- empirical/quantile-based transforms,
- constant-valued variables.
- This layer is what maps real-valued or categorical observations into the latent space used by copulas and back again.
7. Functional PCA path
- The fPCA generator is a separate modeling path alongside copulas.
- It is intended for structured continuous data such as profiles or functional observations, where variation is represented through principal components rather than dependence copulas.
- In architecture terms, it is an alternate generation pipeline sharing the same high-level “fit/generate” user contract.
8. Data transformation boundary
- transform.py likely holds the conversions between original observations, normalized latent variables, and reconstructed samples.
- Conceptually this is the bridge between marginal models and multivariate dependency models: data is standardized/transformed before fitting the latent structure and then inverted after sampling.
9. External dependencies that matter
- xarray is a key architectural dependency because the library preserves labeled dimension/coordinate metadata.
- pyvinecopulib is the main non-Python runtime dependency and enables vine copula support via C++ bindings.
- The rest of the stack is largely pure Python, which keeps the Gaussian copula and fPCA paths portable.
10. Documentation and examples
- docs/ is substantial and includes notebooks for univariate, multivariate Gaussian, vine, discrete/categorical, and fPCA examples.
- This suggests the repo’s usage model is example-driven and that the docs are part of the primary interface for understanding data flow and supported variable types.
11. Testing and CI
- tests/ contains unit/benchmark coverage for generators and utilities, with pytest.ini configuring the test run.
- .github/workflows/ci.yml shows standard CI automation, so architecture changes are expected to be validated through automated tests rather than runtime deployment.
12. Packaging and distribution
- setup.py, environment files, and release metadata indicate a standard Python package distribution workflow.
- The repository is versioned and publishable to PyPI, with citation metadata and a paper included, reinforcing that this is a scientific software package with reproducible-model semantics rather than a service-oriented system.
Main entry points: generators/copula.py Copula gen · transform.py Transform · copulas/copula.py Copula base · parameterizer.py Parameterizer
Components 20
Library core
Main Python package implementing generators, copulas, parameterization, and transforms.
- Public API · Top-level package exports and user-facing surface.
synthia/__init__.py - Copula gen · High-level fit/generate workflow built around copula models.
synthia/generators/copula.py - fPCA gen · Alternative fit/generate pipeline for structured continuous data.
synthia/generators/fpca.py - Copula base · Shared copula interface and common behavior.
synthia/copulas/copula.py - Gaussian · Gaussian copula implementation for multivariate dependence.
synthia/copulas/gaussian.py - Independence · No-dependence baseline model.
synthia/copulas/independence.py - Vine copula · Vine copula implementation backed by external C++ bindings.
synthia/copulas/vine.py - Parameterizer · Common interface for fitting and transforming marginals.
synthia/parameterizers/parameterizer.py - Dist fit · Fits marginal distributions and transforms values.
synthia/parameterizers/distribution.py - Quantile fit · Quantile-based empirical marginal transform.
synthia/parameterizers/quantile.py - Constant · Handles constant-valued variables.
synthia/parameterizers/const.py - Transform · Converts between observed data, latent variables, and reconstructed samples.
synthia/transform.py - Distribution · Shared distribution helpers and abstractions.
synthia/distribution.py - Utilities · Common helpers and dataset-loading utilities.
synthia/util.py
Docs and examples
User-facing documentation and notebook examples that demonstrate supported workflows.
Quality checks
Tests and CI used to validate the package.
Connections 24
- Public API → Copula gen: exports · Public API exposes the main generator path.
- Public API → fPCA gen: exports · Public API exposes the alternative generator path.
- Copula gen → Parameterizer: fits · Generator coordinates marginal parameterization.
- Copula gen → Transform: uses · Generator bridges observed data and latent space.
- Copula gen → Copula base: samples · Generator relies on copula abstractions for dependence modeling.
- fPCA gen → Transform: uses · fPCA pipeline preserves the labeled data shape through transforms.
- fPCA gen → Utilities: uses · fPCA path shares helper utilities.
- Copula base → Gaussian: implements · Gaussian copula is one concrete implementation.
- Copula base → Independence: implements · Independence model is another concrete implementation.
- Copula base → Vine copula: implements · Vine copula extends the same abstraction.
- Vine copula → Distribution: depends on · Vine path interacts with statistical support code.
- Parameterizer → Dist fit: implements · Distribution-based parameterizer is a concrete strategy.
- Parameterizer → Quantile fit: implements · Quantile-based parameterizer is a concrete strategy.
- Parameterizer → Constant: implements · Constant-valued parameterizer is a concrete strategy.
- Dist fit → Distribution: uses · Distribution parameterization shares distribution abstractions.
- Quantile fit → Transform: uses · Quantile transforms are part of the latent-space bridge.
- Constant → Transform: uses · Constant handling participates in transform/inverse-transform flow.
- Transform → Distribution: uses · Transform layer relies on distribution abstractions.
- Docs → Examples: includes · Docs organize the notebook gallery and narrative examples.
- Tests → Utilities: covers · Utility behavior is exercised by tests.
- Tests → Copula gen: covers · Generator workflows are validated by tests.
- CI → Tests: runs · CI executes the test suite.
- Packaging → Public API: packages · Build configuration packages the library entrypoint and modules.
- Paper → Docs: supports · Publication materials reinforce the documentation and scientific framing.