How google/dpsynth works
DPSynth generates differentially private synthetic tabular data through three documented routes: in-memory synthesis, a scalable pipeline, and post-processing of existing noisy marginals. The main workflow describes a schema, measures or supplies marginals, fits a discrete model, and decodes synthetic records. The graph also preserves documented evaluation and relational/typed-data capabilities. The excerpts confirm parts of local synthesis, Beam encoding, and relational processing; links for unsampled subsystems follow README descriptions and are limited to documented responsibilities.
Main entry points: data_generation_v3.py Local Generation · data_generation.py Pipeline Generation · pipeline_transformations Pipeline Transformations · synthesizer.py Relational Synthesis
Components 21
Schema and Encoding
- Domain Schema
dpsynth/domain.py - Constraints
dpsynth/constraints.py - Domain Transformations
dpsynth/transformations.py
Local Synthesis
- TabularConfig
dpsynth/__init__.py - Local Generation
dpsynth/data_generation_v3.py - Local Primitives
dpsynth/local_mode/primitives.py - Discrete Mechanisms
dpsynth/discrete_mechanisms - Marginal Postprocessing
dpsynth/postprocessing.py
Scalable Pipeline
- Pipeline Generation
dpsynth/data_generation.py - Dataset Descriptors
dpsynth/dataset_descriptors - Pipeline Transformations
dpsynth/pipeline_transformations - Beam Adapter
dpsynth/adapters/beam.py - Marginal Computation
dpsynth/pipeline_transformations/marginals_computations.py
Extended Synthesis
- Relational Synthesis
dpsynth/relational/synthesizer.py - Relational Postprocessing
dpsynth/relational/post_processing.py - Nested Synthesis
dpsynth/experimental/nested.py - Private Text Tools
dpsynth/text
Evaluation
- Tabular Evaluation
dpsynth/eval/tabular_eval.py - Dataset Statistics
dpsynth/eval/attribute_statistics.py
Other components
- Data User
- Real or Synthetic Data
Connections 25
- Data User → TabularConfig: configures
- TabularConfig → Local Generation: calibrates
- Domain Schema → Local Generation: defines schema
- Constraints → Local Generation: constrains
- Local Generation → Domain Transformations: encodes data
- Local Generation → Discrete Mechanisms: runs mechanism
- Local Generation → Local Primitives: uses primitives
- Local Generation → Data User: returns synthetic data
- Data User → Pipeline Generation: configures
- Pipeline Generation → Dataset Descriptors: uses descriptors
- Pipeline Generation → Pipeline Transformations: runs operations
- Pipeline Transformations → Beam Adapter: dispatches Beam
- Pipeline Transformations → Marginal Computation: computes marginals
- Pipeline Generation → Data User: produces dataset
- Data User → Marginal Postprocessing: supplies marginals
- Marginal Postprocessing → Data User: returns synthetic data
- Data User → Relational Synthesis: supplies tables
- Relational Synthesis → Local Generation: creates initializers
- Relational Synthesis → Relational Postprocessing: postprocesses tables
- Data User → Nested Synthesis: configures synthesis
- Nested Synthesis → Local Generation: synthesizes components
- Data User → Private Text Tools: uses text tools
- Real or Synthetic Data → Tabular Evaluation: provides dataset
- Tabular Evaluation → Dataset Statistics: computes statistics
- Tabular Evaluation → Data User: returns evaluation