# sdv-dev/sdv architecture

> Architecture diagram of the GitHub repository sdv-dev/sdv, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sdv-dev/sdv
- Repository: https://github.com/sdv-dev/sdv

## Overview

##### Project Type and Purpose
- The SDV is a Python library for generating synthetic data using machine learning models. Its scope covers various data modalities (single table, multi table, sequential) and includes end-to-end capabilities such as data processing, synthesis (via multiple modeling approaches), evaluation, and visualization.
- The README highlights that the SDV handles not only data synthesis but also preprocessing (e.g., anonymization and constraint enforcement) and evaluation (quality reports, visual comparisons).

##### Analysis of the File Structure
- Major top-level directories include:
- “sdv”: This is the central module containing all the core functionalities.
- Subdirectories within “sdv/” include:
- constraints – Implements business logic and constraint handling.
- data_processing – Contains processing modules (e.g., formatting and cleaning of date and numerical data).
- datasets – Provides routines to load both demo and local data.
- evaluation – Implements mechanisms to compare real and synthetic data quality, including report generation and visualization.
- io – Manages input/output operations, including local file operations.
- lite, multi_table, single_table, sequential – Each handling a different modality or architectural variant for synthetic data generation.
- logging, metadata, metrics, sampling, and utils – Provide supporting functionalities: logging configurations, metadata management, measurement of data quality, various sampling strategies, and common utilities.
- Additional directories such as “docs” give insight into documentation and theming, while “tests” (divided into unit and integration) attest to the extensive testing approach.
- The GitHub workflows and CI/CD configurations indicate strong emphasis on code quality and continuous integration.

a. Main Components:
- Data Ingestion & Preprocessing:
- Datasets Module (“sdv/datasets”): Loads demo or local tabular data.
- Data Processing (“sdv/data_processing”): Handles formatting, cleaning, and preparation (e.g., datetime_formatter and numerical_formatter).
- Metadata and Configuration:
- Metadata Management (“sdv/metadata”): Handles dataset descriptions, upgrades, and visualization of metadata.
- Synthesis Engine:
- Modeling Modules (“sdv/single_table”, “sdv/multi_table”, “sdv/sequential”): Contains various synthesizers (like GaussianCopulaSynthesizer, CTGAN, Copulas).
- Constraints (“sdv/constraints”): Enforces constraints on the synthetic data (logical rules, anonymization).
- Sampling (“sdv/sampling”): Implements algorithms for sampling synthetic data.
- Evaluation & Quality Assessment:
- Evaluation (“sdv/evaluation”): Computes quality metrics and generates visual reports comparing synthetic and real data.
- Metrics (“sdv/metrics”): Houses functions and demos for statistical and relational assessments.
- Utility & Support Services:
- Input/Output (“sdv/io”): Manages reading/writing of local data.
- Logging (“sdv/logging”): Provides logging mechanisms to trace operations and errors.
- Miscellaneous and Utilities (“sdv/utils”, “sdv/lite”): Additional modules that support the core functionalities.
- Testing & Continuous Integration:
- Tests Directory: Contains unit tests and integration tests ensuring each component’s correctness.
- CI/CD Files (GitHub workflows, Makefile): Illustrate automated quality checks and build processes.

b. Relationships and Interactions:

c. Architectural Patterns and Design Principles:
- Modularity: Each major functionality (data ingestion, processing, synthesis, evaluation) resides in its own package.
- Separation of Concerns: There is a clear boundary between pre-processing, synthesis, evaluation, and support utilities.
- Extensibility: The design allows for plugging in different models (copulas, CTGAN, hierarchical sampler) and adding new constraints or evaluation metrics.
- Testability: The presence of comprehensive unit and integration tests supports a layered design where components can be independently verified.

d. Technologies, Frameworks and Libraries:

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "Data Ingestion & Preprocessing"
        A["Datasets Module"]:::ingestion
        B["Data Processing"]:::ingestion
    end

    subgraph "Metadata & Configuration"
        C["Metadata Management"]:::metadata
    end

    subgraph "Synthesis Engine"
        D["Single Table Synthesis"]:::synthesis
        E["Multi Table Synthesis"]:::synthesis
        F["Sequential Synthesis"]:::synthesis
        G["Constraints Enforcement"]:::synthesis
        H["Sampling Strategies"]:::synthesis
        I["Lite Support"]:::synthesis
    end

    subgraph "Evaluation & Quality"
        J["Evaluation Module"]:::evaluation
        K["Metrics and Demos"]:::evaluation
    end

    subgraph "Utility & Support Services"
        L["Input/Output Operations"]:::utility
        M["Logging"]:::utility
        N["Miscellaneous Utilities"]:::utility
    end

    subgraph "Testing & Continuous Integration"
        O["Unit & Integration Tests"]:::ci
        P["CI/CD Workflows"]:::ci
    end

    %% Main Data Flow
    A -->|"raw_data"| B
    B -->|"processed_data"| D
    B -->|"processed_data"| E
    B -->|"processed_data"| F
    C -->|"metadata"| D
    C -->|"metadata"| E
    C -->|"metadata"| F
    G -->|"constraints"| D
    G -->|"constraints"| E
    G -->|"constraints"| F
    H -->|"sampling"| D
    H -->|"sampling"| E
    H -->|"sampling"| F
    I -->|"support"| D
    I -->|"support"| E
    I -->|"support"| F
    D -->|"synthetic_output"| J
    E -->|"synthetic_output"| J
    F -->|"synthetic_output"| J
    J -->|"quality_analysis"| K

    %% Utility & Support Dotted Connections
    L -.-> A
    L -.-> B
    L -.-> C
    L -.-> D
    L -.-> E
    L -.-> F
    L -.-> J

    M -.-> A
    M -.-> B
    M -.-> C
    M -.-> D
    M -.-> E
    M -.-> F
    M -.-> J

    N -.-> A
    N -.-> B
    N -.-> C
    N -.-> D
    N -.-> E
    N -.-> F
    N -.-> J

    %% Click Events
    click A "https://github.com/sdv-dev/sdv/tree/main/sdv/datasets"
    click B "https://github.com/sdv-dev/sdv/tree/main/sdv/data_processing"
    click C "https://github.com/sdv-dev/sdv/tree/main/sdv/metadata"
    click D "https://github.com/sdv-dev/sdv/tree/main/sdv/single_table"
    click E "https://github.com/sdv-dev/sdv/tree/main/sdv/multi_table"
    click F "https://github.com/sdv-dev/sdv/tree/main/sdv/sequential"
    click G "https://github.com/sdv-dev/sdv/tree/main/sdv/constraints"
    click H "https://github.com/sdv-dev/sdv/tree/main/sdv/sampling"
    click I "https://github.com/sdv-dev/sdv/tree/main/sdv/lite"
    click J "https://github.com/sdv-dev/sdv/tree/main/sdv/evaluation"
    click K "https://github.com/sdv-dev/sdv/tree/main/sdv/metrics"
    click L "https://github.com/sdv-dev/sdv/tree/main/sdv/io"
    click M "https://github.com/sdv-dev/sdv/tree/main/sdv/logging"
    click N "https://github.com/sdv-dev/sdv/tree/main/sdv/utils"
    click O "https://github.com/sdv-dev/sdv/tree/main/tests"
    click P "https://github.com/sdv-dev/sdv/tree/main/.github/workflows"

    %% Styles
    classDef ingestion fill:#AED6F1,stroke:#1F618D,stroke-width:2px;
    classDef metadata fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
    classDef synthesis fill:#A9DFBF,stroke:#27AE60,stroke-width:2px;
    classDef evaluation fill:#F5B7B1,stroke:#C0392B,stroke-width:2px;
    classDef utility fill:#D7BDE2,stroke:#8E44AD,stroke-width:2px;
    classDef ci fill:#FAD7A0,stroke:#D35400,stroke-width:2px;
```
