# saumya028/msme_data_generation architecture

> Architecture diagram of the GitHub repository saumya028/msme_data_generation, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/saumya028/msme_data_generation
- Repository: https://github.com/saumya028/msme_data_generation

## Overview

CreditIQ Synthetic MSME Financial Data Generator produces linked synthetic MSME profiles and financial records; it does not score credit or compute CreditIQ features. Its main workflow generates profiles and latent parameters, monthly activity, GST/E-Way/UPI records, monthly financial aggregates, then consistency checks and a quality report. An optional comparison workflow fits Copula and conditional-GMM synthesizers to generated data. The README documents major stages; sampled excerpts confirm CLI orchestration and synthesizer calls, while aggregate internals were not shown.

Main entry points: [`generate_data.py`](https://github.com/saumya028/msme_data_generation/blob/main/generate_data.py) (Generation CLI), [`models/compare_synthesizers.py`](https://github.com/saumya028/msme_data_generation/blob/main/models/compare_synthesizers.py) (Synthesizer comparison), [`generators/financial_behavior.py`](https://github.com/saumya028/msme_data_generation/blob/main/generators/financial_behavior.py) (Monthly activity), [`generators/gst_generator.py`](https://github.com/saumya028/msme_data_generation/blob/main/generators/gst_generator.py) (GST returns)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_generation["Rule-based generation"]
  node_cli["Generation CLI<br/>[generate_data.py]"]
  node_profiles["MSME profiles and DNA<br/>[msme_generator.py]"]
  node_activity["Monthly activity"]
  node_gst["GST returns<br/>[gst_generator.py]"]
  node_eway["E-Way bills<br/>[eway_generator.py]"]
  node_upi["UPI transactions<br/>[upi_generator.py]"]
  node_aggregate["Financial aggregation<br/>[aggregate.py]"]
  node_dataset[("Generated datasets")]
end

subgraph group_validation["Validation and reporting"]
  node_checks["Consistency checks<br/>[consistency.py]"]
  node_quality["Quality report<br/>[quality_report.py]"]
  node_reports["Quality outputs"]
end

subgraph group_synthesis["Synthetic augmentation"]
  node_compare["Synthesizer comparison"]
  node_copula["Gaussian Copula"]
  node_gmm["Conditional GMM<br/>[ctgan_generator.py]"]
end

node_operator(("Data operator"))

node_operator -->|"runs"| node_cli
node_cli -->|"generates"| node_profiles
node_cli -->|"generates"| node_activity
node_cli -->|"generates"| node_gst
node_cli -->|"generates"| node_eway
node_cli -->|"generates"| node_upi
node_cli -->|"builds"| node_aggregate
node_cli -->|"writes"| node_dataset
node_cli -->|"runs"| node_checks
node_cli -->|"generates"| node_quality
node_profiles -->|"provides DNA"| node_activity
node_activity -->|"conditions"| node_gst
node_activity -->|"conditions"| node_upi
node_gst -->|"supplies returns"| node_eway
node_gst -->|"aggregates"| node_aggregate
node_eway -->|"aggregates"| node_aggregate
node_upi -->|"aggregates"| node_aggregate
node_cli -->|"checks datasets"| node_checks
node_checks -->|"provides results"| node_quality
node_quality -->|"writes reports"| node_reports
node_operator -.->|"runs optionally"| node_compare
node_dataset -->|"provides seed data"| node_compare
node_compare -->|"fits and samples"| node_copula
node_compare -->|"fits and samples"| node_gmm
node_compare -->|"writes samples"| node_dataset
node_compare -->|"writes comparison"| node_reports

click node_cli "https://github.com/saumya028/msme_data_generation/blob/main/generate_data.py"
click node_profiles "https://github.com/saumya028/msme_data_generation/blob/main/generators/msme_generator.py"
click node_activity "https://github.com/saumya028/msme_data_generation/blob/main/generators/financial_behavior.py"
click node_gst "https://github.com/saumya028/msme_data_generation/blob/main/generators/gst_generator.py"
click node_eway "https://github.com/saumya028/msme_data_generation/blob/main/generators/eway_generator.py"
click node_upi "https://github.com/saumya028/msme_data_generation/blob/main/generators/upi_generator.py"
click node_aggregate "https://github.com/saumya028/msme_data_generation/blob/main/generators/aggregate.py"
click node_dataset "https://github.com/saumya028/msme_data_generation/tree/main/data"
click node_checks "https://github.com/saumya028/msme_data_generation/blob/main/validation/consistency.py"
click node_quality "https://github.com/saumya028/msme_data_generation/blob/main/validation/quality_report.py"
click node_reports "https://github.com/saumya028/msme_data_generation/tree/main/reports"
click node_compare "https://github.com/saumya028/msme_data_generation/blob/main/models/compare_synthesizers.py"
click node_copula "https://github.com/saumya028/msme_data_generation/blob/main/models/copula_generator.py"
click node_gmm "https://github.com/saumya028/msme_data_generation/blob/main/models/ctgan_generator.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_profiles,node_activity,node_gst,node_eway,node_upi,node_aggregate,node_dataset toneBlue
class node_checks,node_quality,node_reports toneAmber
class node_compare,node_copula,node_gmm toneMint
class node_operator toneIndigo
```

## Components

### Rule-based generation

- [Generation CLI](https://github.com/saumya028/msme_data_generation/blob/main/generate_data.py): `generate_data.py`
- [MSME profiles and DNA](https://github.com/saumya028/msme_data_generation/blob/main/generators/msme_generator.py): `generators/msme_generator.py`
- [Monthly activity](https://github.com/saumya028/msme_data_generation/blob/main/generators/financial_behavior.py): `generators/financial_behavior.py`
- [GST returns](https://github.com/saumya028/msme_data_generation/blob/main/generators/gst_generator.py): `generators/gst_generator.py`
- [E-Way bills](https://github.com/saumya028/msme_data_generation/blob/main/generators/eway_generator.py): `generators/eway_generator.py`
- [UPI transactions](https://github.com/saumya028/msme_data_generation/blob/main/generators/upi_generator.py): `generators/upi_generator.py`
- [Financial aggregation](https://github.com/saumya028/msme_data_generation/blob/main/generators/aggregate.py): `generators/aggregate.py`
- [Generated datasets](https://github.com/saumya028/msme_data_generation/tree/main/data): `data`

### Validation and reporting

- [Consistency checks](https://github.com/saumya028/msme_data_generation/blob/main/validation/consistency.py): `validation/consistency.py`
- [Quality report](https://github.com/saumya028/msme_data_generation/blob/main/validation/quality_report.py): `validation/quality_report.py`
- [Quality outputs](https://github.com/saumya028/msme_data_generation/tree/main/reports): `reports`

### Synthetic augmentation

- [Synthesizer comparison](https://github.com/saumya028/msme_data_generation/blob/main/models/compare_synthesizers.py): `models/compare_synthesizers.py`
- [Gaussian Copula](https://github.com/saumya028/msme_data_generation/blob/main/models/copula_generator.py): `models/copula_generator.py`
- [Conditional GMM](https://github.com/saumya028/msme_data_generation/blob/main/models/ctgan_generator.py): `models/ctgan_generator.py`

### Other components

- **Data operator**

## Connections

- Data operator → Generation CLI: runs (evidence: `README.md`)
- Generation CLI → MSME profiles and DNA: generates (evidence: `generate_data.py`)
- Generation CLI → Monthly activity: generates (evidence: `generate_data.py`)
- Generation CLI → GST returns: generates (evidence: `generate_data.py`)
- Generation CLI → E-Way bills: generates (evidence: `generate_data.py`)
- Generation CLI → UPI transactions: generates (evidence: `generate_data.py`)
- Generation CLI → Financial aggregation: builds (evidence: `generate_data.py`)
- Generation CLI → Generated datasets: writes (evidence: `generate_data.py`)
- Generation CLI → Consistency checks: runs (evidence: `generate_data.py`)
- Generation CLI → Quality report: generates (evidence: `generate_data.py`)
- MSME profiles and DNA → Monthly activity: provides DNA (evidence: `README.md`)
- Monthly activity → GST returns: conditions (evidence: `README.md`)
- Monthly activity → UPI transactions: conditions (evidence: `README.md`)
- GST returns → E-Way bills: supplies returns (evidence: `generators/eway_generator.py`)
- GST returns → Financial aggregation: aggregates (evidence: `README.md`)
- E-Way bills → Financial aggregation: aggregates (evidence: `README.md`)
- UPI transactions → Financial aggregation: aggregates (evidence: `README.md`)
- Generation CLI → Consistency checks: checks datasets (evidence: `generate_data.py`)
- Consistency checks → Quality report: provides results (evidence: `generate_data.py`)
- Quality report → Quality outputs: writes reports (evidence: `README.md`)
- Data operator → Synthesizer comparison: runs optionally (evidence: `README.md`)
- Generated datasets → Synthesizer comparison: provides seed data (evidence: `models/compare_synthesizers.py`)
- Synthesizer comparison → Gaussian Copula: fits and samples (evidence: `models/compare_synthesizers.py`)
- Synthesizer comparison → Conditional GMM: fits and samples (evidence: `models/compare_synthesizers.py`)
- Synthesizer comparison → Generated datasets: writes samples (evidence: `README.md`)
- Synthesizer comparison → Quality outputs: writes comparison (evidence: `README.md`)
