How saumya028/msme_data_generation works
CreditIQ Synthetic MSME Financial Data Generator produces linked synthetic MSME profiles and financial records; it does not score credit or compute CreditIQ features. Its main workflow generates profiles and latent parameters, monthly activity, GST/E-Way/UPI records, monthly financial aggregates, then consistency checks and a quality report. An optional comparison workflow fits Copula and conditional-GMM synthesizers to generated data. The README documents major stages; sampled excerpts confirm CLI orchestration and synthesizer calls, while aggregate internals were not shown.
Main entry points: generate_data.py Generation CLI · compare_synthesizers.py Synthesizer comparison · financial_behavior.py Monthly activity · gst_generator.py GST returns
Components 15
Rule-based generation
- Generation CLI
generate_data.py - MSME profiles and DNA
generators/msme_generator.py - Monthly activity
generators/financial_behavior.py - GST returns
generators/gst_generator.py - E-Way bills
generators/eway_generator.py - UPI transactions
generators/upi_generator.py - Financial aggregation
generators/aggregate.py - Generated datasets
data
Validation and reporting
- Consistency checks
validation/consistency.py - Quality report
validation/quality_report.py - Quality outputs
reports
Synthetic augmentation
- Synthesizer comparison
models/compare_synthesizers.py - Gaussian Copula
models/copula_generator.py - Conditional GMM
models/ctgan_generator.py
Other components
- Data operator
Connections 26
- Data operator → Generation CLI: runs
README.md - Generation CLI → MSME profiles and DNA: generates
generate_data.py - Generation CLI → Monthly activity: generates
generate_data.py - Generation CLI → GST returns: generates
generate_data.py - Generation CLI → E-Way bills: generates
generate_data.py - Generation CLI → UPI transactions: generates
generate_data.py - Generation CLI → Financial aggregation: builds
generate_data.py - Generation CLI → Generated datasets: writes
generate_data.py - Generation CLI → Consistency checks: runs
generate_data.py - Generation CLI → Quality report: generates
generate_data.py - MSME profiles and DNA → Monthly activity: provides DNA
README.md - Monthly activity → GST returns: conditions
README.md - Monthly activity → UPI transactions: conditions
README.md - GST returns → E-Way bills: supplies returns
generators/eway_generator.py - GST returns → Financial aggregation: aggregates
README.md - E-Way bills → Financial aggregation: aggregates
README.md - UPI transactions → Financial aggregation: aggregates
README.md - Generation CLI → Consistency checks: checks datasets
generate_data.py - Consistency checks → Quality report: provides results
generate_data.py - Quality report → Quality outputs: writes reports
README.md - Data operator → Synthesizer comparison: runs optionally
README.md - Generated datasets → Synthesizer comparison: provides seed data
models/compare_synthesizers.py - Synthesizer comparison → Gaussian Copula: fits and samples
models/compare_synthesizers.py - Synthesizer comparison → Conditional GMM: fits and samples
models/compare_synthesizers.py - Synthesizer comparison → Generated datasets: writes samples
README.md - Synthesizer comparison → Quality outputs: writes comparison
README.md