How adityaaa89/codekavach-synthetic-pipeline works
CodeKavach is a synthetic cybercrime data pipeline: it calibrates simulation parameters from defaults and reference data, generates relational fraud cases and cash-outs, derives task-specific ML datasets, and validates the generated outputs. The main workflow is explicit in main.py; sampled generator and dataset-builder code supports the principal data transformations. Generation internals and some output wiring are summarized because the generator excerpt is incomplete.
Main entry points: generate_dataset.py Cybercrime Generator · ml_ready ML Training Data · main.py Pipeline Runner · validate_dataset.py Statistical Validator
Components 21
Pipeline
- Pipeline Runner
main.py
Calibration
- Simulation Defaults
config/defaults.json - Reference Data
data/reference - Parameter Calibration
src/calibrate.py - Calibrated Settings
config/calibration_config.json
Fraud Simulation
- Cybercrime Generator
src/generate_dataset.py - Victim Complaints
src/generate_dataset.py - Rings and Mules
src/generate_dataset.py - Split-Converge Flows
src/generate_dataset.py - Cash-out Events
src/generate_dataset.py
Data Products
- Relational Case Data
data/generated - ML Dataset Builder
src/build_ml_datasets.py - ML Training Data
data/ml_ready - Anomaly Detection
data/ml_ready/anomaly_detection.csv - Terminal Node Model
data/ml_ready/terminal_node_training.csv - Channel Model
data/ml_ready/withdrawal_channel_training.csv - Location Model
data/ml_ready/withdrawal_location_training.csv - Withdrawal Time Model
data/ml_ready/withdrawal_time_training.csv
Validation
- Statistical Validator
src/validate_dataset.py - Validation Report
reports/validation_report.json
Other components
- Pipeline User
Connections 26
- Pipeline User → Pipeline Runner: runs
- Pipeline Runner → Parameter Calibration: invokes
- Pipeline Runner → Cybercrime Generator: invokes
- Pipeline Runner → ML Dataset Builder: invokes
- Pipeline Runner → Statistical Validator: invokes
- Parameter Calibration → Simulation Defaults: reads
- Parameter Calibration → Reference Data: reads
- Parameter Calibration → Calibrated Settings: writes
- Cybercrime Generator → Calibrated Settings: reads
- Cybercrime Generator → Reference Data: reads locations
- Cybercrime Generator → Victim Complaints: simulates
- Cybercrime Generator → Rings and Mules: constructs
- Cybercrime Generator → Split-Converge Flows: generates
- Cybercrime Generator → Cash-out Events: generates
- Cybercrime Generator → Relational Case Data: writes
- Relational Case Data → ML Dataset Builder: reads
- ML Dataset Builder → ML Training Data: writes
- ML Training Data → Anomaly Detection: includes
- ML Training Data → Terminal Node Model: includes
- ML Training Data → Channel Model: includes
- ML Training Data → Location Model: includes
- ML Training Data → Withdrawal Time Model: includes
- Statistical Validator → Relational Case Data: reads
- Statistical Validator → Calibrated Settings: reads
- Statistical Validator → Reference Data: checks against
- Statistical Validator → Validation Report: writes