How siphlygon/mphys-rlf works
DiffRACC is a research pipeline for estimating radio-source survey completeness and its effect on the radio luminosity function (RLF). Its principal workflow prepares LoTSS-DR2 cutouts, trains a diffusion model, generates flux-conditioned source images, estimates completeness, and applies corrections in RLF analysis. Additional capabilities include PyBDSF image/catalogue analysis, angular-size estimation, evaluation, and plotting. The README documents the overall workflow; source-index references support the indicated internal calls. Unsampled modules are represented only where their role is clear from the README or tree.
Main entry points: trainer.py Training orchestration · completeness_estimator.py Completeness estimation · apply_preprocessing.py Cutout preprocessing · sampler.py Conditional sampling
Components 31
Data preparation
- Cutout acquisition
diffracc/data/cutout_downloader.py - Cutout preprocessing
diffracc/data/apply_preprocessing.py - Cutout quality flags
diffracc/data/cutout_quality.py - Training dataset
diffracc/data/datasets.py - Flux transforms
diffracc/data/flux_transforms.py
Modeling and generation
- Training orchestration
diffracc/training/trainer.py - Diffusion network
diffracc/model/unet.py - Model utilities
diffracc/model/model_utils.py - Training outputs
diffracc/training/output_manager.py - Conditional sampling
diffracc/model/sampler.py - Generated FITS dataset
diffracc/sampling/generate_fits_files.py
Completeness analysis
- Completeness estimation
diffracc/completeness/completeness_estimator.py - Completeness fit I/O
diffracc/completeness/completeness_io.py - Size-binned completeness
diffracc/completeness/size_binned_completeness.py - Completeness plots
diffracc/completeness/plot_completeness.py
Science analysis
- PyBDSF analysis
diffracc/analysis/image_analyzer.py - AGN selection
diffracc/rlf/agn_selection.py - Radio luminosity function
diffracc/rlf/rlf.py - Image evaluation
diffracc/evaluation/evaluate.py - Angular size estimates
diffracc/las/angular_size_estimator.py
Shared infrastructure
- Scientific functions
diffracc/utils/functions.py - Image data arrays
diffracc/utils/img_data_arrays.py - Data utilities
diffracc/utils/data_utils.py - Path configuration
diffracc/utils/paths.py - Device utilities
diffracc/utils/device_utils.py - Distributed execution
diffracc/utils/distributed.py - Logging
diffracc/utils/logger.py - File scanning
diffracc/utils/recursive_file_analyzer.py - Power transforms
diffracc/utils/power_transform.py
Other components
- Researcher
- Survey catalogue
Connections 40
- Researcher → Cutout acquisition: requests cutouts
- Survey catalogue → Cutout acquisition: provides source data
- Cutout acquisition → Cutout preprocessing: prepares images
- Cutout preprocessing → Cutout quality flags: uses quality flags
diffracc/data/apply_preprocessing.py - Cutout quality flags → Training dataset: filters cutouts
- Cutout preprocessing → Training dataset: creates dataset
- Training orchestration → Training dataset: loads training data
diffracc/training/trainer.py - Training orchestration → Diffusion network: trains network
diffracc/training/trainer.py - Training orchestration → Model utilities: uses model utilities
diffracc/training/trainer.py - Training orchestration → Training outputs: records outputs
diffracc/training/trainer.py - Conditional sampling → Model utilities: loads model
diffracc/model/sampler.py - Conditional sampling → Flux transforms: inverts flux transform
diffracc/model/sampler.py - Generated FITS dataset → Conditional sampling: generates samples
diffracc/sampling/generate_fits_files.py - Generated FITS dataset → PyBDSF analysis: analyzes images
diffracc/sampling/generate_fits_files.py - Researcher → Completeness estimation: estimates completeness
- Generated FITS dataset → Completeness estimation: supplies mock images
- Completeness estimation → Completeness fit I/O: saves fitted parameters
diffracc/completeness/completeness_estimator.py - Completeness plots → Completeness estimation: plots estimates
diffracc/completeness/plot_completeness.py - Completeness plots → Size-binned completeness: uses size-binned results
diffracc/completeness/plot_completeness.py - Researcher → AGN selection: selects AGN
- AGN selection → Radio luminosity function: supplies AGN sample
- Radio luminosity function → Completeness fit I/O: reads completeness fits
diffracc/rlf/rlf.py - Radio luminosity function → Scientific functions: uses scientific functions
diffracc/rlf/rlf.py - Image evaluation → Generated FITS dataset: evaluates generated data
- Angular size estimates → PyBDSF analysis: uses source catalogues
diffracc/las/angular_size_estimator.py - Cutout preprocessing → Data utilities: uses data utilities
diffracc/data/apply_preprocessing.py - Cutout preprocessing → File scanning: scans cutout files
diffracc/data/apply_preprocessing.py - Completeness estimation → Image data arrays: uses image arrays
diffracc/completeness/completeness_estimator.py - Completeness estimation → Scientific functions: uses fit functions
diffracc/completeness/completeness_estimator.py - PyBDSF analysis → Distributed execution: partitions image work
diffracc/analysis/image_analyzer.py - PyBDSF analysis → File scanning: scans input files
diffracc/analysis/image_analyzer.py - Conditional sampling → Device utilities: selects device
diffracc/model/sampler.py - Conditional sampling → Scientific functions: uses shared utilities
- Radio luminosity function → Path configuration: uses configured paths
diffracc/rlf/rlf.py - Training orchestration → Device utilities: selects device
diffracc/training/trainer.py - Training orchestration → Path configuration: uses configured paths
diffracc/training/trainer.py - Training orchestration → Logging: logs training
diffracc/training/trainer.py - Completeness fit I/O → Scientific functions: uses shared functions
diffracc/completeness/completeness_io.py - Angular size estimates → File scanning: scans catalogue files
- Angular size estimates → Power transforms: uses plotting utilities