# siphlygon/mphys-rlf architecture

> Architecture diagram of the GitHub repository siphlygon/mphys-rlf, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/siphlygon/mphys-rlf
- Repository: https://github.com/siphlygon/mphys-rlf

## Overview

DiffRACC is a research pipeline for estimating radio-source survey completeness and its effect on the radio luminosity function (RLF). Its principal workflow prepares LoTSS-DR2 cutouts, trains a diffusion model, generates flux-conditioned source images, estimates completeness, and applies corrections in RLF analysis. Additional capabilities include PyBDSF image/catalogue analysis, angular-size estimation, evaluation, and plotting. The README documents the overall workflow; source-index references support the indicated internal calls. Unsampled modules are represented only where their role is clear from the README or tree.

Main entry points: [`diffracc/training/trainer.py`](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/training/trainer.py) (Training orchestration), [`diffracc/completeness/completeness_estimator.py`](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/completeness/completeness_estimator.py) (Completeness estimation), [`diffracc/data/apply_preprocessing.py`](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/data/apply_preprocessing.py) (Cutout preprocessing), [`diffracc/model/sampler.py`](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/model/sampler.py) (Conditional sampling)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_cutouts["Cutout acquisition"]
  node_preprocess["Cutout preprocessing"]
  node_quality["Cutout quality flags<br/>[cutout_quality.py]"]
  node_dataset[("Training dataset<br/>[datasets.py]")]
  node_flux["Flux transforms<br/>[flux_transforms.py]"]
end

subgraph group_modeling["Modeling and generation"]
  node_trainer["Training orchestration<br/>[trainer.py]"]
  node_unet["Diffusion network<br/>[unet.py]"]
  node_modelutils["Model utilities<br/>[model_utils.py]"]
  node_outputs["Training outputs<br/>[output_manager.py]"]
  node_sampler["Conditional sampling<br/>[sampler.py]"]
  node_generated["Generated FITS dataset"]
end

subgraph group_completeness["Completeness analysis"]
  node_completeness["Completeness estimation"]
  node_compio["Completeness fit I/O<br/>[completeness_io.py]"]
  node_sizecomp["Size-binned completeness"]
  node_compplots["Completeness plots"]
end

subgraph group_science["Science analysis"]
  node_imageanalysis["PyBDSF analysis<br/>[image_analyzer.py]"]
  node_agn["AGN selection<br/>[agn_selection.py]"]
  node_rlf["Radio luminosity function<br/>[rlf.py]"]
  node_evaluation["Image evaluation<br/>[evaluate.py]"]
  node_sizes["Angular size estimates"]
end

subgraph group_shared["Shared infrastructure"]
  node_functions["Scientific functions<br/>[functions.py]"]
  node_imgarrays["Image data arrays<br/>[img_data_arrays.py]"]
  node_datautils["Data utilities<br/>[data_utils.py]"]
  node_paths["Path configuration<br/>[paths.py]"]
  node_device["Device utilities<br/>[device_utils.py]"]
  node_distributed["Distributed execution<br/>[distributed.py]"]
  node_logger["Logging<br/>[logger.py]"]
  node_recursive["File scanning"]
  node_power["Power transforms<br/>[power_transform.py]"]
end

node_researcher(("Researcher"))
node_catalogue[("Survey catalogue")]

node_researcher -->|"requests cutouts"| node_cutouts
node_catalogue -->|"provides source data"| node_cutouts
node_cutouts -->|"prepares images"| node_preprocess
node_preprocess -->|"uses quality flags"| node_quality
node_quality -->|"filters cutouts"| node_dataset
node_preprocess -->|"creates dataset"| node_dataset
node_trainer -->|"loads training data"| node_dataset
node_trainer -->|"trains network"| node_unet
node_trainer -->|"uses model utilities"| node_modelutils
node_trainer -->|"records outputs"| node_outputs
node_sampler -->|"loads model"| node_modelutils
node_sampler -->|"inverts flux transform"| node_flux
node_generated -->|"generates samples"| node_sampler
node_generated -->|"analyzes images"| node_imageanalysis
node_researcher -->|"estimates completeness"| node_completeness
node_generated -->|"supplies mock images"| node_completeness
node_completeness -->|"saves fitted parameters"| node_compio
node_compplots -->|"plots estimates"| node_completeness
node_compplots -->|"uses size-binned results"| node_sizecomp
node_researcher -->|"selects AGN"| node_agn
node_agn -.->|"supplies AGN sample"| node_rlf
node_rlf -->|"reads completeness fits"| node_compio
node_rlf -->|"uses scientific functions"| node_functions
node_evaluation -.->|"evaluates generated data"| node_generated
node_sizes -->|"uses source catalogues"| node_imageanalysis
node_preprocess -->|"uses data utilities"| node_datautils
node_preprocess -->|"scans cutout files"| node_recursive
node_completeness -->|"uses image arrays"| node_imgarrays
node_completeness -->|"uses fit functions"| node_functions
node_imageanalysis -->|"partitions image work"| node_distributed
node_imageanalysis -->|"scans input files"| node_recursive
node_sampler -->|"selects device"| node_device
node_sampler -.->|"uses shared utilities"| node_functions
node_rlf -->|"uses configured paths"| node_paths
node_trainer -->|"selects device"| node_device
node_trainer -->|"uses configured paths"| node_paths
node_trainer -->|"logs training"| node_logger
node_compio -->|"uses shared functions"| node_functions
node_sizes -->|"scans catalogue files"| node_recursive
node_sizes -.->|"uses plotting utilities"| node_power

click node_cutouts "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/data/cutout_downloader.py"
click node_preprocess "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/data/apply_preprocessing.py"
click node_quality "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/data/cutout_quality.py"
click node_dataset "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/data/datasets.py"
click node_flux "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/data/flux_transforms.py"
click node_trainer "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/training/trainer.py"
click node_unet "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/model/unet.py"
click node_modelutils "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/model/model_utils.py"
click node_outputs "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/training/output_manager.py"
click node_sampler "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/model/sampler.py"
click node_generated "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/sampling/generate_fits_files.py"
click node_imageanalysis "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/analysis/image_analyzer.py"
click node_completeness "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/completeness/completeness_estimator.py"
click node_compio "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/completeness/completeness_io.py"
click node_sizecomp "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/completeness/size_binned_completeness.py"
click node_compplots "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/completeness/plot_completeness.py"
click node_agn "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/rlf/agn_selection.py"
click node_rlf "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/rlf/rlf.py"
click node_evaluation "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/evaluation/evaluate.py"
click node_sizes "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/las/angular_size_estimator.py"
click node_functions "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/utils/functions.py"
click node_imgarrays "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/utils/img_data_arrays.py"
click node_datautils "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/utils/data_utils.py"
click node_paths "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/utils/paths.py"
click node_device "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/utils/device_utils.py"
click node_distributed "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/utils/distributed.py"
click node_logger "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/utils/logger.py"
click node_recursive "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/utils/recursive_file_analyzer.py"
click node_power "https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/utils/power_transform.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cutouts,node_preprocess,node_quality,node_dataset,node_flux toneBlue
class node_trainer,node_unet,node_modelutils,node_outputs,node_sampler,node_generated,node_catalogue toneAmber
class node_completeness,node_compio,node_sizecomp,node_compplots toneMint
class node_imageanalysis,node_agn,node_rlf,node_evaluation,node_sizes toneRose
class node_functions,node_imgarrays,node_datautils,node_paths,node_device,node_distributed,node_logger,node_recursive,node_power,node_researcher toneIndigo
```

## Components

### Data preparation

- [Cutout acquisition](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/data/cutout_downloader.py): `diffracc/data/cutout_downloader.py`
- [Cutout preprocessing](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/data/apply_preprocessing.py): `diffracc/data/apply_preprocessing.py`
- [Cutout quality flags](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/data/cutout_quality.py): `diffracc/data/cutout_quality.py`
- [Training dataset](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/data/datasets.py): `diffracc/data/datasets.py`
- [Flux transforms](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/data/flux_transforms.py): `diffracc/data/flux_transforms.py`

### Modeling and generation

- [Training orchestration](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/training/trainer.py): `diffracc/training/trainer.py`
- [Diffusion network](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/model/unet.py): `diffracc/model/unet.py`
- [Model utilities](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/model/model_utils.py): `diffracc/model/model_utils.py`
- [Training outputs](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/training/output_manager.py): `diffracc/training/output_manager.py`
- [Conditional sampling](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/model/sampler.py): `diffracc/model/sampler.py`
- [Generated FITS dataset](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/sampling/generate_fits_files.py): `diffracc/sampling/generate_fits_files.py`

### Completeness analysis

- [Completeness estimation](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/completeness/completeness_estimator.py): `diffracc/completeness/completeness_estimator.py`
- [Completeness fit I/O](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/completeness/completeness_io.py): `diffracc/completeness/completeness_io.py`
- [Size-binned completeness](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/completeness/size_binned_completeness.py): `diffracc/completeness/size_binned_completeness.py`
- [Completeness plots](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/completeness/plot_completeness.py): `diffracc/completeness/plot_completeness.py`

### Science analysis

- [PyBDSF analysis](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/analysis/image_analyzer.py): `diffracc/analysis/image_analyzer.py`
- [AGN selection](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/rlf/agn_selection.py): `diffracc/rlf/agn_selection.py`
- [Radio luminosity function](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/rlf/rlf.py): `diffracc/rlf/rlf.py`
- [Image evaluation](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/evaluation/evaluate.py): `diffracc/evaluation/evaluate.py`
- [Angular size estimates](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/las/angular_size_estimator.py): `diffracc/las/angular_size_estimator.py`

### Shared infrastructure

- [Scientific functions](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/utils/functions.py): `diffracc/utils/functions.py`
- [Image data arrays](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/utils/img_data_arrays.py): `diffracc/utils/img_data_arrays.py`
- [Data utilities](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/utils/data_utils.py): `diffracc/utils/data_utils.py`
- [Path configuration](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/utils/paths.py): `diffracc/utils/paths.py`
- [Device utilities](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/utils/device_utils.py): `diffracc/utils/device_utils.py`
- [Distributed execution](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/utils/distributed.py): `diffracc/utils/distributed.py`
- [Logging](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/utils/logger.py): `diffracc/utils/logger.py`
- [File scanning](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/utils/recursive_file_analyzer.py): `diffracc/utils/recursive_file_analyzer.py`
- [Power transforms](https://github.com/siphlygon/mphys-rlf/blob/master/diffracc/utils/power_transform.py): `diffracc/utils/power_transform.py`

### Other components

- **Researcher**
- **Survey catalogue**

## Connections

- Researcher → Cutout acquisition: requests cutouts
- Survey catalogue → Cutout acquisition: provides source data
- Cutout acquisition → Cutout preprocessing: prepares images
- Cutout preprocessing → Cutout quality flags: uses quality flags (evidence: `diffracc/data/apply_preprocessing.py`)
- Cutout quality flags → Training dataset: filters cutouts
- Cutout preprocessing → Training dataset: creates dataset
- Training orchestration → Training dataset: loads training data (evidence: `diffracc/training/trainer.py`)
- Training orchestration → Diffusion network: trains network (evidence: `diffracc/training/trainer.py`)
- Training orchestration → Model utilities: uses model utilities (evidence: `diffracc/training/trainer.py`)
- Training orchestration → Training outputs: records outputs (evidence: `diffracc/training/trainer.py`)
- Conditional sampling → Model utilities: loads model (evidence: `diffracc/model/sampler.py`)
- Conditional sampling → Flux transforms: inverts flux transform (evidence: `diffracc/model/sampler.py`)
- Generated FITS dataset → Conditional sampling: generates samples (evidence: `diffracc/sampling/generate_fits_files.py`)
- Generated FITS dataset → PyBDSF analysis: analyzes images (evidence: `diffracc/sampling/generate_fits_files.py`)
- Researcher → Completeness estimation: estimates completeness
- Generated FITS dataset → Completeness estimation: supplies mock images
- Completeness estimation → Completeness fit I/O: saves fitted parameters (evidence: `diffracc/completeness/completeness_estimator.py`)
- Completeness plots → Completeness estimation: plots estimates (evidence: `diffracc/completeness/plot_completeness.py`)
- Completeness plots → Size-binned completeness: uses size-binned results (evidence: `diffracc/completeness/plot_completeness.py`)
- Researcher → AGN selection: selects AGN
- AGN selection → Radio luminosity function: supplies AGN sample
- Radio luminosity function → Completeness fit I/O: reads completeness fits (evidence: `diffracc/rlf/rlf.py`)
- Radio luminosity function → Scientific functions: uses scientific functions (evidence: `diffracc/rlf/rlf.py`)
- Image evaluation → Generated FITS dataset: evaluates generated data
- Angular size estimates → PyBDSF analysis: uses source catalogues (evidence: `diffracc/las/angular_size_estimator.py`)
- Cutout preprocessing → Data utilities: uses data utilities (evidence: `diffracc/data/apply_preprocessing.py`)
- Cutout preprocessing → File scanning: scans cutout files (evidence: `diffracc/data/apply_preprocessing.py`)
- Completeness estimation → Image data arrays: uses image arrays (evidence: `diffracc/completeness/completeness_estimator.py`)
- Completeness estimation → Scientific functions: uses fit functions (evidence: `diffracc/completeness/completeness_estimator.py`)
- PyBDSF analysis → Distributed execution: partitions image work (evidence: `diffracc/analysis/image_analyzer.py`)
- PyBDSF analysis → File scanning: scans input files (evidence: `diffracc/analysis/image_analyzer.py`)
- Conditional sampling → Device utilities: selects device (evidence: `diffracc/model/sampler.py`)
- Conditional sampling → Scientific functions: uses shared utilities
- Radio luminosity function → Path configuration: uses configured paths (evidence: `diffracc/rlf/rlf.py`)
- Training orchestration → Device utilities: selects device (evidence: `diffracc/training/trainer.py`)
- Training orchestration → Path configuration: uses configured paths (evidence: `diffracc/training/trainer.py`)
- Training orchestration → Logging: logs training (evidence: `diffracc/training/trainer.py`)
- Completeness fit I/O → Scientific functions: uses shared functions (evidence: `diffracc/completeness/completeness_io.py`)
- Angular size estimates → File scanning: scans catalogue files
- Angular size estimates → Power transforms: uses plotting utilities
