# rafapablos/dropstogrid architecture

> Architecture diagram of the GitHub repository rafapablos/dropstogrid, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rafapablos/dropstogrid
- Repository: https://github.com/rafapablos/dropstogrid

## Overview

DropsToGrid is a rainfall-densification model and training/evaluation pipeline: it combines temporal station observations with radar context to produce stochastic, dense rainfall estimates and uncertainty. The main workflow starts at the CLI, loads synthetic sample data and applies transforms, then trains or evaluates the model and reports metrics. Documented capabilities include multi-scale spatio-temporal fusion, uncertainty, callbacks, checkpointing, and ClearML logging. Several relationships are supported by the SOURCE INDEX rather than full excerpts; unsampled subsystems are represented only where the tree or README supports them.

Main entry points: [`dtg/modules/dtg/dtg.py`](https://github.com/rafapablos/dropstogrid/blob/main/dtg/modules/dtg/dtg.py) (DtG task module), [`dtg/cli.py`](https://github.com/rafapablos/dropstogrid/blob/main/dtg/cli.py) (CLI orchestration), [`dtg/callbacks`](https://github.com/rafapablos/dropstogrid/tree/main/dtg/callbacks) (Training callbacks), [`dtg/networks/dtg/model.py`](https://github.com/rafapablos/dropstogrid/blob/main/dtg/networks/dtg/model.py) (Fusion network)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflow["Run workflow"]
  node_cli["CLI orchestration<br/>[cli.py]"]
  node_trainer["Trainer<br/>[trainer.py]"]
  node_callbacks["Training callbacks"]
  node_ema["EMA callback<br/>[ema.py]"]
  node_weight_avg["Weight averaging"]
  node_checkpoint["Model checkpointing"]
end

subgraph group_data["Data preparation"]
  node_datamodule["Data module<br/>[datamodule.py]"]
  node_fake["Synthetic dataset<br/>[fake_dataset.py]"]
  node_sample["Sample schema<br/>[sample.py]"]
  node_sources["Source definitions<br/>[source.py]"]
  node_transforms["Sample transforms<br/>[transforms.py]"]
  node_transform_factory["Transform setup<br/>[factory.py]"]
end

subgraph group_model["Densification model"]
  node_module["DtG task module<br/>[dtg.py]"]
  node_context["Context handling<br/>[context.py]"]
  node_network["Fusion network<br/>[model.py]"]
  node_bottleneck["Temporal fusion<br/>[bottleneck.py]"]
  node_network_api["Network interface<br/>[network.py]"]
  node_distribution["Rainfall distribution<br/>[distributions.py]"]
end

subgraph group_evaluation["Evaluation and reporting"]
  node_loss["Training loss<br/>[loss.py]"]
  node_metrics["Evaluation metrics"]
  node_outputs["Evaluation outputs<br/>[output.py]"]
  node_animations["Animation logging<br/>[log_animations.py]"]
  node_logger["ClearML logger<br/>[clearml.py]"]
end

node_user(("Run operator"))
node_clearml["ClearML service"]

node_user -->|"starts run"| node_cli
node_cli -->|"configures"| node_datamodule
node_cli -->|"instantiates"| node_module
node_cli -->|"configures"| node_trainer
node_trainer -->|"builds"| node_callbacks
node_trainer -->|"builds"| node_logger
node_datamodule -->|"loads"| node_fake
node_datamodule -->|"requests transforms"| node_transform_factory
node_transform_factory -->|"assembles"| node_transforms
node_transforms -->|"transforms"| node_sample
node_fake -->|"produces"| node_sample
node_fake -->|"uses"| node_sources
node_module -->|"uses"| node_context
node_module -->|"calls"| node_network
node_module -->|"creates transforms"| node_transform_factory
node_network -->|"uses"| node_bottleneck
node_network -->|"implements config"| node_network_api
node_network -->|"constructs"| node_distribution
node_module -->|"computes likelihood"| node_distribution
node_module -->|"computes loss"| node_loss
node_module -->|"updates metrics"| node_metrics
node_module -->|"emits"| node_outputs
node_callbacks -.->|"includes"| node_ema
node_callbacks -.->|"includes"| node_weight_avg
node_callbacks -.->|"includes"| node_checkpoint
node_animations -->|"reads"| node_sample
node_animations -->|"reads"| node_outputs
node_animations -->|"logs through"| node_logger
node_logger -->|"reports metrics"| node_clearml

click node_cli "https://github.com/rafapablos/dropstogrid/blob/main/dtg/cli.py"
click node_trainer "https://github.com/rafapablos/dropstogrid/blob/main/dtg/trainer.py"
click node_datamodule "https://github.com/rafapablos/dropstogrid/blob/main/dtg/data/datamodule.py"
click node_fake "https://github.com/rafapablos/dropstogrid/blob/main/dtg/data/fake_dataset.py"
click node_sample "https://github.com/rafapablos/dropstogrid/blob/main/dtg/data/sample.py"
click node_sources "https://github.com/rafapablos/dropstogrid/blob/main/dtg/data/source.py"
click node_transforms "https://github.com/rafapablos/dropstogrid/blob/main/dtg/transforms/transforms.py"
click node_transform_factory "https://github.com/rafapablos/dropstogrid/blob/main/dtg/transforms/factory.py"
click node_module "https://github.com/rafapablos/dropstogrid/blob/main/dtg/modules/dtg/dtg.py"
click node_context "https://github.com/rafapablos/dropstogrid/blob/main/dtg/modules/dtg/context.py"
click node_network "https://github.com/rafapablos/dropstogrid/blob/main/dtg/networks/dtg/model.py"
click node_bottleneck "https://github.com/rafapablos/dropstogrid/blob/main/dtg/networks/dtg/bottleneck.py"
click node_network_api "https://github.com/rafapablos/dropstogrid/blob/main/dtg/networks/network.py"
click node_distribution "https://github.com/rafapablos/dropstogrid/blob/main/dtg/loss/distributions.py"
click node_loss "https://github.com/rafapablos/dropstogrid/blob/main/dtg/loss/loss.py"
click node_metrics "https://github.com/rafapablos/dropstogrid/tree/main/dtg/metrics"
click node_outputs "https://github.com/rafapablos/dropstogrid/blob/main/dtg/modules/output.py"
click node_callbacks "https://github.com/rafapablos/dropstogrid/tree/main/dtg/callbacks"
click node_ema "https://github.com/rafapablos/dropstogrid/blob/main/dtg/callbacks/ema.py"
click node_weight_avg "https://github.com/rafapablos/dropstogrid/blob/main/dtg/callbacks/weight_averaging.py"
click node_animations "https://github.com/rafapablos/dropstogrid/blob/main/dtg/callbacks/log_animations.py"
click node_logger "https://github.com/rafapablos/dropstogrid/blob/main/dtg/logger/clearml.py"
click node_checkpoint "https://github.com/rafapablos/dropstogrid/blob/main/dtg/callbacks/model_checkpoint.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_trainer,node_callbacks,node_ema,node_weight_avg,node_checkpoint toneBlue
class node_datamodule,node_fake,node_sample,node_sources,node_transforms,node_transform_factory toneAmber
class node_module,node_context,node_network,node_bottleneck,node_network_api,node_distribution toneMint
class node_loss,node_metrics,node_outputs,node_animations,node_logger toneRose
class node_user,node_clearml toneIndigo
```

## Components

### Run workflow

- [CLI orchestration](https://github.com/rafapablos/dropstogrid/blob/main/dtg/cli.py): `dtg/cli.py`
- [Trainer](https://github.com/rafapablos/dropstogrid/blob/main/dtg/trainer.py): `dtg/trainer.py`
- [Training callbacks](https://github.com/rafapablos/dropstogrid/tree/main/dtg/callbacks): `dtg/callbacks`
- [EMA callback](https://github.com/rafapablos/dropstogrid/blob/main/dtg/callbacks/ema.py): `dtg/callbacks/ema.py`
- [Weight averaging](https://github.com/rafapablos/dropstogrid/blob/main/dtg/callbacks/weight_averaging.py): `dtg/callbacks/weight_averaging.py`
- [Model checkpointing](https://github.com/rafapablos/dropstogrid/blob/main/dtg/callbacks/model_checkpoint.py): `dtg/callbacks/model_checkpoint.py`

### Data preparation

- [Data module](https://github.com/rafapablos/dropstogrid/blob/main/dtg/data/datamodule.py): `dtg/data/datamodule.py`
- [Synthetic dataset](https://github.com/rafapablos/dropstogrid/blob/main/dtg/data/fake_dataset.py): `dtg/data/fake_dataset.py`
- [Sample schema](https://github.com/rafapablos/dropstogrid/blob/main/dtg/data/sample.py): `dtg/data/sample.py`
- [Source definitions](https://github.com/rafapablos/dropstogrid/blob/main/dtg/data/source.py): `dtg/data/source.py`
- [Sample transforms](https://github.com/rafapablos/dropstogrid/blob/main/dtg/transforms/transforms.py): `dtg/transforms/transforms.py`
- [Transform setup](https://github.com/rafapablos/dropstogrid/blob/main/dtg/transforms/factory.py): `dtg/transforms/factory.py`

### Densification model

- [DtG task module](https://github.com/rafapablos/dropstogrid/blob/main/dtg/modules/dtg/dtg.py): `dtg/modules/dtg/dtg.py`
- [Context handling](https://github.com/rafapablos/dropstogrid/blob/main/dtg/modules/dtg/context.py): `dtg/modules/dtg/context.py`
- [Fusion network](https://github.com/rafapablos/dropstogrid/blob/main/dtg/networks/dtg/model.py): `dtg/networks/dtg/model.py`
- [Temporal fusion](https://github.com/rafapablos/dropstogrid/blob/main/dtg/networks/dtg/bottleneck.py): `dtg/networks/dtg/bottleneck.py`
- [Network interface](https://github.com/rafapablos/dropstogrid/blob/main/dtg/networks/network.py): `dtg/networks/network.py`
- [Rainfall distribution](https://github.com/rafapablos/dropstogrid/blob/main/dtg/loss/distributions.py): `dtg/loss/distributions.py`

### Evaluation and reporting

- [Training loss](https://github.com/rafapablos/dropstogrid/blob/main/dtg/loss/loss.py): `dtg/loss/loss.py`
- [Evaluation metrics](https://github.com/rafapablos/dropstogrid/tree/main/dtg/metrics): `dtg/metrics`
- [Evaluation outputs](https://github.com/rafapablos/dropstogrid/blob/main/dtg/modules/output.py): `dtg/modules/output.py`
- [Animation logging](https://github.com/rafapablos/dropstogrid/blob/main/dtg/callbacks/log_animations.py): `dtg/callbacks/log_animations.py`
- [ClearML logger](https://github.com/rafapablos/dropstogrid/blob/main/dtg/logger/clearml.py): `dtg/logger/clearml.py`

### Other components

- **Run operator**
- **ClearML service**

## Connections

- Run operator → CLI orchestration: starts run (evidence: `README.md`)
- CLI orchestration → Data module: configures (evidence: `dtg/cli.py`)
- CLI orchestration → DtG task module: instantiates (evidence: `dtg/cli.py`)
- CLI orchestration → Trainer: configures (evidence: `dtg/cli.py`)
- Trainer → Training callbacks: builds (evidence: `dtg/trainer.py`)
- Trainer → ClearML logger: builds (evidence: `dtg/trainer.py`)
- Data module → Synthetic dataset: loads (evidence: `dtg/data/datamodule.py`)
- Data module → Transform setup: requests transforms (evidence: `dtg/data/datamodule.py`)
- Transform setup → Sample transforms: assembles (evidence: `dtg/transforms/factory.py`)
- Sample transforms → Sample schema: transforms (evidence: `dtg/transforms/transforms.py`)
- Synthetic dataset → Sample schema: produces (evidence: `dtg/data/fake_dataset.py`)
- Synthetic dataset → Source definitions: uses (evidence: `dtg/data/fake_dataset.py`)
- DtG task module → Context handling: uses (evidence: `dtg/modules/dtg/dtg.py`)
- DtG task module → Fusion network: calls (evidence: `dtg/modules/dtg/dtg.py`)
- DtG task module → Transform setup: creates transforms (evidence: `dtg/modules/dtg/dtg.py`)
- Fusion network → Temporal fusion: uses (evidence: `dtg/networks/dtg/model.py`)
- Fusion network → Network interface: implements config (evidence: `dtg/networks/dtg/model.py`)
- Fusion network → Rainfall distribution: constructs (evidence: `dtg/networks/dtg/model.py`)
- DtG task module → Rainfall distribution: computes likelihood (evidence: `dtg/modules/dtg/dtg.py`)
- DtG task module → Training loss: computes loss (evidence: `dtg/modules/dtg/dtg.py`)
- DtG task module → Evaluation metrics: updates metrics (evidence: `dtg/modules/dtg/dtg.py`)
- DtG task module → Evaluation outputs: emits (evidence: `dtg/modules/dtg/dtg.py`)
- Training callbacks → EMA callback: includes (evidence: `dtg/callbacks/factory.py`)
- Training callbacks → Weight averaging: includes (evidence: `dtg/callbacks/ema.py`)
- Training callbacks → Model checkpointing: includes (evidence: `dtg/callbacks/factory.py`)
- Animation logging → Sample schema: reads (evidence: `dtg/callbacks/log_animations.py`)
- Animation logging → Evaluation outputs: reads (evidence: `dtg/callbacks/log_animations.py`)
- Animation logging → ClearML logger: logs through (evidence: `dtg/callbacks/log_animations.py`)
- ClearML logger → ClearML service: reports metrics (evidence: `dtg/logger/clearml.py`)
