# adilmunawar/hrnet architecture

> Architecture diagram of the GitHub repository adilmunawar/hrnet, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/adilmunawar/hrnet
- Repository: https://github.com/adilmunawar/hrnet

## Overview

1. **What this repo is**
This is a geospatial parcel-boundary extraction pipeline, not a general ML app. The core model is an HRNet-W48 encoder inside a U-Net segmentation network, trained to predict **parcel boundary pixels** from satellite imagery, then post-processed into parcel polygons.

2. **Overall data flow**
The pipeline is:
- **Input vector parcels** in shapefiles or AOI GeoJSON
- **Download satellite tiles** from Google Maps tile endpoints
- **Rasterize geometry** into boundary masks
- **Train / run segmentation model**
- **Post-process binary boundary raster**
- **Export parcel polygons** as GIS deliverables

3. **Dataset creation subsystem**
`Create_dataset.py` is the most important preprocessing stage. It reads a ZIP of shapefiles, projects geometries, computes tile coverage at zoom 19, downloads imagery tiles, mosaics them into GeoTIFFs, rasterizes parcel boundaries into masks, and then cuts everything into overlapping 512×512 patches. It also filters bad patches such as mostly black tiles, uniform tiles, NaN/Inf imagery, and low-signal empty backgrounds.

4. **Geospatial boundary generation**
The dataset builder is working at two coordinate-system boundaries:
- **Vector space**: shapefiles in a default projected CRS, converted to WGS84 and Web Mercator as needed
- **Raster space**: a Google satellite mosaic in EPSG:3857, with boundary masks rasterized from polygon boundaries  
This is the key architectural boundary in the repo: vector parcel geometry becomes pixel-level supervision for segmentation.

5. **Training subsystem**
`Training.py` loads the patch dataset from `/content/dataset`, applies Albumentations augmentation, and trains `smp.Unet(encoder_name="tu-hrnet_w48")` with a binary output head. It uses mixed precision, gradient accumulation, gradient clipping, and a composite loss that mixes BCE, Dice, Focal, edge loss, and continuity loss.

6. **Training/validation split strategy**
The dataset is split into train/val/test during patch generation, then training samples are sub-sampled per epoch to a fixed budget (`SAMPLES_PER_EPOCH`). Validation uses a fixed random subset of the validation split. The architecture therefore assumes a large tile corpus and keeps training throughput manageable by sampling rather than loading everything each epoch.

7. **Checkpoint and model registry boundary**
Training checkpoints are saved locally under `/content/checkpoints` and asynchronously uploaded to Hugging Face Hub (`AdilMunawar/Model15`). The repo uses Hugging Face as the persistence layer for `best.pth` and `last.pth`, so the model state is externalized from the runtime environment. `HF_Check.py` is likely a utility around that boundary.

8. **Inference subsystem**
`Inference.py` is the deployment-time counterpart to training. It takes an AOI GeoJSON/JSON, downloads the satellite mosaic covering that AOI, loads the best checkpoint either from local disk or Hugging Face, runs sliding-window inference over the mosaic, and writes:
- a floating-point probability raster
- a binary mask raster
- a visual preview image

9. **Inference tiling and blending**
Inference uses overlapping 512×512 patches with a 256-pixel overlap and blends them with a Hann window. That matters architecturally because the output raster is not stitched naïvely; patch-edge artifacts are intentionally softened before thresholding or hysteresis binarization. This is the main image-composition boundary in the inference flow.

10. **Post-processing subsystem**
The post-processing block converts the predicted binary boundary raster into parcel polygons. It removes small boundary objects, fills holes, skeletonizes boundaries, inverts to parcel interiors, polygonizes the raster, heals geometries, simplifies them, filters by area/shape constraints, and optionally resolves overlaps.

11. **Final GIS outputs**
Post-processing exports multiple geospatial deliverables from the final polygon set:
- GeoPackage
- GeoJSON
- zipped Shapefile
- Excel statistics report  
This is the final boundary where raster ML outputs become consumable GIS data.

12. **Support scripts**
The remaining files appear to be utility/QA helpers around the same pipeline:
- `Preview_Labels.py` likely visualizes image/mask pairs for dataset inspection
- `Fix_Broken_Lines.py` likely repairs invalid or fragmented label geometries before dataset creation
- `Cell1-Dependancies.py` looks like an environment/bootstrap notebook cell for Colab-style execution
These are not separate systems; they support dataset hygiene and reproducible execution.

13. **Runtime and environment shape**
The repo is structured like a notebook-driven Colab workflow using `/content/...` paths, GPU training, and heavy geospatial Python libraries such as GeoPandas, Rasterio, Shapely, Mercantile, OpenCV, PyTorch, and `segmentation_models_pytorch`. External dependencies that materially affect the architecture are Google’s tile service for imagery and Hugging Face Hub for model checkpoint storage.

14. **How to draw the diagram**
A good architecture diagram should show three big stages:
- **Preparation**: shapefile ZIP → tile download/mosaic → raster masks → patch dataset
- **Learning/Serving**: patch dataset → HRNet-U-Net training → HF checkpoint storage → checkpoint reload for inference
- **Conversion to GIS**: AOI GeoJSON → inference probability/binary rasters → morphological/vector post-processing → parcel exports  
The most important data boundary is the conversion from vector parcel geometry to raster boundary supervision, and then back from raster predictions to vector parcel polygons.

Main entry points: [`Create_dataset.py`](https://github.com/adilmunawar/hrnet/blob/main/Create_dataset.py) (Patch maker), [`Inference.py`](https://github.com/adilmunawar/hrnet/blob/main/Inference.py) (AOI loader), [`Cell1-Dependancies.py`](https://github.com/adilmunawar/hrnet/blob/main/Cell1-Dependancies.py) (Bootstrap env), [`Training.py`](https://github.com/adilmunawar/hrnet/blob/main/Training.py) (Augment)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_prep["Preparation"]
  node_input_vec["Parcel input<br/>vector data"]
  node_tile_dl["Tile fetch<br/>imagery ingest<br/>[Create_dataset.py]"]
  node_mosaic[("Geo mosaic<br/>raster assembly<br/>[Create_dataset.py]")]
  node_raster_mask["Boundary mask<br/>rasterization<br/>[Create_dataset.py]"]
  node_patch_gen["Patch maker<br/>dataset builder<br/>[Create_dataset.py]"]
end

subgraph group_learn["Learning + Serving"]
  node_train_split["Dataset split<br/>sampling split<br/>[Create_dataset.py]"]
  node_augment["Augment<br/>[Training.py]"]
  node_model["HRNet U-Net<br/>segmentation model<br/>[Training.py]"]
  node_loss_opt["Loss + opt<br/>training loop<br/>[Training.py]"]
  node_ckpt_store[("HF checkpoints<br/>model registry<br/>[HF_Check.py]")]
end

subgraph group_gis["Raster to GIS"]
  node_aoi_infer["AOI loader<br/>inference input<br/>[Inference.py]"]
  node_slide_blend["Sliding window<br/>tiling inference<br/>[Inference.py]"]
  node_prob_mask["Pred raster<br/>prediction outputs<br/>[Inference.py]"]
  node_postproc["Polygonize<br/>vector postprocess<br/>[Inference.py]"]
  node_gis_export["GIS exports<br/>deliverables<br/>[Inference.py]"]
end

subgraph group_support["Support"]
  node_preview["Preview labels<br/>qa helper<br/>[Preview_Labels.py]"]
  node_repair["Fix lines<br/>geometry repair"]
  node_bootstrap["Bootstrap env<br/>runtime setup"]
end

node_input_vec -->|"defines extent"| node_tile_dl
node_tile_dl -->|"assemble"| node_mosaic
node_input_vec -->|"rasterize"| node_raster_mask
node_mosaic -->|"cut patches"| node_patch_gen
node_raster_mask -->|"align labels"| node_patch_gen
node_patch_gen -->|"split"| node_train_split
node_train_split -->|"feed"| node_augment
node_augment -->|"train"| node_model
node_model -->|"optimize"| node_loss_opt
node_loss_opt -->|"save"| node_ckpt_store
node_ckpt_store -->|"reload"| node_aoi_infer
node_aoi_infer -->|"tile AOI"| node_slide_blend
node_slide_blend -->|"stitch"| node_prob_mask
node_prob_mask -->|"convert"| node_postproc
node_postproc -->|"export"| node_gis_export
node_repair -->|"clean"| node_input_vec
node_preview -->|"inspect"| node_patch_gen
node_bootstrap -->|"enable"| node_train_split
node_bootstrap -->|"enable"| node_aoi_infer

click node_tile_dl "https://github.com/adilmunawar/hrnet/blob/main/Create_dataset.py"
click node_mosaic "https://github.com/adilmunawar/hrnet/blob/main/Create_dataset.py"
click node_raster_mask "https://github.com/adilmunawar/hrnet/blob/main/Create_dataset.py"
click node_patch_gen "https://github.com/adilmunawar/hrnet/blob/main/Create_dataset.py"
click node_train_split "https://github.com/adilmunawar/hrnet/blob/main/Create_dataset.py"
click node_augment "https://github.com/adilmunawar/hrnet/blob/main/Training.py"
click node_model "https://github.com/adilmunawar/hrnet/blob/main/Training.py"
click node_loss_opt "https://github.com/adilmunawar/hrnet/blob/main/Training.py"
click node_ckpt_store "https://github.com/adilmunawar/hrnet/blob/main/HF_Check.py"
click node_aoi_infer "https://github.com/adilmunawar/hrnet/blob/main/Inference.py"
click node_slide_blend "https://github.com/adilmunawar/hrnet/blob/main/Inference.py"
click node_prob_mask "https://github.com/adilmunawar/hrnet/blob/main/Inference.py"
click node_postproc "https://github.com/adilmunawar/hrnet/blob/main/Inference.py"
click node_gis_export "https://github.com/adilmunawar/hrnet/blob/main/Inference.py"
click node_preview "https://github.com/adilmunawar/hrnet/blob/main/Preview_Labels.py"
click node_repair "https://github.com/adilmunawar/hrnet/blob/main/Fix_Broken_Lines.py"
click node_bootstrap "https://github.com/adilmunawar/hrnet/blob/main/Cell1-Dependancies.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_input_vec,node_tile_dl,node_mosaic,node_raster_mask,node_patch_gen toneBlue
class node_train_split,node_augment,node_model,node_loss_opt,node_ckpt_store toneAmber
class node_aoi_infer,node_slide_blend,node_prob_mask,node_postproc,node_gis_export toneMint
class node_preview,node_repair,node_bootstrap toneRose
```

## Components

### Preparation

Vector parcels are turned into raster training patches.

- **Parcel input**: vector data · Shapefile ZIPs or AOI GeoJSON/JSON used as parcel sources.
- [Tile fetch](https://github.com/adilmunawar/hrnet/blob/main/Create_dataset.py): imagery ingest · Downloads Google satellite tiles covering parcel extents or AOIs. · `Create_dataset.py`
- [Geo mosaic](https://github.com/adilmunawar/hrnet/blob/main/Create_dataset.py): raster assembly · Merges downloaded tiles into georeferenced satellite mosaics. · `Create_dataset.py`
- [Boundary mask](https://github.com/adilmunawar/hrnet/blob/main/Create_dataset.py): rasterization · Projects parcel boundaries into EPSG:3857 pixel masks for supervision. · `Create_dataset.py`
- [Patch maker](https://github.com/adilmunawar/hrnet/blob/main/Create_dataset.py): dataset builder · Cuts mosaics and masks into overlapping 512×512 patches and filters bad samples. · `Create_dataset.py`

### Learning + Serving

The segmentation model is trained, checkpointed, and reused for inference.

- [Dataset split](https://github.com/adilmunawar/hrnet/blob/main/Create_dataset.py): sampling split · Creates train/val/test patch splits and fixed per-epoch sample budgets. · `Create_dataset.py`
- [Augment](https://github.com/adilmunawar/hrnet/blob/main/Training.py): augmentation · Applies Albumentations transforms during training. · `Training.py`
- [HRNet U-Net](https://github.com/adilmunawar/hrnet/blob/main/Training.py): segmentation model · Binary segmentation network built with tu-hrnet_w48 encoder. · `Training.py`
- [Loss + opt](https://github.com/adilmunawar/hrnet/blob/main/Training.py): training loop · Combines BCE, Dice, Focal, edge, and continuity losses with AMP, clipping, and accumulation. · `Training.py`
- [HF checkpoints](https://github.com/adilmunawar/hrnet/blob/main/HF_Check.py): model registry · Persists best and last weights locally and uploads them to Hugging Face Hub. · `HF_Check.py`

### Raster to GIS

Predictions are converted back into parcel polygons and deliverables.

- [AOI loader](https://github.com/adilmunawar/hrnet/blob/main/Inference.py): inference input · Loads AOI GeoJSON/JSON and prepares the coverage region for deployment inference. · `Inference.py`
- [Sliding window](https://github.com/adilmunawar/hrnet/blob/main/Inference.py): tiling inference · Runs overlapping 512×512 inference patches and blends them with a Hann window. · `Inference.py`
- [Pred raster](https://github.com/adilmunawar/hrnet/blob/main/Inference.py): prediction outputs · Produces probability rasters, binary masks, and preview imagery. · `Inference.py`
- [Polygonize](https://github.com/adilmunawar/hrnet/blob/main/Inference.py): vector postprocess · Cleans boundary rasters, fills holes, skeletonizes, heals, simplifies, and converts to parcel polygons. · `Inference.py`
- [GIS exports](https://github.com/adilmunawar/hrnet/blob/main/Inference.py): deliverables · Writes GeoPackage, GeoJSON, zipped Shapefile, and Excel statistics reports. · `Inference.py`

### Support

Utility scripts for setup, QA, and geometry repair.

- [Preview labels](https://github.com/adilmunawar/hrnet/blob/main/Preview_Labels.py): qa helper · Visualizes image/mask pairs for dataset inspection. · `Preview_Labels.py`
- [Fix lines](https://github.com/adilmunawar/hrnet/blob/main/Fix_Broken_Lines.py): geometry repair · Repairs broken or fragmented label geometries before dataset creation. · `Fix_Broken_Lines.py`
- [Bootstrap env](https://github.com/adilmunawar/hrnet/blob/main/Cell1-Dependancies.py): runtime setup · Colab-style dependency and environment setup cell. · `Cell1-Dependancies.py`

## Connections

- Parcel input → Tile fetch: defines extent · Parcel geometry or AOI bounds determine what imagery to fetch.
- Tile fetch → Geo mosaic: assemble · Downloaded tiles are stitched into georeferenced mosaics.
- Parcel input → Boundary mask: rasterize · Vector parcel boundaries become pixel supervision masks.
- Geo mosaic → Patch maker: cut patches · Mosaics are split into overlapping training tiles.
- Boundary mask → Patch maker: align labels · Masks are cut with the same patch geometry as imagery.
- Patch maker → Dataset split: split · Patch corpus is divided into train, validation, and test subsets.
- Dataset split → Augment: feed · Training subsets are augmented on the fly.
- Augment → HRNet U-Net: train · Augmented patches are consumed by the segmentation network.
- HRNet U-Net → Loss + opt: optimize · Forward passes are optimized with composite losses and mixed precision.
- Loss + opt → HF checkpoints: save · Best and last checkpoints are written and synced to Hugging Face.
- HF checkpoints → AOI loader: reload · Inference can restore the best checkpoint from local disk or the hub.
- AOI loader → Sliding window: tile AOI · The AOI mosaic is processed in overlapping inference windows.
- Sliding window → Pred raster: stitch · Blended patch predictions form probability and binary rasters.
- Pred raster → Polygonize: convert · Binary boundary rasters are turned into parcel polygons.
- Polygonize → GIS exports: export · Final parcel geometries are written to GIS deliverables.
- Fix lines → Parcel input: clean · Broken line repair improves geometry quality before dataset creation.
- Preview labels → Patch maker: inspect · Previewing samples helps validate dataset quality and masks.
- Bootstrap env → Dataset split: enable · Environment setup supports the notebook-style training workflow.
- Bootstrap env → AOI loader: enable · Runtime setup also supports deployment-time inference.