# koleshjr/inundata-mapping-floods-in-south-africa architecture

> Architecture diagram of the GitHub repository koleshjr/inundata-mapping-floods-in-south-africa, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/koleshjr/inundata-mapping-floods-in-south-africa
- Repository: https://github.com/koleshjr/inundata-mapping-floods-in-south-africa

## Overview

##### Main Components
- Data Ingestion & EDA: Represent the initial stage that handles importing composite satellite images, performing Exploratory Data Analysis, and preparing the data. This corresponds to the “first-stage-static-images-eda-data-preparation.ipynb” notebook.
- Image Classification Module: Illustrate the standalone component that uses moisture stress imagery to train an image classifier (using the “eva02_tiny_patch14_224” model) to estimate flood probabilities. This is detailed in “second-stage-image-classifier.ipynb.”
- Modeling Suite: This module should be composite of several subcomponents:
– Various models such as XGBoost, LightGBM, FastAI Tabular, FastAI GatedConv, FastAI 1DConv, FastAI TabTransformer, and TabNet.
– Deep learning models for sequence or time series data like Wavenet-GRU and ResNet1D, with separate training and inference notebooks.
Each of these 9 model approaches can be shown as parallel or a grouped block, indicating they leverage the outputs from the Image Classification Module and additional features (e.g., lagged precipitation, EWMA, etc.).
- Ensembling: Represent the final stage where predictions from all the models are combined using Nelder-Mead optimization and flood probability normalization to achieve the final output. This corresponds to “fourth-stage-models-ensemble.ipynb.”

##### Relationships and Interactions
- Finally, illustrate the aggregation/ensemble block that takes predictions from all the models and outputs the final, normalized prediction.

##### Architectural Patterns and Design Principles
- Pipeline/Multi-Stage Architecture: Emphasize the sequential nature of the workflow.
- Modularity: Each notebook (and thus each stage of the pipeline) acts as a module that is invoked in sequence. This modular design makes it easier to update or replace any stage.
- Parallel Modeling: Within the modeling stage, note the simultaneous training of several distinct models, supporting ensemble learning.
- End-to-End Processing: The design supports an end-to-end process from data preparation to ensemble predictions, ensuring traceable and reproducible experimentation.

##### Technologies and Environment Details
- Explicitly mention that experiments are tailored for execution on Kaggle (or Colab with path modifications), so the environment should be labeled.
- Frameworks employed (e.g., XGBoost, LightGBM, FastAI, TabNet, Wavenet, ResNet1D) as part of the modeling components.
- Notation for the image classification model “eva02_tiny_patch14_224” indicates the usage of a transformer-based architecture for image analysis.

##### Elements
- Clear labels for each component: Data Preparation, Image Classification, Multiple Modeling Modules, and Ensembling.
- Annotations to specify that the system runs in a notebook-based pipeline on Kaggle (or Colab) and to highlight the main frameworks used in each stage.

- Flow into the image classification module (highlighting the use of “Moisture Stress” as a key feature),
- Branch into multiple parallel model training/inference blocks,
- Converge into an ensembling stage that finalizes the predictions.

This detailed breakdown will help ensure that every architectural element—from dependency flows to underlying model frameworks—is captured accurately, providing a clear and comprehensive overall view of the project’s design for any technical stakeholder evaluating the system.

Happy diagramming!

## Diagram (Mermaid)

```mermaid
flowchart TD
    A["Data Ingestion & EDA\n(first-stage-static-images-eda-data-preparation.ipynb)"]:::dataStage
    B["Image Classification Module\n(second-stage-image-classifier.ipynb, Model: eva02_tiny_patch14_224)"]:::imageStage
    C["Model Input"]:::modelStage
    D{"Ensembling\n(fourth-stage-models-ensemble.ipynb)"}:::ensembleStage

    A -->|"EDA_output"| B
    A -->|"EDA_features"| C
    B -->|"flood_prob"| C

    subgraph "Modeling Suite"
        direction TB
        M1["XGBoost Modeling"]:::modelStage
        M2["LightGBM Modeling"]:::modelStage
        M3["FastAI Tabular Modeling"]:::modelStage
        M4["FastAI GatedConv Modeling"]:::modelStage
        M5["FastAI 1DConv Modeling"]:::modelStage
        M6["FastAI TabTransformer Modeling"]:::modelStage
        M7["TabNet Modeling"]:::modelStage
        M8["Wavenet-GRU (Transformer) Modeling"]:::modelStage
        M9["Wavenet Inference Modeling"]:::modelStage
        M10["ResNet1D Inference Modeling"]:::modelStage
        M11["ResNet1D Modeling (0_1_2_3)"]:::modelStage
        M12["ResNet1D Modeling (4_5_6)"]:::modelStage
        M13["ResNet1D Modeling (7_8_9)"]:::modelStage
        MA{"All Models Combined"}:::modelStage

        %% Connect Model Input to each modeling node
        C --> M1
        C --> M2
        C --> M3
        C --> M4
        C --> M5
        C --> M6
        C --> M7
        C --> M8
        C --> M9
        C --> M10
        C --> M11
        C --> M12
        C --> M13

        %% Each model outputs to the aggregator
        M1 --> MA
        M2 --> MA
        M3 --> MA
        M4 --> MA
        M5 --> MA
        M6 --> MA
        M7 --> MA
        M8 --> MA
        M9 --> MA
        M10 --> MA
        M11 --> MA
        M12 --> MA
        M13 --> MA

        %% Aggregated model output goes to Ensembling
        MA --> D
    end

    %% Click Events
    click A "https://github.com/koleshjr/inundata-mapping-floods-in-south-africa/blob/main/first-stage-static-images-eda-data-preparation.ipynb"
    click B "https://github.com/koleshjr/inundata-mapping-floods-in-south-africa/blob/main/second-stage-image-classifier.ipynb"
    click M1 "https://github.com/koleshjr/inundata-mapping-floods-in-south-africa/blob/main/third_stage_xgb_modelling.ipynb"
    click M2 "https://github.com/koleshjr/inundata-mapping-floods-in-south-africa/blob/main/third_stage_lgb_modelling.ipynb"
    click M3 "https://github.com/koleshjr/inundata-mapping-floods-in-south-africa/blob/main/third-stage-fastai-tabular-modelling.ipynb"
    click M4 "https://github.com/koleshjr/inundata-mapping-floods-in-south-africa/blob/main/third_stage_fastai_gatedconv_modelling.ipynb"
    click M5 "https://github.com/koleshjr/inundata-mapping-floods-in-south-africa/blob/main/third_stage_fastai_1dconv_modelling.ipynb"
    click M6 "https://github.com/koleshjr/inundata-mapping-floods-in-south-africa/blob/main/third_stage_fastai_tabtransformer_modelling.ipynb"
    click M7 "https://github.com/koleshjr/inundata-mapping-floods-in-south-africa/blob/main/third_stage_tabnet_modelling.ipynb"
    click M8 "https://github.com/koleshjr/inundata-mapping-floods-in-south-africa/blob/main/third-stage-wavenet-gru-transformer-modelling.ipynb"
    click M9 "https://github.com/koleshjr/inundata-mapping-floods-in-south-africa/blob/main/third-stage-wavenet-inference-modelling.ipynb"
    click M10 "https://github.com/koleshjr/inundata-mapping-floods-in-south-africa/blob/main/third-stage-resnet1d-inference-modelling.ipynb"
    click M11 "https://github.com/koleshjr/inundata-mapping-floods-in-south-africa/blob/main/third_stage_resnet1d_0_1_2_3_modelling.ipynb"
    click M12 "https://github.com/koleshjr/inundata-mapping-floods-in-south-africa/blob/main/third_stage_resnet1d_4_5_6_modelling.ipynb"
    click M13 "https://github.com/koleshjr/inundata-mapping-floods-in-south-africa/blob/main/third_stage_resnet1d_7_8_9_modelling.ipynb"
    click D "https://github.com/koleshjr/inundata-mapping-floods-in-south-africa/blob/main/fourth-stage-models-ensemble.ipynb"

    %% Styles
    classDef dataStage fill:#FFD700,stroke:#333,stroke-width:2px;
    classDef imageStage fill:#87CEEB,stroke:#333,stroke-width:2px;
    classDef modelStage fill:#90EE90,stroke:#333,stroke-width:2px;
    classDef ensembleStage fill:#FFB6C1,stroke:#333,stroke-width:2px;
```
