# amlab-amsterdam/diva architecture

> Architecture diagram of the GitHub repository amlab-amsterdam/diva, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/amlab-amsterdam/diva
- Repository: https://github.com/amlab-amsterdam/diva

## Overview

##### Project type and purpose
- This is a research‐oriented PyTorch codebase implementing and comparing several domain‐adaptation and generative models (CNN, DANN, VAE, DIVA, conditional generators) on two datasets: malaria cell images and rotated MNIST. The main goal is to train and evaluate domain‐invariant variational autoencoders (DIVA) and variants, plus baselines.

##### Main components
a. Data module
- Datasets: “paper_experiments/malaria/dataset” and “paper_experiments/rotated_mnist/dataset”
- Data loaders (mnist_loader.py, data_loader_topk.py, etc.)
- Sup/unsup splits and additional unlabeled domains
b. Model module
- CNN variants under malaria/cnn
- Domain‐adaptation models under malaria/dann and rotated_mnist/additional_unlabeled_domain
- Generative models: VAE (vae/), DIVA (diva/) with and without z_d
- Utility code for model blocks (resnet blocks, pixel_cnn_utils)
c. Experiment scripts
- Individual .py scripts that set up configurations, initialize data loaders, instantiate models, define losses, run training loops, and save checkpoints or outputs
- Organized by dataset & method (e.g., experiment_diva_top10_resnet_new.py, experiment_vae_top10_resnet_new.py)
d. Output & evaluation
- Generated images (conditional_generation/generate_images.py)
- Trained model checkpoints (trained_models directories)
- Test‐set evaluation scripts (calculate_test.py)
e. External frameworks
- PyTorch 1.x, scikit‐image, scikit‐learn, seaborn
- Python 3.6

##### Relationships and interactions
- Experiment script imports Data module to fetch batches
- Batches fed into Model module
- Model’s forward pass → compute losses (reconstruction, domain classification, KL divergence)
- Backpropagation via PyTorch optimizer
- At checkpoints: save model weights, optionally call conditional generation utility
- Evaluation scripts load saved checkpoints, run inference, write metrics

##### Architectural patterns and principles
- Modular separation of data loaders, model definitions, and experiment orchestration
- Reusable building blocks (resnet blocks, VAE encoders/decoders, domain‐classifier heads)
- Clear directory‐by‐method organization encourages plug‐and‐play comparisons
- Scripts follow an “initialize → train → save → evaluate” pipeline

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        DS1(["Malaria dataset"]):::data
        DS2(["Rotated MNIST dataset (+ unlabeled)"]):::data
        DL1["data_loader_topk.py"]:::code
        DL2["mnist_loader.py"]:::code
        DL3["mnist_loader_add_unsup.py"]:::code
        DL4["mnist_loader_more_unsup.py"]:::code
    end

    %% Model Layer
    subgraph "Model Layer"
        MalCNN1["resnet_cnn_top_10_new.py"]:::code
        MalCNN2["resnet_cnn_top_10_new_without_domain_7.py"]:::code
        DANNmd["model_dann.py"]:::code
        DANNutil["functions_dann.py"]:::code
        VAEmd["model_vae.py"]:::code
        DIVAm1["model_diva_resnet_new_linear.py"]:::code
        DIVAm2["model_diva_resnet_new_linear_upsampling.py"]:::code
        PIXU["pixel_cnn_utils.py"]:::code
        ResB["resnet_blocks_batchnorm.py"]:::code
        rmD1["model_diva.py"]:::code
        rmD2["model_diva_without_zd.py"]:::code
        rmVAE["model_vae.py"]:::code
    end

    %% Experiment Layer
    subgraph "Experiment Layer"
        ExDANN1["experiment_dann_top10_resnet_new.py"]:::exp
        ExDANN2["experiment_dann_top10_resnet_new_semi_unsup_domain_7.py"]:::exp
        ExDANN3["experiment_dann_top10_resnet_new_without_domain_7.py"]:::exp
        ExpVAE["experiment_vae_top10_resnet_new.py"]:::exp
        ExpDIV1["experiment_diva_top10_resnet_new.py"]:::exp
        ExpDIV2["experiment_diva_top10_resnet_new_semi_unsup_domain_7.py"]:::exp
        ExpDIV3["experiment_diva_top10_resnet_new_without_domain_7.py"]:::exp
        add30exp1["add_30/experiment.py"]:::exp
        add30exp2["add_30/experiment_only_sup.py"]:::exp
        add30d1["add_30/dann.py"]:::exp
        add30d2["add_30/dann_only_sup.py"]:::exp
    end

    %% Output Layer
    subgraph "Output Layer"
        Ckpt["Checkpoint Storage"]:::data
        GenM["generate_images.py (malaria)"]:::code
        GenRM["generate_images.py (rotated mnist)"]:::code
        Eval["calculate_test.py"]:::exp
    end

    %% Frameworks
    subgraph "Frameworks"
        PyT["PyTorch"]:::fw
        SKI["scikit-image"]:::fw
        SKL["scikit-learn"]:::fw
        SEB["seaborn"]:::fw
    end

    %% Connections Data to Loaders
    DS1 --> DL1
    DS2 --> DL2
    DS2 --> DL3
    DS2 --> DL4

    %% Data to Models
    DL1 --> MalCNN1
    DL1 --> MalCNN2
    DL1 --> DANNmd
    DL1 --> VAEmd
    DL1 --> DIVAm1
    DL1 --> DIVAm2
    DL1 --> ResB

    DL2 --> rmD1
    DL2 --> rmD2
    DL2 --> rmVAE

    %% Models to Experiments
    DANNmd --> ExDANN1
    DANNmd --> ExDANN2
    DANNmd --> ExDANN3
    DANNutil --> ExDANN1

    VAEmd --> ExpVAE

    DIVAm1 --> ExpDIV1
    DIVAm2 --> ExpDIV1
    PIXU --> ExpDIV1

    MalCNN1 --> ExDANN1
    MalCNN1 --> ExpVAE
    MalCNN1 --> ExpDIV1

    rmD1 --> add30exp1
    rmD2 --> add30exp1
    rmVAE --> add30exp1

    %% Experiments to Outputs
    ExDANN1 --> Ckpt
    ExDANN1 --> GenM
    ExDANN1 --> Eval

    ExpVAE --> Ckpt
    ExpVAE --> GenM
    ExpVAE --> Eval

    ExpDIV1 --> Ckpt
    ExpDIV1 --> GenM
    ExpDIV1 --> Eval

    add30exp1 --> Ckpt
    add30exp1 --> GenRM
    add30exp1 --> Eval

    %% Frameworks to all
    PyT --> DL1
    PyT --> MalCNN1
    PyT --> ExDANN1
    SKI --> ExpDIV1
    SKL --> ExpVAE
    SEB --> Eval

    %% Click Events
    click DS1 "https://github.com/amlab-amsterdam/diva/tree/master/paper_experiments/malaria/dataset"
    click DL1 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/malaria/data_loader_topk.py"
    click DL2 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/rotated_mnist/dataset/mnist_loader.py"
    click DL3 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/rotated_mnist/dataset/mnist_loader_add_unsup.py"
    click DL4 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/rotated_mnist/dataset/mnist_loader_more_unsup.py"
    click MalCNN1 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/malaria/cnn/resnet_cnn_top_10_new.py"
    click MalCNN2 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/malaria/cnn/resnet_cnn_top_10_new_without_domain_7.py"
    click DANNmd "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/malaria/dann/model_dann.py"
    click DANNutil "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/malaria/dann/functions_dann.py"
    click ExDANN1 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/malaria/dann/experiment_dann_top10_resnet_new.py"
    click ExDANN2 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/malaria/dann/experiment_dann_top10_resnet_new_semi_unsup_domain_7.py"
    click ExDANN3 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/malaria/dann/experiment_dann_top10_resnet_new_without_domain_7.py"
    click VAEmd "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/malaria/vae/model_vae.py"
    click ExpVAE "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/malaria/vae/experiment_vae_top10_resnet_new.py"
    click DIVAm1 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/malaria/diva/model_diva_resnet_new_linear.py"
    click DIVAm2 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/malaria/diva/model_diva_resnet_new_linear_upsampling.py"
    click PIXU "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/malaria/diva/pixel_cnn_utils.py"
    click ExpDIV1 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/malaria/diva/experiment_diva_top10_resnet_new.py"
    click ExpDIV2 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/malaria/diva/experiment_diva_top10_resnet_new_semi_unsup_domain_7.py"
    click ExpDIV3 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/malaria/diva/experiment_diva_top10_resnet_new_without_domain_7.py"
    click ResB "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/malaria/resnet_blocks_batchnorm.py"
    click GenM "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/malaria/conditional_generation/generate_images.py"
    click GenRM "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/rotated_mnist/conditional_generation/generate_images.py"
    click add30exp1 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/rotated_mnist/additional_unlabeled_domain/add_30/experiment.py"
    click add30exp2 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/rotated_mnist/additional_unlabeled_domain/add_30/experiment_only_sup.py"
    click add30d1 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/rotated_mnist/additional_unlabeled_domain/add_30/dann.py"
    click add30d2 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/rotated_mnist/additional_unlabeled_domain/add_30/dann_only_sup.py"
    click rmD1 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/rotated_mnist/supervised/model_diva.py"
    click rmD2 "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/rotated_mnist/supervised/model_diva_without_zd.py"
    click rmVAE "https://github.com/amlab-amsterdam/diva/blob/master/paper_experiments/rotated_mnist/supervised/model_vae.py"

    %% Styles
    classDef data fill:#b7e4c7,stroke:#2d6a4f,color:#1b4332
    classDef code fill:#9ec1cf,stroke:#0b3d91,color:#073b4c
    classDef exp fill:#f5c2c7,stroke:#89375f,color:#6a040f
    classDef fw fill:#dddddd,stroke:#555555,color:#333333
```
