# brendanartley/byu-competition architecture

> Architecture diagram of the GitHub repository brendanartley/byu-competition, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/brendanartley/byu-competition
- Repository: https://github.com/brendanartley/byu-competition

## Overview

##### Project Type and Purpose
- This is a full-stack machine‐learning training and inference pipeline tailored to a 3D U-Net solution for a Kaggle competition (Locating Bacterial Flagellar Motors).
- It orchestrates data ingestion, preprocessing, augmentation, model definition (3D U-Net and ResNet backbones), training, logging, checkpointing, and inference/thresholding.

##### File Structure Analysis
- Top-level:
- run.sh, train.py – orchestration scripts for training/end-to-end runs
- requirements.txt – Python dependencies
- README.md – high-level instructions and solution summary
- imgs/ – architecture illustration used in docs
- src/ directory (core codebase):
- src/pre/ – preprocessing entry point (src/pre/run.py)
- src/data/ – data loaders, utilities, raw↔processed paths
- src/augs/ – 3D data augmentation modules (mixup, cutmix, custom 3D)
- src/models/ – model definitions: base classes, 3D ResNet layers, U-Net layers
- src/configs/ – Hydra/Config-style experiment configuration
- src/losses/ – custom and auxiliary loss functions
- src/modules/ – higher-level training and metric loops
- src/logging/ – wandb or custom logging utilities
- src/utils/ – general utilities (cfg, torch helpers)
- Data directory (external, described in README): ./data/{raw,processed,checkpoints,model_zoo}

a. Data Ingestion & Preprocessing
– Raw competition tomograms (./data/raw → src/pre/run.py → processed .npy files)
b. Data Augmentation Pipeline
– src/augs/{mixup,cutmix,aug3d} applied inside data loader
c. Model Definition
– 3D U-Net (src/models/unet3d.py) built from layers in src/models/layers
– Optional auxiliary ResNet3D backbone (configs control r3d18 vs r3d200)
d. Loss Computation
– Primary segmentation loss + auxiliary losses (src/losses/base.py)
e. Training Loop & Logging
– src/modules/train.py orchestrates forward/backward, metrics (src/modules/metric.py), wandb logging (src/logging/utils.py)
f. Checkpointing & Model Zoo
– Checkpoints written to ./data/checkpoints, pretrained backbones loaded from model_zoo
g. Inference & Post-processing
– Ranking tomograms by max prediction, quantile thresholding for final binary decision

##### Relationships & Interactions
- Data flows: Raw files → preprocessing → DataLoader → augmentations → model → loss → gradients → update → checkpoint
- Configuration flows: configs load hyperparameters → drive model instantiation, optimizer, training parameters
- Logging: metrics & artifacts streamed to WandB via logging utils
- External services: GPU (CUDA), conda environment, WandB cloud

##### Architectural Patterns & Design Principles
- Modular layering: separate folders for augmentations, models, losses, modules
- Config-driven experiments: reuse of base and override configs for different backbones
- Clear separation of concerns: preprocessing, training, inference components are decoupled
- Utility code centralized under src/utils and src/logging

- Clear labels (e.g., “src/pre/run.py: Preprocessing”)
- Artifacts: config files, checkpoints, model_zoo, WandB

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Entry Points"
        runsh["run.sh"]:::external
        trainpy["train.py"]:::external
    end

    subgraph "Data Layer"
        raw["Raw Tomograms"]:::data
        data_utils["Data Utilities & Paths"]:::data
        preprocess["Preprocessing Module"]:::data
        processed["Processed Data (.npy)"]:::data
    end

    subgraph "Augmentation Layer"
        dataloader["DataLoader"]:::data
        aug3d["3D Augmentation"]:::data
        mixup["Mixup Augmentation"]:::data
        cutmix["CutMix Augmentation"]:::data
    end

    subgraph "Model Layer"
        model_base["Model Base & Helpers"]:::model
        configs["Configuration Management"]:::config
        unet_layers["U-Net Layers"]:::model
        unet_core["3D U-Net Core"]:::model
        resnet["ResNet3D Backbone"]:::model
    end

    subgraph "Loss & Metrics"
        loss["Loss Functions"]:::model
        metric["Metric Computation"]:::model
    end

    subgraph "Training Orchestrator"
        train_loop["Training Loop"]:::orch
    end

    subgraph "Logging & Checkpointing"
        wandb["WandB (Cloud)"]:::external
        checkpoint["Checkpoint Store"]:::storage
    end

    subgraph "Inference & Post-Processing"
        inference["Inference & Thresholding"]:::orch
        prediction["Final Predictions"]:::data
    end

    runsh --> train_loop
    trainpy --> train_loop

    raw --> preprocess --> processed --> dataloader
    data_utils -.-> dataloader

    dataloader --> aug3d
    aug3d --> mixup
    aug3d --> cutmix
    mixup --> unet_core
    cutmix --> unet_core

    configs --> dataloader
    configs --> unet_core
    configs --> train_loop

    model_base --> unet_core
    unet_layers --> unet_core
    resnet --> unet_core

    unet_core --> loss --> train_loop
    train_loop --> checkpoint
    train_loop --> wandb
    train_loop --> metric --> wandb
    checkpoint --> inference --> prediction

    classDef data fill:#D6EFFF,stroke:#003366
    classDef model fill:#D5F5E3,stroke:#1F6320
    classDef orch fill:#FFEFD5,stroke:#CC6600
    classDef storage fill:#E0E0E0,stroke:#666666
    classDef external fill:#F0F8FF,stroke:#1A1A1A
    classDef config fill:#FFF5BA,stroke:#B58900

    click preprocess "https://github.com/brendanartley/byu-competition/blob/main/src/pre/run.py"
    click data_utils "https://github.com/brendanartley/byu-competition/blob/main/src/data/utils.py"
    click aug3d "https://github.com/brendanartley/byu-competition/blob/main/src/augs/aug3d.py"
    click mixup "https://github.com/brendanartley/byu-competition/blob/main/src/augs/mixup.py"
    click cutmix "https://github.com/brendanartley/byu-competition/blob/main/src/augs/cutmix.py"
    click unet_core "https://github.com/brendanartley/byu-competition/blob/main/src/models/unet3d.py"
    click unet_layers "https://github.com/brendanartley/byu-competition/blob/main/src/models/layers/unet3d.py"
    click resnet "https://github.com/brendanartley/byu-competition/blob/main/src/models/layers/resnet3d.py"
    click model_base "https://github.com/brendanartley/byu-competition/blob/main/src/models/_base.py"
    click configs "https://github.com/brendanartley/byu-competition/blob/main/src/configs/_base.py"
    click loss "https://github.com/brendanartley/byu-competition/blob/main/src/losses/base.py"
    click train_loop "https://github.com/brendanartley/byu-competition/blob/main/src/modules/train.py"
    click metric "https://github.com/brendanartley/byu-competition/blob/main/src/modules/metric.py"
    click wandb "https://github.com/brendanartley/byu-competition/blob/main/src/logging/utils.py"
    click runsh "https://github.com/brendanartley/byu-competition/blob/main/run.sh"
    click trainpy "https://github.com/brendanartley/byu-competition/blob/main/train.py"
```
