# zhengpeng7/birefnet architecture

> Architecture diagram of the GitHub repository zhengpeng7/birefnet, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zhengpeng7/birefnet
- Repository: https://github.com/zhengpeng7/birefnet

## Overview

##### Project Type and Purpose
- This is a research‐grade, open‐source Python library and script set for training, evaluating, and deploying a high‐resolution dichotomous image segmentation model (BiRefNet).
- It provides end‐to‐end pipelines: dataset preparation, configuration, model definition, training, evaluation, inference, and ONNX export, plus tutorial notebooks.

##### File‐Structure Analysis
- Top‐level scripts:
- config.py – global hyperparameters and flags (multi‐GPU, backbone choice, loss weights).
- dataset.py & image_proc.py – data‐loading, augmentation, preprocessing.
- train.py / train.sh – training orchestration.
- inference.py – batch and single‐image inference.
- eval_existingOnes.py & evaluation/metrics.py – post‐hoc metric computation and comparison.
- models/
- backbones/ – Swin, PVT, build scripts.
- modules/ – ASPP, decoder blocks, deformable conv, lateral blocks, utils.
- refinement/ – final refiner head.
- birefnet.py – glue logic assembling backbone, module blocks, and refiner.
- utilities: loss.py, utils.py, shell scripts (train.sh, test.sh, make_a_copy.sh), tutorials/*.ipynb.

##### Key Insights from README
- Config‐driven extensibility: easy swap of backbone, loss terms, datasets, multi‐scale inputs, FP16/F32 training.
- Model zoo, ONNX conversion, Hugging Face integration, Google Drive weight hosting: external dependencies and endpoints.

– Data Loader & Preprocessor (blue rectangle): dataset.py + image_proc.py
– Model Definition (blue rectangle): models/birefnet.py orchestrating →
– Loss & Optimization (orange rectangle): loss.py + PyTorch optimizer + torch.compile
– Training Orchestrator (purple rectangle): train.py / train.sh
– Inference Engine (pink rectangle): inference.py + optional ONNX runtime
– Evaluation & Metrics (grey rectangle): evaluation/metrics.py + eval_existingOnes.py
– External Services (hexagon, red): GPU cluster, Hugging Face Model Hub, Google Drive, fal.ai API, TensorRT pipelines.

1. Raw Datasets → Data Loader & Preprocessor
2. Preprocessed Batches → Model Definition
3. Model Outputs → Loss & Optimization → back-prop updates Model
4. At epoch end: Model → Checkpoint Store
5. Trained Checkpoints → Inference Engine
6. Inference Outputs → Evaluation & Metrics → Reports/Plots
7. Optionally → ONNX Conversion → External Inference Runtimes (ONNX, TensorRT)

c. Architectural Patterns
- Modular, layered design: backbone → decoder → refinement, all wired via config.py.
- Pipeline separation: distinct training vs. inference vs. evaluation phases.
- Config‐centric orchestration: paths, tasks, losses, devices, mixed precision.

d. Technologies & Frameworks
- Python 3.10, PyTorch 2.x (torch.compile), CUDA 11.8/12.4, accelerate for FP16.
- ONNX & onnxruntime‐gpu for export/inference.
- Shell scripts for multi‐GPU launching.
- Jupyter notebooks for tutorials.
- HuggingFace transformers integration for model loading.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Preparation"
        dataStore["Raw Datasets (DIS, COD, HRSOD, P3M, etc.)"]:::datastore
        dataLoaderDS["dataset.py"]:::code
        dataLoaderIP["image_proc.py"]:::code
        dataStore --> dataLoaderDS
        dataLoaderDS --> dataLoaderIP
    end

    subgraph "Model Definition"
        config["Configuration Manager (config.py)"]:::config
        birefnet["BiRefNet Glue (models/birefnet.py)"]:::code
        subgraph "Backbone Subcomponent"
            bb1["build_backbone.py"]:::subcode
            bb2["pvt_v2.py"]:::subcode
            bb3["swin_v1.py"]:::subcode
        end
        subgraph "Feature-Processing Modules"
            m1["aspp.py"]:::subcode
            m2["decoder_blocks.py"]:::subcode
            m3["deform_conv.py"]:::subcode
            m4["lateral_blocks.py"]:::subcode
            m5["utils.py"]:::subcode
        end
        subgraph "Refinement Head"
            r1["refiner.py"]:::subcode
            r2["stem_layer.py"]:::subcode
        end
        config --> birefnet
        dataLoaderIP --> birefnet
        bb1 --> birefnet
        bb2 --> birefnet
        bb3 --> birefnet
        m1 --> birefnet
        m2 --> birefnet
        m3 --> birefnet
        m4 --> birefnet
        m5 --> birefnet
        r1 --> birefnet
        r2 --> birefnet
    end

    subgraph "Loss & Optimization"
        lossNode["loss.py + PyTorch Optimizer"]:::trainlogic
    end
    birefnet --> lossNode
    lossNode -->|"backprop"| birefnet

    subgraph "Checkpoint Store"
        checkpoint[".pth Checkpoints"]:::datastore
    end
    birefnet -->|"save at epoch end"| checkpoint

    subgraph "Training Orchestrator"
        trainpy["train.py"]:::train
        trainsh["train.sh"]:::train
        trainpy --> trainsh
        trainpy --> birefnet
        trainsh --> gpu
    end

    subgraph "Inference Engine"
        inference["inference.py"]:::infer
        onnxConv["ONNX Conversion"]:::infer
        inference --> onnxConv
    end
    checkpoint --> inference

    subgraph "Evaluation & Metrics"
        evalScript["eval_existingOnes.py"]:::eval
        metrics["evaluation/metrics.py"]:::eval
        reports["Reports/Plots"]:::eval
        inference --> evalScript
        evalScript --> metrics
        metrics --> reports
    end

    tutorials["Tutorial Notebooks (tutorials/*.ipynb)"]:::tutorial

    subgraph "External Services"
        gpu["GPU Cluster"]:::external
        hf["Hugging Face Model Hub"]:::external
        gdrive["Google Drive Weights"]:::external
        fal["fal.ai API"]:::external
        trt["TensorRT Pipelines"]:::external
        onnxrt["ONNX Runtime"]:::external
        birefnet --> hf
        checkpoint --> gdrive
        onnxConv --> trt
        onnxConv --> onnxrt
    end

    click config "https://github.com/zhengpeng7/birefnet/blob/main/config.py"
    click dataLoaderDS "https://github.com/zhengpeng7/birefnet/blob/main/dataset.py"
    click dataLoaderIP "https://github.com/zhengpeng7/birefnet/blob/main/image_proc.py"
    click birefnet "https://github.com/zhengpeng7/birefnet/blob/main/models/birefnet.py"
    click bb1 "https://github.com/zhengpeng7/birefnet/blob/main/models/backbones/build_backbone.py"
    click bb2 "https://github.com/zhengpeng7/birefnet/blob/main/models/backbones/pvt_v2.py"
    click bb3 "https://github.com/zhengpeng7/birefnet/blob/main/models/backbones/swin_v1.py"
    click m1 "https://github.com/zhengpeng7/birefnet/blob/main/models/modules/aspp.py"
    click m2 "https://github.com/zhengpeng7/birefnet/blob/main/models/modules/decoder_blocks.py"
    click m3 "https://github.com/zhengpeng7/birefnet/blob/main/models/modules/deform_conv.py"
    click m4 "https://github.com/zhengpeng7/birefnet/blob/main/models/modules/lateral_blocks.py"
    click m5 "https://github.com/zhengpeng7/birefnet/blob/main/models/modules/utils.py"
    click r1 "https://github.com/zhengpeng7/birefnet/blob/main/models/refinement/refiner.py"
    click r2 "https://github.com/zhengpeng7/birefnet/blob/main/models/refinement/stem_layer.py"
    click lossNode "https://github.com/zhengpeng7/birefnet/blob/main/loss.py"
    click trainpy "https://github.com/zhengpeng7/birefnet/blob/main/train.py"
    click trainsh "https://github.com/zhengpeng7/birefnet/blob/main/train.sh"
    click inference "https://github.com/zhengpeng7/birefnet/blob/main/inference.py"
    click evalScript "https://github.com/zhengpeng7/birefnet/blob/main/eval_existingOnes.py"
    click metrics "https://github.com/zhengpeng7/birefnet/blob/main/evaluation/metrics.py"
    click tutorials "https://github.com/zhengpeng7/birefnet/blob/main/tutorials/*.ipynb"

    classDef datastore fill:#9f6,stroke:#333,stroke-width:1px
    classDef code fill:#bbf,stroke:#333,stroke-width:1px
    classDef subcode fill:#def,stroke:#333,stroke-width:1px
    classDef config fill:#ccf,stroke:#333,stroke-width:1px
    classDef train fill:#dcb,stroke:#333,stroke-width:1px
    classDef trainlogic fill:#fa8,stroke:#333,stroke-width:1px
    classDef infer fill:#fba,stroke:#333,stroke-width:1px
    classDef eval fill:#ccc,stroke:#333,stroke-width:1px
    classDef tutorial stroke-dasharray:5 5,fill:#fff,stroke:#bbb
    classDef external fill:#f88,stroke:#900,stroke-width:1px
```
