# ganchaofan0000/dac architecture

> Architecture diagram of the GitHub repository ganchaofan0000/dac, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ganchaofan0000/dac
- Repository: https://github.com/ganchaofan0000/dac

## Overview

##### Project type and purpose
- DAC is a research‐oriented deep learning framework (PyTorch) implementing a 2D–3D cross‐modal retrieval method with noisy label correction.
- It is not a client‐server or web application, but a modular training and evaluation pipeline for experiments from the ACM MM 2024 paper.

##### Top‐level component analysis
File tree:
- data/
– train_200.json: noisy labels for Objaverse‐N200
- losses/
– CCL_loss.py, MAE.py, cross_modal_loss.py
- models/
– image_encoder.py, pt_encoder.py, fused_layer.py, resnet.py
- tools/
– dataset.py (data loading & preprocessing), utils.py
- train.sh, train_dual.py (entry points)

##### Key architectural patterns and dependencies
- Modular separation:
a. Data ingestion & preprocessing (tools/)
b. Model definitions (models/)
c. Loss computations (losses/)
d. Training orchestration (train_dual.py / train.sh)
- Pluggable components: easy to swap encoders or loss functions.
- Single‐process, GPU‐driven training loop (no microservices or distributed components here).

A. Data Store (data/)
– JSON label file, point-cloud features, 2D images (external storage)
B. Data Loader & Preprocessor (tools/dataset.py)
– Reads data, applies transforms, yields batches
C. Model Encoders
1. Image Encoder (models/image_encoder.py + resnet backbone)
2. Point-cloud Encoder (models/pt_encoder.py)
D. Fusion Module (models/fused_layer.py)
– Combines 2D and 3D embeddings
E. Loss Modules (losses/)
– Cross‐modal loss, CCL, MAE
– Adaptive alignment & correction logic
F. Training Orchestrator (train_dual.py + train.sh)
– Instantiate DataLoader, Models, Losses, Optimizer
– Runs forward/backward passes, logging, checkpointing
G. External Pretrained Models
– Openshape, ResNet weights, Uni3D for label generation
H. Outputs
– Checkpoints, metrics, tensorboard logs, evaluation results

##### Relationships and data flows
1. The Training Orchestrator triggers the Data Loader.
2. Data Loader fetches raw images & point features, outputs paired batches.
3. Batches flow into the two encoders in parallel.
4. Embeddings converge into the fused_layer module.
5. Fused embeddings + true/noisy labels are passed to Loss Modules.
6. Loss Modules compute multiple sub‐losses (cross‐modal, CCL, MAE), sum them.
7. Backprop through the encoders and fusion module.
8. Optimizer updates model weights.
9. Loop until convergence; logs and checkpoints are persisted.

##### Architectural patterns & design principles
- Single Responsibility: Each folder handles one aspect (data, model, loss, utils).
- Dependency Injection: train_dual.py wires together components at runtime.
- Modular Pluggability: New losses or encoders can be added by dropping files into losses/ or models/.
- Clear separation between data pipeline, model definition, and training logic.

– Clear textual labels on every component
– Version identifiers for key dependencies (e.g., PyTorch 1.7.1, Python 3.8)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        DS[(Data Store)]:::data
    end

    DS -->|batch of images + point clouds| Loader["Data Loader & Preprocessor"]:::data

    click DS "https://github.com/ganchaofan0000/dac/tree/main/data/"
    click Loader "https://github.com/ganchaofan0000/dac/blob/main/tools/dataset.py"

    %% Model Layer
    subgraph "Model Layer"
        ImageEnc["Image Encoder"]:::model
        ResNet["ResNet Backbone"]:::model
        PointEnc["Point-cloud Encoder"]:::model
        Fusion["Fusion Module"]:::model
    end

    Loader --> ImageEnc
    Loader --> PointEnc
    ImageEnc --> ResNet
    ImageEnc --> Fusion
    PointEnc --> Fusion

    click ImageEnc "https://github.com/ganchaofan0000/dac/blob/main/models/image_encoder.py"
    click ResNet "https://github.com/ganchaofan0000/dac/blob/main/models/resnet.py"
    click PointEnc "https://github.com/ganchaofan0000/dac/blob/main/models/pt_encoder.py"
    click Fusion "https://github.com/ganchaofan0000/dac/blob/main/models/fused_layer.py"

    %% External Pretrained Models
    ExtPre["External Pretrained Models"]:::external
    ExtPre --> ImageEnc
    ExtPre --> PointEnc

    %% Loss Layer
    subgraph "Loss Layer"
        CrossModal["Cross-modal Loss"]:::loss
        CCL["Contrastive Correction Loss (CCL)"]:::loss
        MAE["Masked Autoencoder Loss (MAE)"]:::loss
        Sum["Loss Aggregator"]:::loss
    end

    Fusion -->|embeddings + labels| CrossModal
    Fusion -->|embeddings + labels| CCL
    Fusion -->|embeddings + labels| MAE

    click CrossModal "https://github.com/ganchaofan0000/dac/blob/main/losses/cross_modal_loss.py"
    click CCL "https://github.com/ganchaofan0000/dac/blob/main/losses/CCL_loss.py"
    click MAE "https://github.com/ganchaofan0000/dac/blob/main/losses/MAE.py"

    CrossModal --> Sum
    CCL --> Sum
    MAE --> Sum

    %% Orchestration Layer
    Sum -->|total loss| Backprop["Backpropagation & Optimizer"]:::orchestration

    subgraph "Orchestration Layer"
        Orch["Training Orchestrator"]:::orchestration
        Shell["Shell Wrapper"]:::orchestration
        Checkpoints[(Checkpoints & Logs)]:::data
    end

    Backprop --> Orch
    Orch --> Checkpoints

    click Orch "https://github.com/ganchaofan0000/dac/blob/main/train_dual.py"
    click Shell "https://github.com/ganchaofan0000/dac/blob/main/train.sh"

    %% Legend / Versioning
    Note["Python 3.8<br/>PyTorch 1.7.1"]:::external

    %% Styles
    classDef data fill:#e0f7fa,stroke:#006064,color:#006064;
    classDef model fill:#e8f5e9,stroke:#1b5e20,color:#1b5e20;
    classDef loss fill:#fff3e0,stroke:#ef6c00,color:#ef6c00;
    classDef orchestration fill:#eceff1,stroke:#37474f,color:#37474f;
    classDef external fill:#f3e5f5,stroke:#6a1b9a,color:#6a1b9a;
```
