# roeeben/generating-band-limited-adversarial-surfaces-using-neural-networks architecture

> Architecture diagram of the GitHub repository roeeben/generating-band-limited-adversarial-surfaces-using-neural-networks, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/roeeben/generating-band-limited-adversarial-surfaces-using-neural-networks
- Repository: https://github.com/roeeben/generating-band-limited-adversarial-surfaces-using-neural-networks

## Overview

##### Project’s Main Purpose and Type
- The README clarifies that the project is about generating adversarial examples on 3D data (meshes/point-clouds) using neural networks.
- It uses PointNet’s architecture (with modifications) both for classification and to build the adversarial network (auto-encoder based).
- The purpose is split between training a classifier (to be robust) and generating adversarial perturbations via a neural network.
- There is integration with tracking tools (e.g., Weights & Biases) and it leverages the PyTorch3D library for 3D data, rendering, and related operations.

##### File Structure for Architectural Components
- Top-Level Scripts and Files:
- band_limited_runner.py, classifier_trainer_main.py, model_trainer_main.py: Entry points for running attacks and for training models.
- run_config.py: Likely a configuration file that sets hyperparameters or paths, affecting training runs.
- setup.sh and requirements.txt: Environment setup and dependency specification.
- assets and saved_params: Hold visual assets (images, GIFs used in the README) and saved model checkpoints respectively.
- The src/ Directory Contains the Bulk of the Code Arranged as Follows:
a. src/adversarial:
- Contains implementations of adversarial attack algorithms (e.g., PGD, Carlini-Wagner) and supporting modules (base classes and output handlers).
- Represents the strategies or methods for generating adversarial perturbations.

b. src/dataset:
- Houses data loaders, dataset-specific files for FAUST and SHREC14, and utilities for preprocessing (e.g., downscaling).
- This is critical for ingesting the 3D mesh/point-cloud inputs and preparing them for training.

c. src/deep_adv_3d:
- Contains key components for the training of the adversarial network: custom loss functions, training loops, TensorBoard logging and utility functions.
- Implements the auto-encoder style architecture that generates perturbations.

d. src/models:
- Contains definitions of network architectures such as the modified PointNet and other model variants (e.g., deep_adv_3d_model1).

e. src/pytorch3d:
- Contains a very extensive suite of utilities and operations that extend PyTorch3D functionality.
- Covers functionalities from common tensors and types, CUDA/C++ implementations (for ball query, blending, compositing, etc.), datasets (ShapeNet, R2N2), I/O for different 3D formats, loss functions, and more.
- Serves as the backbone for all 3D processing, rendering, transformation, and differential rendering needs.

f. src/utils and src/vista:
- Utilities for mesh decimation, eigenpairs, transforms, visualization, and other helper functions.
- The vista folder, for example, is oriented toward visualizing results and plotting adversarial attacks.

##### Explaining the Main Components and Their Interactions
a. Data Input & Pre-Processing:
- Component: Dataset Loader (src/dataset)
- Function: Reads raw 3D data (from FAUST and SHREC14 datasets), applies downscaling or other transformations, and prepares batched input for subsequent modules.

b. Classifier (Victim Model):
- Component: PointNet (located within src/models or as a part of the adversarial training process).

c. Adversarial Generator (Attacking Network):
- Component: Auto-Encoder–Based Network (e.g., deep_adv_3d_model1 in src/models, with auxiliary codes in src/deep_adv_3d).
- Function: Generates subtle, band-limited perturbations intended to mislead the classifier. It uses modified PointNet architecture concepts.

d. Adversarial Strategies and Algorithms:
- Component: Adversarial Methods (src/adversarial)
- Function: Implements attack algorithms like PGD and Carlini-Wagner variants, which guide or refine how noise is applied.

e. 3D Processing & Rendering:
- Component: PyTorch3D Extensions (src/pytorch3d)
- Function: Provides detailed methods for mesh processing, rasterization, 3D transforms, and rendering gradients that are essential for computing differential losses and visualizing outputs.

f. Training & Experimentation Loop:
- Component: Training Loop & Loss Handling (src/deep_adv_3d/train_loop.py, src/deep_adv_3d/loss.py)
- Function: Controls experiment runs, hyperparameter sweeps (integrated with Weights & Biases), loss computation, and backpropagation.

g. Visualization & Results Output:
- Component: Vista Utilities (src/vista) and Logging (tensor_board in deep_adv_3d and utils settings).
- Function: Visualizes intermediate results including the adversarial attacks and tracked training metrics.

h. Configuration & Environment:
- Component: run_config.py, setup.sh, requirements.txt
- Function: Manage environment setup and define parameters that affect overall system behavior.

##### Relationships and Data Flows
- The attack’s output (adversarial perturbations) then affects the input of the classifier, closing the loop for adversarial evaluation.
- The training loop interacts with both the adversarial network and the classifier, feeding losses back to adjust weights.
- The PyTorch3D components underscore almost every step that involves 3D data conversion, rendering, and differentiable operations.
- Logs and visual outputs flow from the training components to the visualization modules (Weights & Biases/TensorBoard).

##### Architectural Patterns and Design Principles
- Separation of Concerns: Distinct directories are responsible for adversarial methods, data handling, network models, and low-level 3D operations.
- Modularity: Clear separation between core functionalities, allowing one to update or swap a module (for example, testing a different adversarial algorithm) without impacting others.
- Scalability: With configuration files and a clear run configuration, the architecture supports extensive hyperparameter searches and experimentation.
- Reusability: The heavy use of PyTorch3D and modular utilities (e.g., for mesh processing and visualization) ensures that components can be reused across experiments.

1. Data Input & Preprocessing (Dataset Loaders from src/dataset)
2. Classifier (PointNet module in src/models) and Adversarial Generator (Auto-encoder from src/deep_adv_3d and src/models)
3. Adversarial Methods (PGD, Carlini-Wagner implementations in src/adversarial)
4. 3D Processing & Rendering (PyTorch3D submodules from src/pytorch3d)
5. Training Loop & Loss Computation (src/deep_adv_3d/train_loop.py and loss functions)
6. Visualization & Experiment Tracking (Weights & Biases integration, TensorBoard logging, vista utils)
7. Configuration & Environment (run_config.py and setup scripts)
- Sparing interactions between modules (for example, how the adversarial algorithms alter the autoencoder’s behavior; how PyTorch3D functions are called upon for 3D operations).
- Integration points with any external frameworks/libraries.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Acquisition & Preprocessing"
        DS["Dataset Loaders (src/dataset)"]:::dataset
    end

    subgraph "Model Components"
        CL["Classifier (PointNet)"]:::model
        AG["Adversarial Generator (AutoEncoder)"]:::model
    end

    subgraph "Adversarial Strategies"
        AS["Attack Algorithms (PGD,Carlini-Wagner)"]:::adversarial
    end

    subgraph "3D Processing & Rendering"
        PT3["PyTorch3D Extensions"]:::pytorch3d
    end

    subgraph "Training & Experimentation"
        TL["Training Loop & Loss Handling"]:::training
        TB["TensorBoard&WandB Integration"]:::training
    end

    subgraph "Visualization & Results"
        VIS["Visualization Utilities (Vista)"]:::visualization
    end

    subgraph "Configuration & Environment"
        CONF["Configuration (run_config.py,setup.sh,requirements.txt)"]:::config
    end

    DS -->|"preprocessed3Ddata"| CL
    DS -->|"preprocessed3Ddata"| AG

    AG -->|"perturbationsApplied"| CL
    AS -->|"modulatesPerturbations"| AG

    TL -->|"trainsNetworks"| AG
    TL -->|"trainsNetworks"| CL
    TB -->|"logsMetrics"| TL

    PT3 -->|"3DoperationsSupport"| AG
    PT3 -->|"3DrenderingSupport"| CL

    VIS -->|"displaysOutputs"| TL

    CONF -->|"definesParameters"| DS
    CONF -->|"definesParameters"| TL
    CONF -->|"definesParameters"| AG
    CONF -->|"definesParameters"| CL

    AS -->|"influencesTraining"| TL

    click DS "https://github.com/roeeben/generating-band-limited-adversarial-surfaces-using-neural-networks/tree/main/src/dataset"
    click CL "https://github.com/roeeben/generating-band-limited-adversarial-surfaces-using-neural-networks/blob/main/src/models/pointnet.py"
    click AG "https://github.com/roeeben/generating-band-limited-adversarial-surfaces-using-neural-networks/blob/main/src/models/deep_adv_3d_model1.py"
    click TL "https://github.com/roeeben/generating-band-limited-adversarial-surfaces-using-neural-networks/blob/main/src/deep_adv_3d/train_loop.py"
    click AS "https://github.com/roeeben/generating-band-limited-adversarial-surfaces-using-neural-networks/tree/main/src/adversarial"
    click PT3 "https://github.com/roeeben/generating-band-limited-adversarial-surfaces-using-neural-networks/tree/main/src/pytorch3d"
    click VIS "https://github.com/roeeben/generating-band-limited-adversarial-surfaces-using-neural-networks/tree/main/src/vista"
    click CONF "https://github.com/roeeben/generating-band-limited-adversarial-surfaces-using-neural-networks/blob/main/run_config.py"

    classDef dataset fill:#fdd,stroke:#333,stroke-width:2px;
    classDef model fill:#dfd,stroke:#333,stroke-width:2px;
    classDef adversarial fill:#cdf,stroke:#333,stroke-width:2px;
    classDef pytorch3d fill:#ffd,stroke:#333,stroke-width:2px;
    classDef training fill:#fcf,stroke:#333,stroke-width:2px;
    classDef visualization fill:#cfd,stroke:#333,stroke-width:2px;
    classDef config fill:#cff,stroke:#333,stroke-width:2px;
```
