# xjtuyw/pnp architecture

> Architecture diagram of the GitHub repository xjtuyw/pnp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/xjtuyw/pnp
- Repository: https://github.com/xjtuyw/pnp

## Overview

##### Project type and purpose
- This is a research-oriented deep learning training pipeline (Python/PyTorch) for “Generalized Class Discovery” (GCD).
- The codebase implements data ingestion, prototype-based clustering, a self-supervised prototype expansion mechanism, model architectures (ResNet, ViT, two-head ResNet), and training/evaluation scripts.

##### File structure
- Top-level scripts and executables:
- my_train.py – main training entry point
- cluster.py / estimate_k_gcd.py – clustering utilities
- config.py – global configuration and hyperparameters
- scripts/*.sh – bash wrappers to launch experiments on various datasets
- Core modules:
- data/ – dataset definitions (CIFAR, CARS, AIRCRAFT, HERBARIUM, PETS, IMAGENET, utilities for loading and batching)
- models/ – model definitions (ResNet variants, Vision Transformer, two-head designs)
- util/ – shared utilities (clustering and logging, general helper functions)
- Supporting:
- figs/ – method overview images (not part of runtime pipeline)

##### Key insights from the README
- The pipeline first clusters only unlabelled data, then augments prototypes via a learnable probing mechanism, optimizing end-to-end with prototypical contrastive learning.
- Scripts automate runs per dataset.
- Built atop an existing SimGCD codebase.

##### Elements
- another for model/training modules (e.g., green),
- a third for utility/configuration (e.g., gray),
- script/launcher as a separate parallelogram.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Experiment Launcher
    subgraph "Experiment"
        direction TB
        ExpLaunch["Experiment Launcher"]:::experiment
    end

    %% Configuration Manager
    subgraph "Configuration"
        direction TB
        Config["Configuration Manager\n(config.py)"]:::config
    end

    %% Utilities
    subgraph "Utilities"
        direction TB
        ClusterLogUtils["Cluster & Log Utils"]:::util
        GenUtils["General Utils"]:::util
    end

    %% Data Module
    subgraph "Data Module"
        direction TB
        DataLoaderDir["Data Loader Module\ndata/"]:::data
        DataUtils["data_utils.py"]:::data
        GetDatasets["get_datasets.py"]:::data
        DatasetStorage[(Local Dataset Storage)]:::storage
    end

    %% Clustering Engine
    subgraph "Clustering Engine"
        direction TB
        ClustEngine["cluster.py"]:::processing
        EstK["estimate_k_gcd.py"]:::processing
    end

    %% Model Training
    subgraph "Model Training"
        direction TB
        TrainOrch["Training Orchestrator\n(my_train.py)"]:::training
        PrototypeLearner["Prototype Expansion & SS Learner\n(model.py)"]:::training
        subgraph "Model Architectures"
            direction TB
            ResNet["ResNet\n(ResNet.py)"]:::model
            EncResNet["Encoder ResNet\n(encoder_resnet.py)"]:::model
            ViT["Vision Transformer\n(vision_transformer.py)"]:::model
            ResTwoHead["Two-Head ResNet\n(resnet_twohead.py)"]:::model
        end
    end

    %% Connections
    ExpLaunch -->|invokes| TrainOrch
    Config -.->|provides configs| TrainOrch

    TrainOrch -->|loads data| DataLoaderDir
    DataLoaderDir -->|uses definitions| DataUtils
    DataLoaderDir -->|uses loader| GetDatasets
    DataLoaderDir -->|reads from| DatasetStorage

    TrainOrch -->|calls clustering| ClustEngine
    ClustEngine -->|estimates K| EstK
    EstK -->|initial centers| PrototypeLearner

    TrainOrch -->|passes clusters| PrototypeLearner
    PrototypeLearner -->|updates prototypes| TrainOrch

    TrainOrch -->|forwards inputs| ResNet
    TrainOrch -->|forwards inputs| EncResNet
    TrainOrch -->|forwards inputs| ViT
    TrainOrch -->|forwards inputs| ResTwoHead

    TrainOrch -->|logs metrics| ClusterLogUtils
    TrainOrch -->|utility calls| GenUtils
    ClustEngine -->|utility calls| GenUtils
    PrototypeLearner -->|utility calls| GenUtils

    %% Click Events
    click ExpLaunch "https://github.com/xjtuyw/pnp/tree/master/scripts/"
    click Config "https://github.com/xjtuyw/pnp/blob/master/config.py"
    click DataLoaderDir "https://github.com/xjtuyw/pnp/tree/master/data/"
    click DataUtils "https://github.com/xjtuyw/pnp/blob/master/data/data_utils.py"
    click GetDatasets "https://github.com/xjtuyw/pnp/blob/master/data/get_datasets.py"
    click ClustEngine "https://github.com/xjtuyw/pnp/blob/master/cluster.py"
    click EstK "https://github.com/xjtuyw/pnp/blob/master/estimate_k_gcd.py"
    click PrototypeLearner "https://github.com/xjtuyw/pnp/blob/master/model.py"
    click ResNet "https://github.com/xjtuyw/pnp/blob/master/models/ResNet.py"
    click EncResNet "https://github.com/xjtuyw/pnp/blob/master/models/encoder_resnet.py"
    click ViT "https://github.com/xjtuyw/pnp/blob/master/models/vision_transformer.py"
    click ResTwoHead "https://github.com/xjtuyw/pnp/blob/master/models/resnet_twohead.py"
    click ClusterLogUtils "https://github.com/xjtuyw/pnp/blob/master/util/cluster_and_log_utils.py"
    click GenUtils "https://github.com/xjtuyw/pnp/blob/master/util/general_utils.py"

    %% Styles
    classDef experiment fill:#FFD699,stroke:#D67F00,color:#000
    classDef config fill:#EEEEEE,stroke:#AAAAAA,color:#333
    classDef util fill:#DDDDDD,stroke:#888888,color:#333
    classDef data fill:#BBDEFB,stroke:#1E88E5,color:#000
    classDef storage fill:#90CAF9,stroke:#1565C0,color:#000,stroke-dasharray: 5 5
    classDef processing fill:#FFF9C4,stroke:#FBC02D,color:#000
    classDef training fill:#C8E6C9,stroke:#43A047,color:#000
    classDef model fill:#A5D6A7,stroke:#2E7D32,color:#000
```
