# ai45lab/reef architecture

> Architecture diagram of the GitHub repository ai45lab/reef, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ai45lab/reef
- Repository: https://github.com/ai45lab/reef

## Overview

##### Project Type & Purpose
- Key use cases: model-fingerprinting (CKA), classifier training (linear/MLP/CNN/GCN), and comparative experiments (PCS, ICS).

##### File Structure
Top-level:
– LICENSE, README.md, requirements.txt (project metadata)
– src/ (all Python code, datasets, notebooks, helper scripts)
Under src/:
– datasets/ (.csv files for various benchmarks)
– scripts/ (shell wrappers: save_activation.sh, save_logits.sh)
– compute_cka.py, generate_activations.py, generate_head_activations.py, logits.py, pcs.py, ics.py, train_cls.py, train_cls_gcn.py, transfer_cls.py, model.py, utils.py
– plot.ipynb (visualization notebook)

##### Read README for Architecture Insights
– Data sources: CSV benchmarks + pre-trained LLM checkpoints (e.g., llama-2-7b, vicuna-7b)
– Modules invoked by shell scripts (activation & logits extraction)
– Main computational steps: generate activations → compute CKA → train/transfer classifiers → plot
– Comparative experiments: PCS, ICS

##### Main Components & Their Interactions
A. Data Layer
- Raw datasets (.csv)
- Pre-trained models (Hugging Face checkpoint downloads)
B. Extraction Layer
- generate_activations.py (invoked by save_activation.sh)
- generate_head_activations.py (for per-head features)
- logits.py (invoked by save_logits.sh)
C. Similarity Computation
- compute_cka.py → imports model.py + utils.py
- Outputs a CKA similarity matrix
D. Classifier Module
- train_cls.py (linear/MLP/CNN)
- train_cls_gcn.py (graph-based classifier)
- transfer_cls.py (applies pre-trained classifier to suspect models)
E. Comparative Experiments
- pcs.py, ics.py (other fingerprinting methods)
F. Visualization
- plot.ipynb (heatmaps, graphs)
G. Orchestration
- Shell scripts in src/scripts/
- CLI arguments in Python entrypoints

##### Architectural Patterns & Design Principles
– Modular Python package: each major function (activation, CKA, classification) lives in its own script
– Separation of concerns: data ingestion, feature extraction, similarity computation, classifier training, visualization
– Reusable utility code in utils.py and centralized model loading in model.py
– Shell scripts as lightweight pipelines for reproducibility

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Layer
    DataSources["Data Sources\nCSV datasets\nHugging Face Models"]:::data
    click DataSources "https://github.com/ai45lab/reef/tree/master/src/datasets/"

    %% Shell Orchestration
    subgraph "Shell Orchestrator"
        SaveAct["save_activation.sh"]:::orchestr
        SaveLog["save_logits.sh"]:::orchestr
    end
    click SaveAct "https://github.com/ai45lab/reef/blob/master/src/scripts/save_activation.sh"
    click SaveLog "https://github.com/ai45lab/reef/blob/master/src/scripts/save_logits.sh"

    %% Extraction Services
    subgraph "Extraction Services"
        ActEx["Activation Extractor\ngenerate_activations.py"]:::compute
        HeadEx["Head Activation Extractor\ngenerate_head_activations.py"]:::compute
        LogitEx["Logit Extractor\nlogits.py"]:::compute
    end
    click ActEx "https://github.com/ai45lab/reef/blob/master/src/generate_activations.py"
    click HeadEx "https://github.com/ai45lab/reef/blob/master/src/generate_head_activations.py"
    click LogitEx "https://github.com/ai45lab/reef/blob/master/src/logits.py"

    %% Similarity Service
    CKACalc["CKA Calculator\ncompute_cka.py"]:::compute
    click CKACalc "https://github.com/ai45lab/reef/blob/master/src/compute_cka.py"

    %% Classifier Service
    subgraph "Classifier Service"
        ClsLin["Linear/MLP/CNN Trainer\ntrain_cls.py"]:::compute
        ClsGCN["GCN Trainer\ntrain_cls_gcn.py"]:::compute
        ClsTrans["Classifier Transfer\ntransfer_cls.py"]:::compute
    end
    click ClsLin "https://github.com/ai45lab/reef/blob/master/src/train_cls.py"
    click ClsGCN "https://github.com/ai45lab/reef/blob/master/src/train_cls_gcn.py"
    click ClsTrans "https://github.com/ai45lab/reef/blob/master/src/transfer_cls.py"

    %% Comparative Methods
    subgraph "Comparative Methods"
        PCS["PCS (Principal Component Similarity)\npcs.py"]:::comparative
        ICS["ICS (Instance Covariance Similarity)\nics.py"]:::comparative
    end
    click PCS "https://github.com/ai45lab/reef/blob/master/src/pcs.py"
    click ICS "https://github.com/ai45lab/reef/blob/master/src/ics.py"

    %% Visualization
    Viz["Visualization\nplot.ipynb"]:::viz
    click Viz "https://github.com/ai45lab/reef/blob/master/src/plot.ipynb"

    %% Utilities
    subgraph "Utilities"
        UtilsModel["Model Loader\nmodel.py"]:::util
        UtilsHelper["Helper Functions\nutils.py"]:::util
    end
    click UtilsModel "https://github.com/ai45lab/reef/blob/master/src/model.py"
    click UtilsHelper "https://github.com/ai45lab/reef/blob/master/src/utils.py"

    %% Data Flow
    DataSources -->|"raw CSV"| ActEx
    DataSources -->|"raw CSV"| LogitEx

    SaveAct --> ActEx
    SaveAct --> HeadEx
    SaveLog --> LogitEx

    ActEx -->|"activations.npy"| CKACalc
    HeadEx -->|"head_feats.npy"| CKACalc

    ActEx -->|"activations.npy"| ClsLin
    ActEx -->|"activations.npy"| ClsGCN
    ActEx -->|"activations.npy"| ClsTrans

    ActEx -->|"activations.npy"| PCS
    ActEx -->|"activations.npy"| ICS

    CKACalc -->|"cka_matrix.csv"| Viz

    ClsLin -->|"classifier.pt"| Viz
    ClsGCN -->|"gcn_model.pt"| Viz
    ClsTrans -->|"transferred.pt"| Viz

    PCS -->|"pcs_results.csv"| Viz
    ICS -->|"ics_results.csv"| Viz

    UtilsModel --> CKACalc
    UtilsHelper --> CKACalc
    UtilsModel --> ClsLin
    UtilsHelper --> ClsLin
    UtilsModel --> PCS
    UtilsHelper --> PCS

    classDef data fill:#AED6F1,stroke:#2E86C1,color:#1B4F72;
    classDef compute fill:#A9DFBF,stroke:#27AE60,color:#145A32;
    classDef comparative fill:#F9E79F,stroke:#D4AC0D,color:#7E5109;
    classDef viz fill:#D7BDE2,stroke:#8E44AD,color:#512E5F;
    classDef util fill:#E5E8E8,stroke:#B2BABB,color:#566573;
    classDef orchestr fill:#E5E8E8,stroke:#566573,color:#1C2833;
```
