# modeltc/l2_compression architecture

> Architecture diagram of the GitHub repository modeltc/l2_compression, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/modeltc/l2_compression
- Repository: https://github.com/modeltc/l2_compression

## Overview

##### Main Components

- Data Processing and Datasets
- The directory data_new provides modules for loading and preprocessing datasets (e.g., ImageNet). This is critical since the training scripts (found in the train directory) rely on these datasets.

- Model Architectures
- Two directories—cifar10_models and model—contain various network architectures (ResNet, MobileNetV2, DenseNet, etc.). These represent the selectable deep learning models on which the compression will be applied.

- Compression and Quantization Algorithms
- The compress directory holds modules for lossless compression algorithms (Huffman coding and range coding).
- The quant directory contains quantization routines that adjust weight precision (e.g., soft quant methods).

- Training and Calibration Pipelines
- The train directory includes training scripts, loggers, and experiment modules for preparing the models and running calibration. This module processes checkpoints and facilitates adjustments like weight transformations.
- The README examples point to train/train_final, which is the main entry point for adapting model checkpoints and calibrating datasets.

- Transformation Operations
- The transform directory contains various weight transformation functions (e.g., edge_scale, log, scale) that are applied during the compression process to achieve the L^2 compression objectives.

- Compiled Acceleration Libraries
- The clib directory aggregates several performance-critical routines implemented in C++ and CUDA (organized into cos_stat, lin_stat, and tri_stat subfolders). These are used to offload heavy computation tasks in functions critical for compression or statistics collection.

- Utility and Helper Functions
- The common directory contains shared code and tools that are used across modules.
- Additionally, cal_bit.py and cal_bit_huffman.py are provided as scripts for encoding/decoding and testing the compression operations.

##### Out Relationships and Data Flow

- Input data (e.g., ImageNet images) is loaded and preprocessed from the data_new modules.

- Model Definition and Selection:
- The appropriate model (from cifar10_models or model) is loaded and instantiated. The training scripts interact with these classes.

- Training and Calibration Pipeline:
- The train directory orchestrates the training or fine-tuning process. The trainer reads model checkpoints, configures training hyperparameters, and makes calls to both transformation and quantization modules. Calibration data (as specified in train.txt) is used to adjust parameters.

- Weight Transformation and Compression:
- The transform modules apply specific mathematical transformations (edgescale, scale, etc.) to the model weights.
- Next, the quant module further processes these weights (through quantization methods like soft quantization).
- The compress modules handle the final encoding (using Huffman or range coding) to get the compressed representation.

- Integration of Compiled Libraries:
- Specialized routines in clib (implemented in both C++ and CUDA) are invoked by higher-level Python modules for efficient computation of statistical metrics (e.g., cosine, linear, or triangular statistical measures) required during weight transformation and compression.

##### Architectural Patterns and Key Design Principles

- Modularity and Separation of Concerns:
- The project clearly separates concerns: model definition, data management, training, transformation, quantization, and compression are all handled by distinct modules.

- Layered Architecture:

- Extensibility:
- With the distinct directories for models and compression techniques, the design allows for replacing or extending certain components (e.g., adding new model architectures or compression algorithms) without major rework.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Pre-processing Layer
    subgraph "Pre-Processing Layer"
        dp["Data Processing (data_new)"]:::data
        cal["Calibration Data (train.txt)"]:::data
    end

    %% Model Layer
    subgraph "Model Architectures"
        cifar["CIFAR10 Models (cifar10_models)"]:::model
        mod["Model Architectures (model)"]:::model
    end

    %% Training Pipeline
    subgraph "Training Pipeline"
        train["Training Pipeline (train)"]:::training
    end

    %% Post-Training Compression Pipeline
    subgraph "Post-Training Compression Pipeline"
        trans["Weight Transformation (transform)"]:::transform
        quant["Quantization (quant)"]:::quant
        comp["Lossless Compression (compress)"]:::compress
    end

    %% Support Layer
    subgraph "Support Layer"
        clib["Acceleration Libraries (clib)"]:::clib
        util["Utility Functions (common)"]:::utility
        cs["Compression Scripts (cal_bit)"]:::utility
    end

    %% Data Flow & Relationships
    dp -->|"provides data"| train
    cal -->|"calibration data"| train

    cifar -->|"model selection"| train
    mod -->|"model selection"| train

    train -->|"triggers transformation"| trans
    trans -->|"feeds weights"| quant
    quant -->|"prepares bit stream"| comp

    train -->|"uses for compute"| clib
    trans ---|"accelerated by"| clib
    quant ---|"accelerated by"| clib

    train -->|"supports via"| util
    cs -->|"validates encoding"| comp

    %% Class Styles
    classDef data fill:#A5D6A7,stroke:#388E3C,color:#000;
    classDef model fill:#BBDEFB,stroke:#1976D2,color:#000;
    classDef training fill:#FFE082,stroke:#FFA000,color:#000;
    classDef transform fill:#E1BEE7,stroke:#8E24AA,color:#000;
    classDef quant fill:#F48FB1,stroke:#C2185B,color:#000;
    classDef compress fill:#D1C4E9,stroke:#673AB7,color:#000;
    classDef clib fill:#80DEEA,stroke:#00838F,color:#000;
    classDef utility fill:#FFF59D,stroke:#F9A825,color:#000;

    %% Click Events
    click dp "https://github.com/modeltc/l2_compression/blob/main/data_new/__init__.py"
    click cifar "https://github.com/modeltc/l2_compression/blob/main/cifar10_models/resnet.py"
    click mod "https://github.com/modeltc/l2_compression/blob/main/model/resnet.py"
    click train "https://github.com/modeltc/l2_compression/blob/main/train/train_final.py"
    click cal "https://github.com/modeltc/l2_compression/blob/main/train.txt"
    click trans "https://github.com/modeltc/l2_compression/blob/main/transform/edge_scale.py"
    click quant "https://github.com/modeltc/l2_compression/blob/main/quant/soft_quant.py"
    click comp "https://github.com/modeltc/l2_compression/blob/main/compress/huffman.py"
    click clib "https://github.com/modeltc/l2_compression/tree/main/clib/cos_stat"
    click util "https://github.com/modeltc/l2_compression/blob/main/common/__init__.py"
    click cs "https://github.com/modeltc/l2_compression/blob/main/cal_bit.py"
```
