How pym1024/swap works
Project type and purpose
• This is a research tool/library for computing the SWAP‐Score—a zero‐cost proxy metric that predicts neural network performance without full training.
• It provides:
• A command‐line entrypoint (correlation.py) to compute correlations between SWAP‐Score and actual CIFAR-10 accuracies.
• Core metric implementation under src/metrics/swap.py.
• Utilities for data loading, search‐space definition, and generic helper functions.
File structure
• Top‐level:
• correlation.py – script to read a precomputed CSV of architectures+accuracies, compute SWAP scores, and output correlations.
• datasets/ – contains the CSV file DARTS_archs_CIFAR10.csv for 1,000 architectures and results.
• requirements.txt – Python dependencies.
• src/ package decomposition:
• src/datasets/ – modules to load CIFAR-10 or Downsampled ImageNet data (DownsampledImageNet.py, generic utils).
• src/search_space/ – definitions of architectural primitives (operations.py, networks.py).
• src/metrics/ – core SWAP‐Score logic (swap.py).
• src/utils/ – shared helper functions.
README for architectural hints
• Emphasizes that src/metrics/swap.py contains the core algorithm.
• The pipeline: install dependencies → run python correlation.py → measure correlation between SWAP and true accuracies.
a. Main components
• Entrypoint: correlation.py
• Data sources:
• datasets/DARTS_archs_CIFAR10.csv
• (Optional) Downsampled ImageNet loader in src/datasets
• Search‐space module (src/search_space)
• SWAP‐Score metric module (src/metrics/swap.py)
• Utility functions (src/utils)
• Output: correlation results (console/table/graph)
b. Component interactions
1. correlation.py reads CSV → hands off architecture definitions and true accuracies.
2. correlation.py invokes search_space to reconstruct each model’s graph and parameters.
3. correlation.py calls the SWAP module, passing models + sample data loaders → computes SWAP‐Scores.
4. correlation.py computes statistical correlation between SWAP‐Scores and true accuracies.
5. Results are printed or saved.
c. Architectural patterns
• Modular package structure separating concerns: data ingestion, search‐space abstraction, metric computation, and orchestration.
• Pipeline pattern: linear data flow from entrypoint → loader → model definition → metric computation → analysis.
d. Key technologies
• Python 3.x
• pandas / numpy (for CSV and data manipulation)
• PyTorch or NumPy (within swap.py for activation hooks and pattern extraction)
• Virtual environment via requirements.txt