How ml-security-research-lab/deepcompress-vit works
Project type and purpose
• DeepCompress-ViT is a research codebase for compressing and fine-tuning Vision Transformer (ViT) models and running inference at the edge.
• It provides two CLI entrypoints: one for training compressed models (main.py) and one for inference/evaluation (inference.py).
• It uses PyTorch/timm, implements quantization, autoencoder-based compression, and distillation.
File structure
• Top level:
• main.py – training CLI driver
• inference.py – inference/evaluation CLI driver
• requirements.txt – dependency list
• saved_models/ (populated at runtime with checkpoints)
• src/
• models/ – definitions of ViT, autoencoder, quantization, encoder modules
• training/engine.py – training loop, optimizer, scheduler, distillation logic
• utils/ – data loading, metrics calculation, model-save/load helpers
README for additional insights
• Training steps: select model_name, rank, distillation weight, mixed_precision, device, etc.
• Checkpoint storage on Google Drive and local file system.
• Inference supports different datasets (ImageNet vs. CIFAR-10).