How alibaba/tinyneuralnetwork works
Main Components
• Framework Core (tinynn): This is the heart of the project and contains several submodules:
⁃ Graph Module (tinynn/graph)
▪ Graph Tracer: Captures connectivity from PyTorch models.
▪ Graph Modifier: Adjusts and resolves dependencies after modifications.
▪ Quantization Submodule (tinynn/graph/quantization): Contains algorithms, fused modules, observers, QAT modules, and quantization utilities.
▪ Code Generation & Mask Management: Supports generation of model description files and operator masking.
⁃ Pruning Module (tinynn/prune)
▪ Implements various pruning algorithms (OneShot, ADMM, NetAdapt, etc.).
⁃ Converter Module (tinynn/converter)
▪ Converts floating-point and quantized PyTorch models to TFLite (and potentially other formats).
⁃ Utility Module (tinynn/util)
▪ Provides helper routines (BN restore, distributed utilities, train utilities, etc.).
⁃ LLM Quant Module (tinynn/llm_quant)
▪ Focus on quantizing large language models (with examples and tests specific to these workloads).
• Examples, Models & Tests:
⁃ Examples Directory: Demonstrates converters, quantization strategies, pruning examples, and basic end-to-end workflows.
⁃ Models Directory: Provides pre-trained models that serve as quick-start references.
⁃ Tests Directory: Contains unit tests ensuring the reliability and correctness of each module.
Relationships and Data Flows
• Incoming PyTorch models (from models/examples) are processed initially by the Graph Tracer, which captures the computational graph.
• Based on user goals (compression, quantization, or pruning), the workflow may branch:
▪ Pruner Module is engaged to apply one of the various pruning algorithms.
▪ Quantization algorithms (in tinynn/graph/quantization) are applied next. These modules interact closely with QAT (Quantization-Aware Training) processes.
• Once the model is modified, the Converter module handles conversion tasks (for example, from PyTorch to TFLite) to enable efficient deployment.
• Supplementary utility functions (from tinynn/util) assist throughout the pipeline, streamlining tasks such as optimization and distributed processing.
Architectural Patterns and Design Principles
• Modular Separation: Each major concern (tracing, modifying, pruning, quantizing, converting) is encapsulated in its own submodule.
• Pipeline/Data Flow: Represent a clear sequential or parallel data flow from model input to final converted model.
• Interaction with External Dependencies: Illustrate that TinyNeuralNetwork builds on PyTorch (for QAT and model representation) and supports Docker-based deployments (as shown in the Dockerfile).