How mit-han-lab/mcunet works
Project Type & Purpose
• The project is focused on enabling deep learning on microcontrollers under tight memory constraints. It is a research and reference implementation that combines a neural architecture search / network design component (TinyNAS) with a lightweight inference engine (TinyEngine).
• The README makes clear that the framework is about “tiny deep learning” with components to build, evaluate, and deploy both PyTorch and TF-Lite models on low-memory devices.
Analyzing the File Structure
• At the top level, you see configuration files (e.g., .github, setup.py, requirements.txt) which support building, continuous integration, and deployment.
• The primary code resides in the “mcunet” directory. Within it, there are two key submodules:
– TinyNAS (under mcunet/tinynas): Contains directories like elastic_nn, elastic_nn/modules, elastic_nn/networks, nn (with its own modules/networks), and a separate TensorFlow codebase. This is where the model design/search and structural modules are implemented.
– TinyEngine (under mcunet/tinyengine): Dedicated for the inference engine which is used during deployment and evaluation of models on microcontrollers.
• The “model_zoo.py” in mcunet points to a registry or factory of available models, and the eval scripts (eval_det.py, eval_tflite.py, eval_torch.py) provide command-line drivers to test and benchmark the models.
Key Components
• MCUNet Core: Represent this as the central orchestrator which ties together the model construction, training or architecture search (TinyNAS), and deployment/inference (TinyEngine).
• TinyNAS Module: Show components such as “elastic_nn” and “nn” modules. Detail that these submodules contain dynamic layers, operations, and network architectures (like OFA Proxyless and MobileNet V2 variants) for automatic network design.
• TinyEngine Module: Emphasize its role in model inference – it is an efficient engine designed for microcontrollers that improves speed and reduces memory usage.
• Model Zoo: Include a component that aggregates model definitions and configurations (from “mcunet/model_zoo.py”), and ties into the build_model and download_tflite methods.
• Evaluation / Test Drivers: Highlight the presence of evaluation scripts for both PyTorch and TF-Lite, as well as the demo for detection (eval_det.py). These act as entry-points/scripts to trigger experiments.
• Utility Libraries: Represent the “mcunet/utils” component, which contains common helper functions, configuration utilities, and module definitions that support the functionality of TinyNAS and TinyEngine.
Interactions & Data Flow
• From a “User / Experiment” input (or CI/CD pipeline from .github/workflows) to the Evaluation Scripts.
• From Evaluation Scripts → Model Zoo: They call build_model() or download_tflite() to obtain network models.
• From Model Zoo → TinyNAS: For models that require architecture search, dynamic layer configuration, or network construction.
• Additionally, the Utility libraries support the above modules (TinyNAS and TinyEngine) with functions like batch normalization utilities, PyTorch module wrappers, etc.