How jepetau/pynnlib works
Main Components
• Core Library Components – At the top, you have modules such as architecture.py, capabilities.py, core.py, framework.py, and logger.py. These files represent the foundational functionality and utilities for the library.
• Model Handling and Abstraction Layer – The primary API is exposed through the “nnlib” interface (seen in the README examples using functions like open(), convert_to_tensorrt(), session(), and set_session_constructor()). This layer abstracts model initialization, conversion, and inference.
• Framework-Specific Backends – There are three major directories for backend implementations:
– nn_pytorch – Contains many subdirectories for different architectures (e.g., AnimeSR, BSVD, EfficientTAM, etc.). It implements model construction, session handling, and inference for PyTorch models.
– nn_onnx – Provides conversion and inference scripts for ONNX models.
– nn_tensor_rt – Contains similar files for building inference sessions via TensorRT.
• Scripts and Utility Tools – Under “scripts” and “utils”, there are helper scripts (like convert_model.py, parse_models.py, and img_infer.py) that demonstrate usage of the library and the conversion/inference workflow.
• Session Management – The README shows how custom inference sessions can be set up with the set_session_constructor() API. This abstraction also includes components managing device (e.g., CUDA, CPU) and data type conversions.
Relationships and Interactions
• User Code/Applications call the high-level API functions exposed by the core library (the “nnlib” interface) to load models, initiate conversions, and create inference sessions.
• The “open()” function delegates the task of model loading to underlying modules in the appropriate framework-specific subfolder (for example, within nn_pytorch, nn_onnx, etc.).
• If a conversion is required (e.g., converting a PyTorch model to TensorRT), the conversion functions (such as convert_to_tensorrt()) will interface with modules in the nn_tensor_rt directory and might also interact with common components in framework.py and import_libs.py.
• Inference sessions are created using the “session” API, which under the hood relies on the specific logic (found in nn_pytorch/inference, nn_onnx/inference, etc.) for transferring images/data and running execution.
• Script utilities (from the scripts folder) provide user-friendly command line interfaces to tasks such as model conversion, image inference, and model parsing. They serve as example orchestrations and can be depicted as “client” nodes that invoke the core library functions.
Architectural Patterns and Design Principles
• Abstraction & Modularity – The design is modular, with a clear separation between framework-agnostic core functionalities and framework-specific implementations. This allows the library to be easily integrated as a submodule and extended over time.
• Plug-in Mechanism for Sessions – The use of set_session_constructor() to plug in custom sessions for PyTorch and TensorRT shows that the library is designed with flexibility and extensibility in mind.
• Hierarchical Organization – The deep directory structure under nn_pytorch (and similarly under nn_onnx, nn_tensor_rt) represents the separation of concerns for different neural network architectures and converters.
• Submodule Integration – The README emphasizes that pynnlib is to be integrated as a submodule. This encourages a loosely coupled design where the internal API is not considered stable.
Specific Elements
• An additional block for “Scripts / CLI Tools” that shows utilities like parse_models, convert_model, and img_infer.
– API request flow from the core interface to the framework-specific modules.
– Data flow between model loading → conversion → inference session → output.