How tencent/wechat-tfcc works
Main Components
• TFCC Core Library (located in the “tfcc” directory):
– Implements the core deep learning functionalities.
– Contains submodules for allocators, dataloaders, exception handling, the framework (sessions, device management, tensor/variable handling, etc.), various operations (arithmetic, neural network, RNN, etc.), layers, and interfaces for abstracting different operations.
– Has separate implementations or interfaces to support both MKL (for CPU) and CUDA (for GPU) environments.
• TFCC Code Generator (in “tfcc_code_generator”):
– A toolchain that optimizes and converts high-level models from formats such as ONNX or TensorFlow into the TFCC model format.
– The backend part involves generating C++ code, using node generators, IR representations (using modules like ir, ir/optimizer, etc.), and runtime helpers.
• TFCC Runtime (in “tfcc_runtime”):
– Provides the infrastructure for loading and inferring a TFCC model.
– Contains components for model loading, operation execution, graph management, and communication/utilities.
Key Relationships and Interactions
• The Runtime component loads a TFCC model (as a string or binary blob) and interacts with the Core Library components to execute operations. Show a directional flow from “TFCC Runtime” to “TFCC Core Library.”
• The Core Library itself is internally modularized, interacting with low-level subcomponents in distinct layers:
– An “Interfaces/Operations Layer” supports both CPU (MKL) and GPU (CUDA) by abstracting backend-specific operations.
– “Exception Management,” “Dataloaders,” and “Allocators” provide foundational support.
• Third-party libraries (like xbyak in “3rd”) support dynamic code generation and may have utility roles especially in performance-critical parts. Represent these as auxiliary dependencies linked to the Core Library.
Architectural Patterns and Design Principles
• Modular design separating concerns: The core inference logic, code generation, and runtime loading are all independent modules that communicate in a pipeline.
• Abstraction for cross-platform execution: Multiple subdirectories under “tfcc” (and its “cuda” and “mkl” subdirectories) clearly show the use of the Strategy or Bridge pattern by abstracting operations through interfaces, enabling multiple backend implementations.
Technologies, Frameworks, and Tools
• C++ as the primary language, with CMake build scripts evident in many directories (e.g., tfcc/CMakeLists.txt, tfcc_code_generator).
• CUDA and MKL libraries to support GPU and high-performance CPU operations.
• Dynamic code generation utilities (e.g., xbyak) integrated into the system.
• Testing frameworks and sample code provided in “samples” and “tfcc/test”.