How neoml-lib/neoml works
Project Type and Purpose
• NeoML is an open-source ML framework (not a web or mobile app).
• Purpose: provide a C++-based algorithms library (NeoML), a separate math-engine module (NeoMathEngine), language wrappers (Python, Java/Kotlin, Objective-C/Swift), ONNX import support, and sample/test applications.
• Key use cases: training and inference of neural networks and traditional ML algorithms across CPU/GPU and multiple platforms (Windows, Linux, macOS, iOS, Android).
High-Level File Structure
• Top-level modules/directories:
• Build/ & CMake scripts – build orchestration
• NeoML/ – core algorithms (C++, wrappers, docs, samples, tests)
• NeoMathEngine/ – low-level compute backends (CPU, CUDA, Vulkan, Metal)
• NeoOnnx/ – ONNX importer
• NeoProxy/ – a lightweight proxy layer
• Samples (Python, Kotlin, Swift) and Test suites (unit tests, desktop/mobile)
• Notice layering: algorithms library → math-engine backends → language bindings → client/sample apps.
README Insights
• Two main C++ libraries: “NeoML” (algorithms) and “NeoMathEngine” (compute).
• Multiple build profiles: full C++ (training + inference) vs. inference only (Java/ObjC).
• External dependencies: CMake, Intel MKL, CUDA, Vulkan, Google Test, Protocol Buffers, xbyak.
Main Components
a. NeoML Algorithms Library
– Neural networks (100+ layer types)
– Traditional ML (20+ algorithms)
– ONNX importer (NeoOnnx)
b. NeoMathEngine Backends
– CPU (plain C++, MKL)
– GPU: CUDA, Vulkan, Metal
– Plugin/strategy pattern for engine selection
c. Language Bindings
– Python wrapper (SWIG/C++ extension via setup.py)
– Java/Kotlin JNI inference API
– Objective-C/Swift inference API
d. Build & Packaging
– CMake scripts, build pipelines (Azure DevOps & shell/cmd scripts)
– Output artifacts: libNeoML, libNeoMathEngine, Python wheel, JAR, frameworks
e. Client/Sample Applications
– Python scripts (tutorials)
– Android (Java/Kotlin) demo
– iOS (Swift/ObjC) demo
– Desktop “full test” app
f. External Services/Dependencies
– Intel MKL, CUDA runtime, Vulkan SDK, Protocol Buffers, xbyak JIT
Relationships & Data Flows
• Algorithms Library invokes MathEngine through a common interface.
• At runtime the application selects a MathEngine implementation (CPU or GPU).
• Language bindings call into the C++ libraries via JNI or C-API.
• ONNX importer feeds a graph into the NeoML library before executing with a MathEngine.
• Build system coordinates compilation of all components and links external libraries.
Architectural Patterns & Principles
• Layering: Presentation (samples/bindings) → Business logic (NeoML) → Compute (NeoMathEngine)
• Plugin/Strategy for math engines
• Separation of interface vs. implementation for cross-platform portability
• Bridge pattern in language bindings (C++ core bridged to Python, Java, ObjC)
• Components: NeoML core, NeoMathEngine variants, NeoOnnx, NeoProxy, Language wrappers, Samples
• Build orchestration: CMake & scripts → artifacts (shared/static libs, wheels, JARs, frameworks)
• External libraries/services