How pipikai/android-ncnn-yolo11 works
This project is an Android application that integrates both a Java/Kotlin layer and a native C++ layer for running the YOLO object detection model using Tencent’s ncnn framework. In other words, it is a mobile application that performs computer vision inference (YOLOv11) on-device using native libraries and leverages OpenCV for some image processing tasks.
Overall Project Tiers
• Android Presentation Layer: The UI and interaction are handled by Android activities (e.g., MainActivity.kt) and related resource files.
• JNI/Bridge Layer: The Kotlin class (YoloNcnn.kt) acts as a bridge and provides an interface between the Java/Kotlin code and the C++ native code through JNI calls.
• Native Inference Engine: C++ source files (native-lib.cpp, yolo11.cpp, yolo11.h) and accompanying CMake lists build and integrate the ncnn libraries. This native portion utilizes prebuilt ncnn binaries/libraries (included per architecture – arm64-v8a, armeabi-v7a, x86, x86_64) and OpenCV (provided via the opencv-mobile subfolder).
• Assets: The app/src/main/assets folder holds the YOLO model files (e.g., .bin, .param files) that are loaded at runtime.
Detailed Architectural Components
A. Android Frontend:
• MainActivity.kt: The main entry point for the Android UI.
• Layout files (activity_main.xml): Define the UI.
B. JNI/Interface Layer:
• YoloNcnn.kt: A Kotlin class that exposes methods to perform inference by calling the native (JNI) functions.
• Gradle build configurations (build.gradle, settings.gradle): Configure the Android build and integration of the native libraries.
C. Native Processing Layer:
• C++ sources in app/src/main/cpp (including native-lib.cpp and yolo11.cpp/h): Implement the core inference logic.
• CMakeLists.txt: Used to compile native code and link the appropriate libraries.
• ncnn library integration:
• The ncnn libraries are provided in a structured way for different CPU architectures, along with support files and configuration scripts (the directories labeled ncnn-20240410-android-vulkan for arm64-v8a, armeabi-v7a, x86, x86_64).
• OpenCV Mobile integration:
• The opencv-mobile-2.4.13.7-android folder contains precompiled OpenCV libraries and header files used to support image processing functions.
D. Build and Configuration:
• Gradle (build.gradle, gradlew files) and CMake files coordinate the build process.
• Assets and configuration files (AndroidManifest.xml, proguard-rules.pro) define application behavior and security.
Relationships and Data Flow
• The Android UI (MainActivity) initiates detection by invoking methods in YoloNcnn.
• YoloNcnn, via JNI, sends image data (or references) to native methods.
• The native C++ layer (native-lib.cpp/yolo11.cpp) loads models from the assets folder and utilizes ncnn (and optionally OpenCV) to perform inference.
• The results of the inference are passed back through the JNI bridge to be displayed or used in the UI.
Architectural Patterns and Technologies
• Layer Separation: Clear division between presentation (Java/Kotlin), business logic/bridge (JNI), and computational backend (C++ with ncnn).
• Modular Build: Use of Gradle for Android/framework build management and CMake for native build integration.
• Multi-architecture Support: Native library directories show support for various CPU architectures (ARM, x86, etc.), highlighting the emphasis on performance portability.
• Use of External Libraries: ncnn (from Tencent) and OpenCV for machine learning inference and image processing.