owned this note
owned this note
Published
Linked with GitHub
# Integrating NCNN on Android using CMake
###### tags: `android` `ncnn` `deep learning`
## Pre-Requirement
* Android Studio
* Please install Android SDK and NDK packages. (see [Android Native Development Kit (NDK)](https://hackmd.io/WO-nZYsfR1icmYbWy3jFrQ))
* You may also want to import OpenCV library in your native code. (see [How to Import OpenCV in Android NDK](https://hackmd.io/pdIcTgr7RuKGlYjwaGDwhw))
* NCNN (see [How to Build NCNN on Mac](https://hackmd.io/4i6x5TeFRJKrq2s5XjgPBg?view))
:::danger
**Please be careful that if you add a custom layer into your NCNN source code, the library may not be used for the model that is transfered from the original source code (building before add the custom layer), even you don't have the custom layer in your model.**
:::
## Demo APP - NcnnDemo
### Build NCNN for Android
* please also refer to [How to Build NCNN on Mac](https://hackmd.io/4i6x5TeFRJKrq2s5XjgPBg?view)
* build
```
> mkdir build-android
> cd build-android
> cmake -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK_HOME/build/cmake/android.toolchain.cmake \
-DANDROID_ABI="armeabi-v7a" -DANDROID_ARM_NEON=ON \
-DANDROID_PLATFORM=android-14 ..
> make
> make install
```
* After successfully building, you should find *libncnn.a* is generated in /ncnn/build-android/install/lib
:::danger
please make sure that your $ANDROID_NDK_HOME is set to your ndk folder
:::
* you can also use the prebuild .a file in https://github.com/Tencent/ncnn/releases, but recommend to build on your own environment.
### Create a New Project
* Enable C++ support
![](https://i.imgur.com/0UKv3gO.png)
* Enable Exception and RTTI Support
![](https://i.imgur.com/tAocgnd.png)
### Import NCNN Libraries
* Copy the folder (/ncnn/build-android/install/include) to /app/src/main/cpp/
![](https://i.imgur.com/PnOU27j.png)
* Copy /ncnn/build-android/install/lib/libncnn.a to /app/src/main/cpp/jniLibs/armeabi-v7a (if fold not exists, create one)
* Modify CmakeList.txt in Android
* set the library path and its properties.
```
include_directories(${CMAKE_SOURCE_DIR}/src/main/cpp/include)
add_library(
ncnn
STATIC
IMPORTED
)
set_target_properties(
ncnn
PROPERTIES IMPORTED_LOCATION
${CMAKE_CURRENT_LIST_DIR}/src/main/jniLibs/${ANDROID_ABI}/libncnn.a
)
```
* link all the library we need
```
target_link_libraries( # Specifies the target library.
native-lib
# Links the target library to the log library
# included in the NDK.
ncnn
jnigraphics
${android-lib}
${log-lib} )
```
* Modify the build.gradle(Module:app)
```
defaultConfig {
applicationId "com.jd.ncnndemo"
minSdkVersion 23
targetSdkVersion 27
versionCode 1
versionName "1.0"
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
externalNativeBuild {
cmake {
cppFlags "-frtti -fexceptions"
}
}
ndk {
// Specifies the ABI configurations of your native
// libraries Gradle should build and package with your APK.
abiFilters 'armeabi-v7a' // new line
}
}
```
* Rebuild the project and it is supposed to generate several C++ libraries in your cpp folder.
![](https://i.imgur.com/1QVykk9.png =400x450)
* Then, we can include NCNN libraries in our native code (native-lib.cpp).
```cpp=
// native-lib.cpp
#include "net.h"
```
### Load Model
* Put our model, *ncnnmodel.bin* and *ncnnmodel.param.bin* ,in /app/src/main/assets folder (create one if not exists).
![](https://i.imgur.com/xxUPKBV.png)
* Put *ncnnmodel.id.h* into /app/src/main/cpp.
![](https://i.imgur.com/4gBXr0m.png)
* Declare our model
```cpp=
// native-lib.cpp
static std::vector<unsigned char> model_param;
static std::vector<unsigned char> model_bin;
static ncnn::Net model;
```
* Define the native function
```cpp=
// native-lib.cpp
extern "C" JNIEXPORT jboolean
JNICALL
Java_com_jd_ncnndemo_NCNN_initNCNN(JNIEnv* env, jobject thiz, jbyteArray param, jbyteArray bin)
{
// init param
{
int len = env->GetArrayLength(param);
model_param.resize(len);
env->GetByteArrayRegion(param, 0, len, (jbyte*)model_param.data());
int ret = model.load_param(model_param.data());
alog("load_param %d %d", ret, len);
}
// init bin
{
int len = env->GetArrayLength(bin);
model_bin.resize(len);
env->GetByteArrayRegion(bin, 0, len, (jbyte*)model_bin.data());
int ret = model.load_model(model_bin.data());
alog("load_model %d %d", ret, len);
}
return JNI_TRUE;
}
```
* Declare the native function in java
```java=
//NCNN.java
public native static boolean initNCNN(byte[] param, byte[] bin);
```
* Init the model
```java=
// NCNN.java
protected static void init(Context context) throws IOException
{
byte[] param = null;
byte[] bin = null;
{
// load .param.bin file
InputStream assetsInputStream = context.getAssets().open("ncnnmodel.param.bin");
int available = assetsInputStream.available();
param = new byte[available];
assetsInputStream.read(param);
assetsInputStream.close();
}
{
// load .bin file
InputStream assetsInputStream = context.getAssets().open("ncnnmodel.bin");
int available = assetsInputStream.available();
bin = new byte[available];
assetsInputStream.read(bin);
assetsInputStream.close();
}
// call native function to init the model
initNCNN(param, bin);
}
```
* call from MainActivity.java
```java=
// init model
try {
NCNN.init(this);
} catch (IOException e) {
e.printStackTrace();
}
```
### Model Prediction
```cpp=
//nativ-lib.cpp
#include "ncnnmodel.id.h"
extern "C" JNIEXPORT void
JNICALL
Java_com_jd_ncnndemo_NCNN_NcnnProcess(
JNIEnv *env,
jobject /* this */) {
// image preprocessing
// ...
// ...
// ncnn process
/*
ncnn::Extractor ex = model.create_extractor();
ex.set_light_mode(true);
ex.set_num_threads(4);
//input
ncnn::Mat input(IMG_INPUT_WIDTH,IMG_INPUT_HEIGHT,3,(float*)input_img);
ex.input(ncnnmodel.param_id::BLOB_data, input);
//output
ncnn::Mat out;
ex.extract(ncnnmodel_param_id::BLOB_prob, out);
*/
}
```
```java=
// NCNN.java
public native static void NcnnProcess();
```
* call from MainActivity.java button click event
```java=
// MainActivity.java
btn_proc = findViewById(R.id.btn_proc);
btn_proc.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
// Example of a call to a native method
NCNN.NcnnProcess();
}
});
```
## Problem Shooting
### error: undefined reference to 'omp_xxxxx' or '__kmpc_xxxxx'
* Error Message
```
Build command failed.
Error while executing process /Users/xxxxx/Library/Android/sdk/cmake/3.6.4111459/bin/cmake with arguments {--build /Users/xxxxx/Projects/NcnnDemo/app/.externalNativeBuild/cmake/debug/armeabi-v7a --target native-lib}
[1/2] Building CXX object CMakeFiles/native-lib.dir/src/main/cpp/native-lib.cpp.o
[2/2] Linking CXX shared library ../../../../build/intermediates/cmake/debug/obj/armeabi-v7a/libnative-lib.so
FAILED: : && /Users/xxxxxx/Library/Android/sdk/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++ --target=armv7-none-linux-androideabi --gcc-toolchain=/Users/xxxxxx/Library/Android/sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64 --sysroot=/Users/xxxxxx/Library/Android/sdk/ndk-bundle/sysroot -fPIC -isystem /Users/xxxxxx/Library/Android/sdk/ndk-bundle/sysroot/usr/include/arm-linux-androideabi -D__ANDROID_API__=23 -g -DANDROID -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -fno-integrated-as -mthumb -Wa,--noexecstack -Wformat -Werror=format-security -frtti -fexceptions -O0 -fno-limit-debug-info -Wl,--exclude-libs,libgcc.a --sysroot /Users/xxxxxxxx/Library/Android/sdk/ndk-bundle/platforms/android-23/arch-arm -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--fix-cortex-a8 -Wl,--no-undefined -Wl,-z,noexecstack -Qunused-arguments -Wl,-z,relro -Wl,-z,now -shared -Wl,-soname,libnative-lib.so -o ../../../../build/intermediates/cmake/debug/obj/armeabi-v7a/libnative-lib.so CMakeFiles/native-lib.dir/src/main/cpp/native-lib.cpp.o ../../../../src/main/jniLibs/armeabi-v7a/libncnn.a -ljnigraphics -llog -lm "/Users/xxxxxxxx/Library/Android/sdk/ndk-bundle/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi-v7a/libgnustl_static.a" && :
/Users/xxxxxxxx/Desktop/ncnn/src/net.cpp:822: error: undefined reference to 'omp_get_dynamic'
/Users/xxxxxxxx/Desktop/ncnn/src/net.cpp:823: error: undefined reference to 'omp_get_num_threads'
/Users/xxxxxxxx/Desktop/ncnn/src/net.cpp:824: error: undefined reference to 'omp_set_dynamic'
/Users/xxxxxxxx/Desktop/ncnn/src/net.cpp:825: error: undefined reference to 'omp_set_num_threads'
/Users/xxxxxxxx/Desktop/ncnn/src/net.cpp:834: error: undefined reference to 'omp_set_dynamic'
/Users/xxxxxxxx/Desktop/ncnn/src/net.cpp:835: error: undefined reference to 'omp_set_num_threads'
/Users/xxxxxxxx/Desktop/ncnn/src/net.cpp:874: error: undefined reference to 'omp_get_dynamic'
/Users/xxxxxxxx/Desktop/ncnn/src/net.cpp:875: error: undefined reference to 'omp_get_num_threads'
/Users/xxxxxxxx/Desktop/ncnn/src/net.cpp:876: error: undefined reference to 'omp_set_dynamic'
/Users/xxxxxxxx/Desktop/ncnn/src/net.cpp:877: error: undefined reference to 'omp_set_num_threads'
/Users/xxxxxxxx/Desktop/ncnn/src/net.cpp:886: error: undefined reference to 'omp_set_dynamic'
/Users/xxxxxxxx/Desktop/ncnn/src/net.cpp:887: error: undefined reference to 'omp_set_num_threads'
/Users/xxxxxxxx/Desktop/ncnn/src/layer/arm/absval_arm.cpp:32: error: undefined reference to '__kmpc_fork_call'
/Users/xxxxxxxx/Desktop/ncnn/src/layer/arm/absval_arm.cpp:33: error: undefined reference to '__kmpc_for_static_init_4'
/Users/xxxxxxxx/Desktop/ncnn/src/layer/arm/absval_arm.cpp:32: error: undefined reference to '__kmpc_for_static_fini'
/Users/xxxxxxxx/Desktop/ncnn/src/layer/arm/batchnorm_arm.cpp:41: error: undefined reference to '__kmpc_fork_call'
/Users/xxxxxxxx/Desktop/ncnn/src/layer/arm/batchnorm_arm.cpp:42: error: undefined reference to '__kmpc_for_static_init_4'
```
* Solution : add a cppFlag '-fopenmp' in build.gradle(Module:app)
```
defaultConfig {
applicationId "com.jd.ncnndemo"
minSdkVersion 23
targetSdkVersion 27
versionCode 1
versionName "1.0"
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
externalNativeBuild {
cmake {
cppFlags "-frtti -fexceptions -fopenmp" //new line
}
}
ndk {
// Specifies the ABI configurations of your native
// libraries Gradle should build and package with your APK.
abiFilters 'armeabi-v7a'
}
}
```
### clang++ error : unknown argment '-nostdlib'
* Solution : add a cppFlag '-std=c++11' in build.gradle(Module:app)
```
defaultConfig {
applicationId "com.jd.ncnndemo"
minSdkVersion 23
targetSdkVersion 27
versionCode 1
versionName "1.0"
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
externalNativeBuild {
cmake {
cppFlags "-frtti -fexceptions -std=c++11" //new line
}
}
ndk {
// Specifies the ABI configurations of your native
// libraries Gradle should build and package with your APK.
abiFilters 'armeabi-v7a'
}
}
```
## Reference
* [NCNN源码编译及Android Demo跑通](https://www.jianshu.com/p/f6b7bbb4fa96)
* [在ncnn上把玩mobileNet](https://blog.csdn.net/computerme/article/details/77876633)
* [Ncnn使用详解(2)——Android端](https://blog.csdn.net/qq_36982160/article/details/79931741)
* [深度学习【19】ncnn安卓搭建并使用自己的模型](https://blog.csdn.net/linmingan/article/details/77988382)