Mediapipe on Android Studio complete setup tutorial

This is a guide for running an basic hand tracking example of Mediapepe installed on Windows.

Notices

• This was build on April 2023.
• It is not guaranteed to run successfully on newer mediapipe versions.
• If you encounter some errors reading the official document might be helpful.
• WSL version: Ubuntu 22.04

Steps

1. Install Windows Subsystem for Linux (WSL)
2. Setting up Android SDK and NDK
3. Build a MediaPipe Framework AAR
4. Running it on Android Studio (on your Windows PC)

Install Windows Subsystem for Linux (WSL)

Open PowerShell in administrator mode by right-clicking and selecting "Run as administrator", enter the wsl –install command, then restart your machine.

wsl --install

Run wsl –list –online to see a list of available distros.

wsl --list --online

Run wsl –install -d Ubuntu to install Ubuntu.

wsl --install -d Ubuntu

Once you have installed, you will need to create a user account and password.

Update apt-get and install some necessary dependencies. This might takes few minutes.

sudo apt-get update && sudo apt-get install -y build-essential git python3 zip adb openjdk-8-jdk

Install adb on WSL and check what version you have installed.

sudo apt-get install android-tools-adb
adb version

Install adb on Windows

• Download the same adb version to your Windows PC from this.
• Unzip it to get adb.exe, AdbWinApi.dll and AdbWinUsbApi.dll, then copy these 3 files to C:\Windows\System32 and C:\Windows\SysWOW64.
• Open PowerShell and execute adb version to check the adb version is same with WSL(Ubuntu).

Install Bazel on WSL

curl -sLO --retry 5 --retry-max-time 10 https://storage.googleapis.com/bazel/3.1.0/release/bazel-3.1.0-installer-linux-x86_64.sh && \
sudo mkdir -p /usr/local/bazel/3.1.0 && \
chmod 755 bazel-3.1.0-installer-linux-x86_64.sh && \
sudo ./bazel-3.1.0-installer-linux-x86_64.sh --prefix=/usr/local/bazel/3.1.0 && \
source /usr/local/bazel/3.1.0/lib/bazel/bin/bazel-complete.bash /usr/local/bazel/3.1.0/lib/bazel/bin/bazel version && \
alias bazel='/usr/local/bazel/3.1.0/lib/bazel/bin/bazel'

Put alias bazel='/usr/local/bazel/3.1.0/lib/bazel/bin/bazel to /etc/profile at the end.

If you don't know how to use vim, check this.

sudo vim /etc/profile

source /etc/profile     # run it to make the new /etc/profile work

Download mediapipe to WSL

git clone https://github.com/google/mediapipe.git

Install OpenCV and FFmpeg

cd mediapipe

sudo apt-get install libopencv-core-dev libopencv-highgui-dev \
                       libopencv-calib3d-dev libopencv-features2d-dev \
                       libopencv-imgproc-dev libopencv-video-dev

Run the Hello World! in C++ example

export GLOG_logtostderr=1 # Need bazel flag 'MEDIAPIPE_DISABLE_GPU=1' as desktop GPU is currently not supported


bazel run --define MEDIAPIPE_DISABLE_GPU=1 \
    mediapipe/examples/desktop/hello_world:hello_world

# Should print:
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!

Setting up Android SDK and NDK

Run setup_android_sdk_and_ndk.sh file to download sdk and ndk

Note : It might takes few hours.

cd mediapipe

chmod 755 setup_android_sdk_and_ndk.sh

sudo ./setup_android_sdk_and_ndk.sh

Add the path to /etc/profile file at the end

export ANDROID_HOME=$PATH<your path of Android SDK>
export ANDROID_NDK_HOME=$PATH<your path of Android NDK>

# for example
export ANDROID_HOME=$PATH:/home/username/Android/Sdk
export ANDROID_NDK_HOME=$PATH:/home/username/Android/Sdk/ndk-bundle/android-ndk-r18b

source /etc/profile     # run it to make the new /etc/profile work

Build a MediaPipe Framework AAR

New a handtrackinggpu_aar folder

mkdir mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu_aar

New the assets and libs folder

mkdir mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu_aar/assets

mkdir mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu_aar/libs

Create a BUILD file under handtrackinggpu_aar

vim BUILD

Modify the BUILD file

load("//mediapipe/java/com/google/mediapipe:mediapipe_aar.bzl", "mediapipe_aar")

mediapipe_aar(
             name = "mediapipe_hand_tracking",
             calculators = ["//mediapipe/graphs/hand_tracking:mobile_calculators"],
    )

Run the Bazel build command to generate the AAR.

cd mediapipe

chmod -R 755 mediapipe/

bazel build -c opt --strip=ALWAYS    \
    --host_crosstool_top=@bazel_tools//tools/cpp:toolchain     \
    --fat_apk_cpu=arm64-v8a,armeabi-v7a     \
    --legacy_whole_archive=0     \
    --features=-legacy_whole_archive     \
    --copt=-fvisibility=hidden     \
    --copt=-ffunction-sections     \
    --copt=-fdata-sections     \
    --copt=-fstack-protector     \
    --copt=-Oz     \
    --copt=-fomit-frame-pointer     \
    --copt=-DABSL_MIN_LOG_LEVEL=2     \
    --linkopt=-Wl,--gc-sections,--strip-all         \
    //mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu_aar:mediapipe_hand_tracking

Copy the AAR into app/libs

Note: change the 'username' to your username

cp bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu_aar/mediapipe_hand_tracking.aar //home/username/mediapipe/mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu_aar/libs

Build the MediaPipe binary graph

bazel build -c opt mediapipe/graphs/hand_tracking:hand_tracking_mobile_gpu_binary_graph

Copy the binary graph to assets

Note: change the 'username' to your username

cp bazel-bin/mediapipe/graphs/hand_tracking/hand_tracking_mobile_gpu.binarypb //home/username/mediapipe/mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu_aar/assets

Running it on Android Studio

Create a new project on Android Studio

Note : on Windows not on the WSL

Download jniLibs (whole directory) and copy it to your Android Studio project

Download link

Copy some files to your Android Studio project

• mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu_aar/libs —> /app
• mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu_aar/assets —> /app/src/main
• handedness.txt—> /app/src/main/assets
• All .tflite in assets—> /app/src/main/assets
• AndroidManifest.xml —> /app/src/main
• MainActivity.java —> /app/src/main/java/com/example/myapplication
• style.xml —>/app/src/main/res/values/
• activity_main.xml—>/app/src/main/res/layout/

Modify app/build.gradle

Modify the minSdk to 21

defaultConfig {
        applicationId "com.example.myapplication"
        minSdk 21
        targetSdk 31
        versionCode 1
        versionName "1.0"

        testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
    }

Copy it to the dependencies section

    implementation fileTree(dir: 'libs', include: ['*.jar', '*.aar'])
    implementation 'androidx.appcompat:appcompat:1.4.1'
    implementation 'com.google.android.material:material:1.5.0'
    implementation 'androidx.constraintlayout:constraintlayout:2.1.3'
    testImplementation 'junit:junit:4.13.2'
    androidTestImplementation 'androidx.test.ext:junit:1.1.3'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0'
    // MediaPipe deps
    implementation 'com.google.flogger:flogger:latest.release'
    implementation 'com.google.flogger:flogger-system-backend:latest.release'
    implementation 'com.google.code.findbugs:jsr305:latest.release'
    implementation 'com.google.guava:guava:27.0.1-android'
    implementation 'com.google.protobuf:protobuf-javalite:3.19.1'
    // CameraX core library
    def camerax_version = "1.0.0-beta10"
    implementation "androidx.camera:camera-core:$camerax_version"
    implementation "androidx.camera:camera-camera2:$camerax_version"
    implementation "androidx.camera:camera-lifecycle:$camerax_version"
    // AutoValue
    def auto_value_version = "1.8.1"
    implementation "com.google.auto.value:auto-value-annotations:$auto_value_version"
    annotationProcessor "com.google.auto.value:auto-value:$auto_value_version"

Modify MainActivity.java and AndroidManifest.xml

Change myapplication to your application name

Sync Project with Gradle Files

Connect to your phone and click run

DONE!!!

References

Mediapipe

Github Repo

mediapipe_hand-tracking