Running TFLite on Your Mobile Devices - Koan-Sin Tan

tags: COSCUP2020 入門 TR212

歡迎來到 https://hackmd.io/@coscup/2020 共筆

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →

點擊本頁上方的 開始用 Markdown 一起寫筆記！
手機版請點選上方 按鈕展開議程列表。

請從這裡開始

GDG 攤位在 TR312 有貼紙可以拿喔!

涵蓋google技術，開放源碼相關內容技術，一整天議程歡迎聆聽。

Slide Link

Why tf-lite?

  1. tensorflow: ML framework.
  2. tf-lite: lightweight framework for tensorflow
  3. vs. Pytorch: Not on mobile device yet.

TFlite in Android and IOS device.

Mobile -> IOT device

“There is an old network saying: Bandwidth problems can be cured with money. Latency problems are harder because the speed of light is fixed — you can't bribe God.” – David D. Clark

TFLite actually has two main parts

• interpreter: loads and runs a model on various hardware
• converter: converts TF models to a TFLite specific format to be used by the interpreter.

introduction materials

See https://www.tensorflow.org/lite/guide for more introduction materials

Language

There is a good guide on how to load a model and do inference on devices using TFLite interpreter, in Java(Android), Swift(IOS), Objective-C, C++, and Python
• https://www.tensorflow.org/lite/guide/inference

TFLite metadata and TFLite Android code generator.

Metadata

In TFLite metadata, there are three parts in the schema:
• Model information
• Input information
• Output information

Supported Input / Output types

• Feature
• Image
• Bounding box

Pack the associated files, e.g., label file(s)
Normalization and quantization parameters

With example at here, we can create a image classifier with
• image input
• label output

Test TFLite metadata and codegen

https://github.com/freedomtan/CameraxTFLite

Some new features: CoreML delegate and XNNPACK delegate.

“A TensorFlow Lite delegate is a way to delegate part or all of graph execution to another executor.”

Flashback: COSCUP 2019 slide
how NNAPI and GPU delegates work, https://www.slideshare.net/kstan2/tflite-nnapi-and-gpu-delegates

XNNPACK: Paper

Concluding remarks

  1. TFLite is getting more and more mature and comprehensive
  2. TFLite metadata: Only Android code generators
    • nope, there is no iOS code generator (yet)
  3. To speed up execution of NN models, use TFL delegates
    • note that not all accelerators are created equal
    • some are fp only; some are int/quant only

The note is written by Simon

Select a repo