# Edge Computing with CPU accelerator --- ## Edge Computing  --- ## Why Edge Computing * Stability * self-driving * Inference speed * shared memory vs Http service * Low cost --- ## ML platform intel: [OpenVINO](https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit.html) arm: [armnn](https://developer.arm.com/ip-products/processors/machine-learning/arm-nn) --- ## Armnn - ML platform Neon: advanced Single Instruction Multiple Data (SIMD)  --- ## Model inference with armnn on CPU 1. Model selection 2. Model transform 3. Prepare depended library 4. Cross compile with inference program 5. Inference model ---- ### Model selection * inference speed and model size is the first priority | model | size(MB) | Top-5 Acc (%) | | ------------- | -------- | ------------- | | Inception-v2 | 44 | 95.22 | | ShuffleNet-v2 | 9.2 | 88.32 | | VGG16 | 527.9 | 91.21 | | ResNet-18 | 44.7 | 89.29 | ---- ### Model transform Armnn support Caffe, TF, TFlite, Onnx Transform model if you used other framework like: pytorch, mxnet ---- ### Prepare depended library File size is always the big issue in Soc( System on a Chip ) Consider the library size when * Image preprocessing tool * opencv: Opencv is a Integrated library for compute vision, however it depended on lots of other libraries. e.g. ffmpeg * opencv size: ~200MB vs stb_image: ~500KB ---- ### Cross compile with inference program * Cross compile: compile code for multiple platforms from one development host. * Cross compile command: * `gcc-arm-linux-gnueabi -o main main.c ...` ---- ### Inference model  ---- ### Inference speed | model | framework | neon | inference time(s) | | ---------- | --------- |:------------ | ----------------- | | shuffleNet | tflite | - | 0.82 | | shuffleNet | armnn | with neon | 0.34 | | shuffleNet | armnn | without neon | 86 | ----