# Arm 架構驅動雲端原生 DevOps 與生成式 AI 的創新轉型：高效能、低功耗的雙贏實踐與成功案例 - 沈綸銘(Odin Shen)

{%hackmd @HWDC/BJOE4qInR %}

>#### 》[議程介紹](https://hwdc.ithome.com.tw/2024/session-page/3326)
>#### 》[填寫議程滿意度問卷｜回饋建言給辛苦的講者](https://forms.gle/D6hVC5R5DXNzk7QK8)

(不開放錄影與拍照)

# Brief

為什麼要講軟體?

# Agenda

* Arm Computing Platform
* Arm Software Ecosystem

* Key focus
* Cloud Migration
* AI Perference
* Windows On Arm
* Arm Development Program

# Who is ARM

作CPU, GPU架構授權的公司。
* 290+ Billion chip
* 跟1000多家公司有合作，像是slack, Github, HuggingFace
* 20 milliion的開發者再用ARM的開發平台

# Arm Computing Platform Around the World

* 手機
* 基礎建設
* 汽車
* IoT

# Arm Neoverse in the Cloud

AWS, Microsoft, Google, NVDIA都使用Neoverse V2架構

* AWS 60%
* Azure 40%
* GCP 60%

這樣大家不就是打架?因為ARM很有彈性讓各家公司自己設計需要的晶片

* Such as AWS Gravition4

## Google Axion Processor

## Alibaba Cloud Yitian*(倚天) 710 Uplift

# AI is Everywhere
# AI is Expensive
(power consumption)
2027年會是2倍的power consumption of Google Data Centers
PS. 22B kWh in 2022

# The Evolution of AI - New Challenages

# Arm CARES about Software

# Arm's Pivotal Role in the Software Ecosystem
- 40% of Arm Engineers focused on software
- 1000 Open source engineers in Arm

很多底層的Open Source的Foundation都是由ARM的平台與工程師設計出來與優化的

# Arm's Open-Source Investemets
- 1200+ Open-Source project from Cloud to edge
- e.g. linaro / Windows
- arm mali GPU
- Collabora
- python
- CMSIS
- armRAN
- 投資大量資源在opens source，擴展生態

# Windows on Arm (WoA) Ecosystem readiness
- Apps ISVs
- OEM Devices
- Arm在Windows上跟很多Software 合作能原生支援ARM

# The Software Developer Challenges
- Hardware
Optimizing code performance & depolyment
- Software
Blurring the lines between verticals
- AI
Staggering growth - developers need to keep up

# 3 types of scenarios - Key focus areas for developers on Arm
* Cloud Migration
* Seamlessly Migrate Your Code to an Energy- Efficient Cloud Platform
* AI Performance
* Unleash the AI performance on arm CPUs
* Windows On Arm
* Seamlessly Migrate your app to Windows on Arm

# Cloud Migration

## Docker on Arm

* 跨平臺，開發者效率高
* 2019開始就在不同的作業系統上面開始合作
* 在Docker上面會有ARM原生的Image

## Pre-installed AArch64 Docker Images

* Pre-install TensorFlow, TensorFlowLite, Pytorch & ArmNN AArch64 docker images for developing across operating systems and easily sharing project

## Arm64 on Github Actions
[LINK](https://github.blog/news-insights/product-news/arm64-on-github-actions-powering-faster-more-efficient-build-systems/)

Github Actions在後台會自動launch一個ARM的系統執行Action
Github Actions + MLOps

## Github Actions MLOPs Example

[..Live Demo..] from Arm-Labs/gh_armrunner_mlops_gtsrb

https://learn.arm.com/migration

## Select Architecture & Cloud

## Plan, Test and Optimize

# AI performance

## Accelerating Hugging Face Models using Arm Neoverse

[Live Demo]

* 很多HuggingFace上的工具不一定需要nvidia的GPU，用ARM的CPU就可以了

## Run a Large Language ....

## Introducting Arm Kleidi

* KleidiAI: 用手機就可以使用大語言模型
* KleidiCV:
* 聲稱可加速Llama3 190%
## Arm KlediAi Accelerate Generative AI and Modeling

## Arm Kleidi Libraries

## KleidiAI Tutorial & Demo

Learn more
[Unlocking New Real-world Generative AI Use Cases on the Mobile CPU](https://newsroom.arm.com/blog/generative-ai-use-cases-mobile-cpu)
[Live Demo]

# Windows on Arm
[windowsperf-gui-the-visual-studio-2022-extension](https://www.linaro.org/blog/introducing-the-windowsperf-gui-the-visual-studio-2022-extension/)

# Arm Developer Program
[LINK](https://www.arm.com/zh-TW/resources/developer-program)

11,000+ Members
130 ambassadors

Arm ecosystem dashboard [[LINK](https://www.arm.com/developer-hub/ecosystem-dashboard)]

==QA==

1.如果我有APP 測看看在arm上有沒有問題, 我該怎麼做:
1.1 如果不是太複雜的話用Github, 再來用雲上AWS 用coupon

2. Github vs Gitlab
gilab will be the next

3.

==以下聊天區====

> 他沒有提供投影片哪來連結啊
>> 我剛也在想這件事，感覺會後應該會提供吧 XD

找Key word 找連結

會後會提供「安全版」簡報

> 他這邊一直在說的軟體，是指介於作業系統跟編譯器之間的軟體嗎？因為對於上層的開發者來說應該不用去管用什麼 hardware 吧？只要那個 programming language 有支援跨平台就可以跑
>> 應該是說支援上層開發者的軟體，比如Github Actions的instance可以用ARM

但是我看之前ARM PC的評測很慘? 電子垃圾等級，好像是用高通得CPU
> 請問 Apple 的 M2, M4 也是 arm 嗎？
>> 是

有點難想像為什麼 cloud instance 為什麼要用 arm ?
> 便宜，成本低
> 像使用樹莓派(Arm)來跑docker，這時候可以無痛遷上雲
> 相容性還是不如x86，不過有在進步，不要太冷門就好，我之前用s390x(IBM的mainframe CPU)，超慘，快跳樓了

ARM或x86有支援Ollama或是llama.cpp的加速嗎? (有回答了)

感覺這個 Talk 的目的是在宣傳 arm
> 第一贊助商
>> 原來是金主爸爸，失敬失敬
>> Keynote不都是這樣嗎? 不過也是說了不少東西