or
or
By clicking below, you agree to our terms of service.
New to HackMD? Sign up
Syntax | Example | Reference | |
---|---|---|---|
# Header | Header | 基本排版 | |
- Unordered List |
|
||
1. Ordered List |
|
||
- [ ] Todo List |
|
||
> Blockquote | Blockquote |
||
**Bold font** | Bold font | ||
*Italics font* | Italics font | ||
~~Strikethrough~~ | |||
19^th^ | 19th | ||
H~2~O | H2O | ||
++Inserted text++ | Inserted text | ||
==Marked text== | Marked text | ||
[link text](https:// "title") | Link | ||
 | Image | ||
`Code` | Code |
在筆記中貼入程式碼 | |
```javascript var i = 0; ``` |
|
||
:smile: | ![]() |
Emoji list | |
{%youtube youtube_id %} | Externals | ||
$L^aT_eX$ | LaTeX | ||
:::info This is a alert area. ::: |
This is a alert area. |
On a scale of 0-10, how likely is it that you would recommend HackMD to your friends, family or business associates?
Please give us some advice and help us improve HackMD.
Do you want to remove this version name and description?
Syncing
xxxxxxxxxx
AI世界的DevOps - 蘇冠羽
歡迎來到 DevOpsDay Taipei 2022 共筆
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →共筆入口:https://hackmd.io/@DevOpsDay/2022
手機版請點選上方 按鈕展開議程列表。
tags:
DevOpsDays Taipei 2022
MLaaS1.0 ETL
Machine Learning as a Service: AI建模運算平台
1.0 上線後的問題
可觀測性低、自動化程度低
排程管理者須請 DBA 操作 DB 才能調整排程開關
所有專案共用 DB 帳號,難以判讀 SQL 來源
開發者無法直接觀看 ETL 執行狀況與結果
元件耦合性高,牽一髮動全身
ETL、API 在同 Folder 下,只改一個元件整個程式碼都要跑一遍 Pipeline
OS執行環境共用、資源分配不易
專案間互搶資源
MLaaS2.0 AirFlow
Airflow 是使用 python 編寫的管理平台
dag(Directed Acyclic Graph) 是由一個個 task 排序而成
從 UI 上就能直接看出 task 狀態
初期架構思考:
40 個專案要有 40 座 Airflow 嗎?
如何中心化管理,並解決 1.0 的問題?
以 Kubernetes Executer 做到環境隔離
以 Airflow 的 RBAC(Role-based Access Control) 做到權限隔離
在 CI 階段使用 folder 做專案切分
透過 Airflow Connection,每個專案有自己的 DB 帳號