# FMOps：大語言模型訓練及部署 如何在 AWS 上選擇適合的工具進行模型的訓練及部署： 以 Model : Llama 3 70B 為例：（✓ 越多，代表操作越簡易；✗ 不適用） | Service | Host | Train (Full) | Train (Adaptor) | |-----------|------|-------|-------| | Bedrock | ✓✓✓✓ <br>(支援模型有限制) | ✗ | ✗ | SageMaker JumpStart | ✓✓✓ | ✗ | ✓✓✓ <br>(支援模型眾多) | SageMaker Training | ✗ | ✓✓ <br>(可自定義) | ✓✓ <br>(可自定義) | SageMaker Hosting | ✓✓ | ✗ | ✗ | SageMaker HyperPod | ✓ | ✓ | ✓ | EC2 instances | ✓ | ✓ | ✓ 注意： - 每個區域支援的可調優模型不同。Nova 僅在 us-east-1，而 us-west-2 有支援 Llama 3.1 8B/70B Instruct 模型的 finetuning。 - 採用 Bedrock funetuning 的模型須使用 PT 部署。 --- ## 在 AWS 上訓練模型 :::success AWS blog: [Generative AI foundation model training on Amazon SageMaker](https://aws.amazon.com/blogs/machine-learning/generative-ai-foundation-model-training-on-amazon-sagemaker/) 更新時間：2024-10-22 ::: ### 使用 Bedrock 訓練文字語言模型的資料格式 :::info 請參考 [官網](https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-prepare.html) ::: #### 非監督式訓練 Domain adaptation fine-tuning 僅支援 ```python # .jsonl {"input": "<input text>"} {"input": "<input text>"} {"input": "<input text>"} ``` #### 任務型監督式訓練 Instruction tuning format ```python # .jsonl {"prompt": "<prompt1>", "completion": "<expected generated text>"} {"prompt": "<prompt2>", "completion": "<expected generated text>"} ``` ### 使用 Bedrock 進行模型訓練 :::info 支援的模型少，支援的模型請見[官網](https://docs.aws.amazon.com/bedrock/latest/userguide/custom-model-supported.html) ::: ### 使用 SageMaker 訓練文字語言模型的資料格式 #### 非監督式訓練 Domain adaptation fine-tuning ```text # .jsonl {"text": "<input text>"} {"text": "<input text>"} {"text": "<input text>"} ``` #### 非監督式訓練 Chat completion [範例資料](https://huggingface.co/datasets/ruslanmv/ai-medical-chatbot/viewer/default/train?p=1&row=101) ```text [ {"role": "user", "content": "{Human conversation}"}, {"role": "assistant", "content": "{AI conversation}"} ] ``` #### 任務型監督式訓練 Instruction tuning format 訓練資料必須使用 JSON lines (.jsonl) 格式，其中每一行都是一個JSON object，代表單個資料樣本。所有訓練資料必須放在同一個資料夾中，不過可以分散儲存在多個 jsonl 檔案中。必須使用 .jsonl 副檔名。訓練資料夾中也可以包含一個 template.json 檔案，用來描述輸入和輸出的格式。 如果沒有提供範本檔案，系統將使用以下預設範本，使用 `prompt` 和 `completion`： ```text # 預設 { "prompt": "{prompt}", "completion": "{completion}" } ``` 可以自定義提示詞中可動態置換的參數名稱，像是 `instruction`, `context`, 和 `response`： ```python! # 準備 template.json { "prompt": "Below is an instruction that describes a task." "Write a response that appropriately completes the request.\n\n" "### Instruction:\n{instruction}\n\n### Response:\n", "completion": "{output}" } # 或者 { "prompt": "Below is an instruction that describes a task, paired with an input that provides further context. " "Write a response that appropriately completes the request.\n\n" "### Instruction:\n{instruction}\n\n### Input:\n{context}\n\n### Response:\n", "completion": " {response}", } # 準備 train.jsonl { "instruction": "Who painted the Two Monkeys", "context": "Two Monkeys or Two Chained Monkeys is a 1562 painting by Dutch and Flemish Renaissance artist Pieter Bruegel the Elder. The work is now in the Gemäldegalerie (Painting Gallery) of the Berlin State Museums.", "response": "The two Monkeys or Two Chained Monkeys is a 1562 painting by Dutch and Flemish Renaissance artist Pieter Bruegel the Elder. The work is now in the Gemaeldegalerie (Painting Gallery) of the Berlin State Museums." } ``` ### 使用 SageMaker JumpStart 進行 Llama 模型訓練 :::info 支援的模型眾多，主要使用 LoRA/QLoRA 訓練 ::: :::success AWS blog: [Use Llama 3.1 405B for synthetic data generation and distillation to fine-tune smaller models](https://aws.amazon.com/blogs/machine-learning/use-llama-3-1-405b-to-generate-synthetic-data-for-fine-tuning-tasks/) 更新時間：2024-07-23 AWS blog: [Fine-tune Meta Llama 3.1 models for generative AI inference using Amazon SageMaker JumpStart](https://aws.amazon.com/blogs/machine-learning/fine-tune-meta-llama-3-1-models-for-generative-ai-inference-using-amazon-sagemaker-jumpstart/) 更新時間：2024-08-21 AWS blog: [Fine-tune Llama 3 for text generation on Amazon SageMaker JumpStart](https://aws.amazon.com/blogs/machine-learning/fine-tune-llama-3-for-text-generation-on-amazon-sagemaker-jumpstart/) 更新時間：2024-09-06 ::: ![Screenshot 2024-10-20 at 9.44.01 PM](https://hackmd.io/_uploads/rJovH9zlJl.png) ### 使用 SageMaker Bring Your Own Script, BYOS 進行模型訓練 [AWS GitHub BYOS](https://github.com/aws/amazon-sagemaker-examples/tree/default/%20%20%20%20generative_ai) | [script for encoder-decoder](https://github.com/aws/amazon-sagemaker-examples/blob/default/%20%20%20%20generative_ai/sm-finetuning_huggingface_with_your_own_scripts_and_data/source/train.py) | [script for mistral](https://github.com/aws/amazon-sagemaker-examples/blob/default/%20%20%20%20generative_ai/sm-mixtral_8x7b_fine_tune_and_deploy/scripts/run_clm.py) ### 模型調優函式庫及工具 [torchtune](https://github.com/pytorch/torchtune) [transformers](https://github.com/huggingface/transformers) | [guide](https://www.philschmid.de/sagemaker-train-deploy-llama3) [autotrain-advanced](https://github.com/huggingface/autotrain-advanced/tree/main) [meta/llama-recipes](https://github.com/meta-llama/llama-recipes/tree/main) [TRL - Transformer Reinforcement Learning](https://huggingface.co/docs/trl/main/en/index) [Weights & Biases (W&B)](https://wandb.ai/site) --- ## 部署模型 ### 使用 Bedrock Imported Models 部署及調用模型 (Preview) :::info - 採用 on-demand 方式計費，需要先提交 quota `Imported models per account` 申請，詳情請見[官網](https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-import-model.html) - 請注意：Bedrock Imported Models 支援模型請參考[官網](https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-import-model.html) - 關於 [產品定價](https://aws.amazon.com/bedrock/pricing/) ::: :::success AWS blog: [Amazon Bedrock Custom Model Import now generally available](https://aws.amazon.com/blogs/machine-learning/amazon-bedrock-custom-model-import-now-generally-available/) 更新時間：2024-10-21 ::: 您可以從 HuggingFace 將 model artifacts 下載到 S3，透過 Bedrock Imported Models 進行模型的部署和調用 ![Screenshot 2024-10-20 at 9.29.18 PM](https://hackmd.io/_uploads/rkJv0tGeJe.png) 或者，您也可以透過 Bedrock 部署及調用 SageMaker JumpStart 訓練的模型， 僅需要找出 model artifacts 的儲存位置即可(如下圖位置) ![Screenshot 2024-10-29 at 9.13.39 AM](https://hackmd.io/_uploads/rki4v3pgyl.png) Bedrock Imported Models Job 在部署 Llama 3 70B 的模型，約莫需要25分鐘的時間 ![Screenshot 2024-10-20 at 9.29.27 PM](https://hackmd.io/_uploads/B1--y9fgJe.png) Import job 提交後，若您選擇創建新的 IAM role，須在頁面上停留等待資源建立。 ![Screenshot 2024-10-29 at 9.19.14 AM](https://hackmd.io/_uploads/BJnsj26gJg.png) 完成 import job 後，<font color="#f00">需要等待模型部署約莫數分鐘</font>的時間，才能在 Playgrounds 使用，否則您將會看到下圖的警告紅框 ![Screenshot 2024-10-20 at 10.27.10 PM](https://hackmd.io/_uploads/ryhOX5zxkx.png =49%x) ![Screenshot 2024-10-20 at 9.29.40 PM](https://hackmd.io/_uploads/SJ1SJ5zgkx.png =49%x) 目前<font color="#f00">尚不能</font>使用 Bedrock imported models 進行模型的二次訓練，或是選擇 Provisioned throughput 進行部署 ![Screenshot 2024-10-20 at 9.30.07 PM](https://hackmd.io/_uploads/ryruk9GlJx.png) --- ## 模型輸出結果評測 :::success AWS samples: [Large Language model and prompt evaluation](https://github.com/aws-samples/build-an-automated-large-language-model-evaluation-pipeline-on-aws) ::: GitHub: [aws/fmeval](https://github.com/aws/fmeval) GitHub: [Evaluate Bedrock Imported Models](https://github.com/aws-samples/amazon-bedrock-samples/blob/main/custom-models/import_models/fmeval_imported_models.ipynb) --- ## 推理效能評測 [GitHub AWS samples](https://github.com/aws-samples/amazon-bedrock-samples/blob/main/custom-models/import_models/perf-test-imported_model.ipynb) ## 調優部署流程自動化 :::success AWS blog: [Automate fine-tuning of Llama 3.x models with the new visual designer for Amazon SageMaker Pipelines](https://aws.amazon.com/blogs/machine-learning/automate-fine-tuning-of-llama-3-x-models-with-the-new-visual-designer-for-amazon-sagemaker-pipelines/) 更新時間：2024-10-22 ::: --- ## 更多學習資源 AWS samples: [Fine-tune Foundation Models on Amazon SageMaker using @remote decorator](https://github.com/aws-samples/amazon-sagemaker-llm-fine-tuning-remote-decorator/tree/main) [Meta Fine-tune](https://www.llama.com/docs/how-to-guides/Fine-tuning) [Meta Quantization](https://www.llama.com/docs/how-to-guides/quantization) YouTube: [Chat Fine tuning](https://www.youtube.com/watch?v=71x8EMrB0Gc) Medium: [LLM domain adaptation using continued pre-training](https://medium.com/@aris.tsakpinis/llm-domain-adaptation-using-continued-pre-training-part-4-4-e4fc3acffac7) [Transformers API](https://github.com/huggingface/transformers)