# TMLS Workshop Guide
This guide should provide you a step-by-step guide on how to fine-tune our CodeLama adapter to your own custom github repository; deploy it to huggingface model hub and use it in inference inside of Flyte!
## Workshop Requirements
- Skill Level: Intermediate
- Unix based system (MacOSX / Debian / Ubuntu)
- Python/Pip v3.9+
- An IDE for development (pycharm, VS Code, etc)
- Docker
## Step 1: Union Cloud Access
As this workshop will require and leverage advanced and powerful Cloud GPU Servers; we'll be providing temporary access to Union Ai's Managed Cloud.
Please provide the host or attendant with your email (if you haven't already) to ensure you have access.
You can verify your access level by visiting https://demo.hosted.unionai.cloud/ in your web browser and logging in with google auth.
download uctl at https://docs.union.ai/getting-started/installing-development-tools#install-uctl
run the following commands to ensure you're applying to the right cluster:
```bash!
uctl config init --host https://demo.hosted.unionai.cloud
export FLYTECTL_CONFIG=~/.uctl/config.yaml
```
## Step 2: Code & Dependencies Setup
In a terminal, git clone the llm-fine-tuning repo and cd to the workshop directory
```bash
git clone https://github.com/unionai-oss/workshops.git
cd workshops/flyte_llm
```
Next, you'll want to create a local virtual environment so you can register and compile your flyte code.
You'll also want to install the requirements.txt dependencies.
```bash
python -m venv ~/venvs/flyte-llm
source ~/venvs/flyte-llm/bin/activate
pip install -r requirements.txt
```
## Step 3: Rename your code directories
As flyte uses the directory structure of your code for naming, it's recommended to rename the default package and workflow.py files to something more useful:
```
flyte_llm/
workflows.py
```
renamed to
```
my_username_here/
worklows.py
```
Also, lets set that new name to an environment variable so we can use it later
```bash!
export MY_PACKAGE_NAME=my_username_here
```
This will make it easier to search for your specific version of the flyte_llm code on the flyte cluster; and avoid overriding other peoples projects.
## Step 3a: Data Exploration (optional)
Create the Dataset Locally
```bash!
python flyte_llm/dataset.py --output-path ~/datasets/flyte_llama
```
Get file extension count:
```bash!
find ~/datasets/flyte_llama -type f -name '*.*' | sed 's|.*\.||' | sort | uniq -c | sort
```
It’s often useful to see how the data is loaded when model is trained on.
In a REPL like ipython
```python!
from flyte_llama.dataloader import get_dataset
from pathlib import Path
dataset = get_dataset(Path.home() / "datasets" / "flyte_llama")
print(dataset[100]["text"])
```
## Step 4a: Train a Model
Now you're ready to get started, let's run execute your new workflow with Flyte:
```bash!
pip install flytekit==1.10.0
```
and then
```bash!
pyflyte run --remote --copy-all --project tmls-2023 --domain development $MY_PACKAGE_NAME/workflows.py train_workflow --config config/flyte_pythia_70m_full_v0.json
```
This will return a flyte execution link that you can use to inspect the status, the execution id is used for weights & biases management.
**Note: If you want to make modifications to your model training configuration, such as it's huggingface name; edit the config file at `config/flyte_pythia_1b_full_v0.json`**
## Step 4b: Train a LoRA Model
Same as above, but lets change the config file:
```bash!
pyflyte run --remote --copy-all --project tmls-2023 --domain development $MY_PACKAGE_NAME/workflows.py train_workflow --config config/flyte_pythia_1b_lora_v0.json
```
**Note: If you want to make modifications to your model training configuration, such as it's huggingface name; edit the config file at `config/flyte_pythia_1b_lora_v0.json`**
Now we've launched a separate execution that is using LoRA to train just a small adjecent matrix, instead of adjusting all of the weights in the original model.
## Step 4c: Train a LoRA Model with your own github repo
Same as above, but lets add a custom repo path:
```bash!
pyflyte run --remote --copy-all --project tmls-2023 \
--domain development $MY_PACKAGE_NAME/workflows.py \
train_workflow \
--config config/flyte_pythia_1b_lora_v0.json \
--custom_github_url https://github.com/numpy/numpy
```
# Step 5: Reviewing the training jobs
Checking out weights and biases, since we built out an integration!
We have provided all participants automatic weights and biases access to their experiments
All you need to do is to add your flyte execution id to the end of this weights&biases url.
`https://wandb.ai/zeryx/unionai-flyte-llama/runs/<execution_id_from_flyte>`
An example of one of these runs can be found here:
https://wandb.ai/zeryx/unionai-flyte-llama/runs/aklwwth42xj2vrk9bzjq
# Step 6: register your lora adapter to huggingface!
Lets assume one of your training jobs succeeded; awesome! That's a big step. Lets deploy it to huggingface hub so you can keep playing with it at home.
```bash!
pyflyte run --project tmls-2023 --domain development --remote --copy-all flyte_llm/workflows.py publish_model_workflow --model_dir <your s3 output path> --adapter_name "<your custom adapter name>" --config config/<the_config_file_used>
```
# Step 7: Setting up inference in a Task
Now that you've registered your adapter to huggingface, lets use it! We can call a simple inference workflow from within Flyte
```bash!
pyflyte run --project tmls-2023 --domain development --remote --copy-all flyte_llm/workflows.py inference_workflow --config config/flyte_llama_7b_qlora_v0.json --adapter_path "unionai/FlyteLlama-v0-7b-hf-flyte-repos" --prompt "This below represents a basic Flyte workflow leveraging GPUs"
```
Sign in with Wallet
Connect another wallet