- Ping Wang·Last edited by Ping Wang on Mar 14, 2025Linked with GitHubContributed by
UW HYAK Notes
SALLOC
- Request for GPU
salloc --partition=gpu-l40 --account=stf --mem=10G --gres=gpu:1 --cpus-per-task=1 --time=2:00:00
- Check if GPU is requested
scontrol show job 24333466 | grep gpu
Conda Reinstall
rm -rf '/gscratch/scrubbed/andysu/miniconda3'
bash Miniconda3-latest-Linux-x86_64.sh -p /gscratch/scrubbed/andysu/miniconda3
python -m pip install --force-reinstall --upgrade setuptools pip
看哪個節點閒置
sinfo -t idle
salloc --partition=ckpt-all --gres=gpu:1 --nodelist=g3091 --time=8:00:00
GPU 確認 code
module load cuda/11.8.0
python -c "import torch; print(torch.cuda.is_available())"
- python code
import torch
print(torch.__version__)
print(torch.version.cuda) # 確保 PyTorch 版本支援 CUDA
print(torch.backends.cudnn.enabled)
看有沒有GPU裝置
scontrol show job 24202314 | grep TRES
如果是用salloc，登出後要回原本computing node
srun --jobid=<jobid> --pty bash
flash_attn 嘗試2.6.1 (考慮cuda版本) -> 安裝成功
Conda Related Commands
- Create conda environment
conda create --name my_env python=3.9
conda activate my_env
Read more
Summary: Decoder-Only Transformers: The Workhorse
Introduction This article delves into the architecture and mechanics of decoder-only transformers, which are a crucial component of many large language models (LLMs). It highlights the structure, attention mechanisms, and embedding techniques that make these models effective for various natural language processing (NLP) tasks. Decoder-Only Transformer Architecture Overview Decoder-only transformers, unlike the traditional encoder-decoder structure, use only the decoder component to process and generate text. This architecture is particularly suited for tasks that involve sequential generation, such as text completion and language modeling. Structure The decoder-only transformer consists of multiple layers, each containing self-attention mechanisms and feed-forward neural networks.
Jun 26, 2024Summary: The Illustrated Transformer
Introduction This article provides a visual and intuitive explanation of the transformer architecture, which has revolutionized natural language processing (NLP) by enabling efficient handling of sequential data through self-attention mechanisms. It covers the structure, mechanics, and key components such as the encoder, decoder, attention mechanisms, and embeddings. Transformer Architecture Overview The transformer architecture, introduced by Vaswani et al. in 2017, eliminates the need for recurrent layers by using self-attention mechanisms, allowing for parallel processing and better handling of long-range dependencies. Encoder-Decoder Structure The transformer model consists of an encoder-decoder architecture, each composed of multiple layers.
Jun 26, 2024Summary: Transformers Explained
This article offers a detailed explanation of transformers, a revolutionary architecture in natural language processing (NLP) that has significantly advanced the capabilities of large language models (LLMs). It covers the structure, mechanics, and key components of transformers, including the encoder, decoder, attention mechanisms, and embeddings.
Jun 26, 2024Summary: Introduction to Large Language Models and the Transformer Architecture
Introduction This article provides a foundational overview of large language models (LLMs) and the transformer architecture, explaining their structure, mechanics, and key components such as the encoder, decoder, attention mechanisms, and embeddings. Large Language Models (LLMs) Definition and Purpose LLMs are advanced neural networks trained on massive datasets to understand and generate human language. They are designed to handle various natural language processing (NLP) tasks such as translation, summarization, and question answering. Evolution LLMs have evolved significantly, with the introduction of models like BERT, GPT, and T5, which leverage transformer architectures to achieve state-of-the-art performance in many NLP benchmarks.
Jun 26, 2024
HackMD