# [Exp] Pytorch experimental env setup ###### tags: `research-DLRM` [TOC] ## Build Pytorch from source ### Prerequsite - Use conda or virtualenv (preferred) to create python virtual environments - Install dependency ```bash pip install astunparse numpy ninja pyyaml setuptools cmake cffi typing_extensions future six requests dataclasses ``` - Install Intel MKL ```bash pip install mkl mkl-include ``` - Nvidia GPU support (if any) ```bash pip install -c pytorch magma-cudaxxx # or the magma-cuda* that matches your CUDA version from https://anaconda.org/pytorch/repo ``` ### Get source code - Git clone ```bash= git clone --recursive https://github.com/pytorch/pytorch cd pytorch ``` - Existing repo ```bash= git submodule sync git submodule update --init --recursive --jobs 0 ``` ### Build options - `DEBUG=1` will enable debug builds (-g -O0) - `REL_WITH_DEB_INFO=1` will enable debug symbols with optimizations (-g -O3) - `USE_DISTRIBUTED=0` will disable distributed (c10d, gloo, mpi, etc.) build. - `USE_MKLDNN=0` will disable using MKL-DNN. - `USE_CUDA=0` will disable compiling CUDA (in case you are developing on something not CUDA related), to save compile time. - `BUILD_TEST=0` will disable building C++ test binaries. - `USE_FBGEMM=0` will disable using FBGEMM (quantized 8-bit server operators). - `USE_NNPACK=0` will disable compiling with NNPACK. - `USE_QNNPACK=0` will disable QNNPACK build (quantized 8-bit operators). - `USE_XNNPACK=0` will disable compiling with XNNPACK. ### Build and install pytorch for debugging ```bash= DEBUG=1 REL_WITH_DEB_INFO=1 USE_DISTRIBUTED=0 USE_MKLDNN=0 USE_CUDA=0 BUILD_TEST=0 USE_FBGEMM=0 USE_NNPACK=0 USE_QNNPACK=0 USE_XNNPACK=0 python setup.py develop ``` # Reference - [How embedding_bag exactly works in PyTorch](https://stackoverflow.com/questions/62052734/how-embedding-bag-exactly-works-in-pytorch) - [what does offsets mean in pytorch nn.EmbeddingBag?](https://stackoverflow.com/questions/65547335/what-does-offsets-mean-in-pytorch-nn-embeddingbag) - [Pytorch EmbeddingBag doc](https://pytorch.org/docs/stable/generated/torch.nn.EmbeddingBag.html#embeddingbag) - [Pytorch Embedding doc](https://pytorch.org/docs/stable/generated/torch.nn.Embedding.html#torch.nn.Embedding) - [通俗讲解pytorch中nn.Embedding原理及使用](https://www.jianshu.com/p/63e7acc5e890) - [Explaining the PyTorch EmbeddingBag Layer](https://jamesmccaffrey.wordpress.com/2021/04/14/explaining-the-pytorch-embeddingbag-layer/)