---
# System prepended metadata

title: Untitled

---

idist-snippets test on multiple  configurations:

On data cluster:

nvidia-smi

    +-----------------------------------------------------------------------------+
    | NVIDIA-SMI 460.32.03    Driver Version: 460.32.03    CUDA Version: 11.2     |
    |-------------------------------+----------------------+----------------------+
    | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
    | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
    |                               |                      |               MIG M. |
    |===============================+======================+======================|
    |   0  Tesla V100-PCIE...  Off  | 00000000:3B:00.0 Off |                    0 |
    | N/A   31C    P0    35W / 250W |      0MiB / 16160MiB |      0%      Default |
    |                               |                      |                  N/A |
    +-------------------------------+----------------------+----------------------+
    |   1  Tesla V100-PCIE...  Off  | 00000000:AF:00.0 Off |                    0 |
    | N/A   35C    P0    37W / 250W |      0MiB / 16160MiB |      0%      Default |
    |                               |                      |                  N/A |


pip freeze 

	certifi==2021.5.30
	numpy==1.21.0
	Pillow==8.2.0
	pytorch-ignite==0.5.0.dev20210623
	torch==1.9.0
	torchvision==0.10.0
	typing-extensions==3.10.0.0
    
RUN OK:

with torch spawner
    
        python -u ignite_idist.py --nproc_per_node 1 --backend nccl
        python -u ignite_idist.py --nproc_per_node 2 --backend nccl

        python -u ignite_idist.py --nproc_per_node 1 --backend gloo
        python -u ignite_idist.py --nproc_per_node 2 --backend gloo

        with docker hvd-vision:latest (pytorch-ignite==0.4.4 torch==1.8.1)
        python -u ignite_idist.py --backend horovod --nproc_per_node 1
        python -u ignite_idist.py --backend horovod --nproc_per_node 2
    
    
with torch.distributed.launch

        python -m torch.distributed.launch --nproc_per_node 1 --use_env ignite_idist.py --backend gloo
        python -m torch.distributed.launch --nproc_per_node 1 --use_env ignite_idist.py --backend nccl
    
        python -m torch.distributed.launch --nproc_per_node 2 --use_env ignite_idist.py --backend gloo
        python -m torch.distributed.launch --nproc_per_node 2 --use_env ignite_idist.py --backend nccl
        
        
with horovodrun with docker hvd-vision:latest (pytorch-ignite==0.4.4 torch==1.8.1)

    horovodrun -np 1 -H localhost:2 python -u  ignite_idist.py --backend horovod
    horovodrun -np 2 -H localhost:2 python -u  ignite_idist.py --backend horovod

On HPC cluster

with slurm on hpc cluster with conda pack (same env as data cluster)


OK with NCCL

    srun --nodes=1 --ntasks-per-node=2  --time=00:03:00 --partition=gpgpu --gres=gpu:2 python ignite_idist.py --backend nccl --nb_samples 256

OK with gloo ( note had to increase memory )

    srun --nodes=1 --ntasks-per-node=2  --time=00:03:00 --partition=gpgpu --gres=gpu:2 --mem=10G python ignite_idist.py --backend gloo --nb_samples 256


ON COLAB


PyTorch version: 1.9.0+cu102
PyTorch xla version: 1.9
PyTorch-Ignite version: 0.5.0.dev20210623

OK

python -u ignite_idist.py --backend xla-tpu --nproc_per_node 8 --batch_size 64 --nb_samples 512

    
    