---
# System prepended metadata

title: Run SGLang Thor & Spark

---

# Run SGLang Thor & Spark

1. Install uv
```bash 
curl -LsSf https://astral.sh/uv/install.sh | sh
```
2. Create environment

```bash
uv venv .sglang --python 3.12
source .sglang/bin/activate
sudo apt install python3-dev python3.12-dev
```

3. Export variables
```bash
export TORCH_CUDA_ARCH_LIST=12.1a # Spark, for Thor 11.0a
export TRITON_PTXAS_PATH=/usr/local/cuda/bin/ptxas
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
```

4. Install SGLang
```bash
uv pip install sgl-kernel --prerelease=allow --index-url https://docs.sglang.ai/whl/cu130/
uv pip install sglang --prerelease=allow 
uv pip install --force-reinstall torch torchvision torchaudio triton --index-url https://download.pytorch.org/whl/cu130

uv pip install flashinfer-python
```

5. Clean memory
```bash
sudo sysctl -w vm.drop_caches=3
```
6. Run nemotron nvfp4

```bash 
python3 -m sglang.launch_server --model-path nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4 \
  --trust-remote-code \
  --tp 1 \
  --attention-backend flashinfer \
  --tool-call-parser qwen3_coder \
  --reasoning-parser nano_v3
```