How to Use Nsight Compute for Deep Learning Workload Analysis

<h1> How to Use Nsight Compute for Deep Learning Workload Analysis </h1> ![deep-learning](https://hackmd.io/_uploads/B1TAEDafZl.jpg) Nsight Compute is NVIDIA’s kernel profiler. If Nsight Systems tells you where time goes across CPU, GPU, and the runtime, Nsight Compute tells you why a specific GPU kernel is slow by showing occupancy, memory behavior, instruction mix, Tensor Core usage, and more. For deep learning, this is where you confirm whether your key kernels are compute bound, memory bound, or limited by something more subtle like poor launch configuration or low Tensor Core utilization. This blog walks through a practical workflow for using Nsight Compute to analyze deep learning workloads, with examples you can adapt for PyTorch, TensorFlow, and custom CUDA kernels. <h2>When to use Nsight Compute for deep learning</h2> Use Nsight Compute when you already know which kernels matter and you want deeper answers such as: <ul> <li>Is my GEMM or convolution limited by memory bandwidth or math throughput? </li> <li>Are Tensor Cores being used effectively? </li> <li>Is occupancy low because of register pressure or shared memory usage? </li> <li>Are memory loads uncoalesced or thrashing cache? </li> <li>Why is this fused kernel slower than expected? </li> </ul> If you do not yet know which kernels to inspect, start with Nsight Systems or PyTorch Profiler to identify hotspots, then zoom in with Nsight Compute. <h2>Key concept: you are profiling kernels, not “the model”</h2> Deep learning frameworks launch many GPU kernels per iteration. Nsight Compute profiles individual kernel launches. Your job is to identify the few kernels that dominate runtime, then analyze them deeply. In many models, the heavy hitters are: <ul> <li>GEMMs from matmul and linear layers (often cuBLAS) </li> <li>Convolutions (often cuDNN) </li> <li>Attention primitives, including softmax, layernorm, and fused kernels </li> <li>Reductions and elementwise ops if fusion is poor </li> <li>Data movement kernels if the pipeline is inefficient </li> </ul> <h2>Getting set up</h2> <h3>Install and verify access</h3> Nsight Compute comes with the <a href="https://acecloud.ai/blog/nvidia-cuda-cores-explained/">NVIDIA CUDA</a> toolkit or as a separate package depending on your platform. Confirm you can run the CLI: ncu --version For server environments, the CLI is usually enough. The GUI is great for interactive exploration and roofline charts, but it is optional. <h3>Profile in a stable environment</h3> Profiling changes timing and can perturb scheduling. For best results: <ul> <li>Use a <a href="https://acecloud.ai/cloud/gpu/">dedicated GPU</a> if possible </li> <li>Fix input shapes and batch size </li> <li>Warm up the model before profiling </li> <li>Avoid profiling the first iteration if kernels are still caching or autotuning </li> </ul> <h2>Step 1: Narrow to a small capture window</h2> Do not profile an entire training job. Instead, capture a short window around steady state. For PyTorch, a simple approach is to run a few warmup iterations, then a small number of profiled iterations. If you cannot easily “pause” the run, Nsight Compute supports capturing only specific kernels or only the first N launches, which is often good enough. <h2>Step 2: Collect a lightweight kernel list first</h2> Start with a minimal metric set to see which kernels dominate and to confirm you are targeting the right work. A typical pattern: ncu --target-processes all \ --set default \ --profile-from-start off \ --launch-skip 100 \ --launch-count 50 \ -o ncu_report \ python train_or_infer.py Notes: <ul> <li>--launch-skip and --launch-count help you skip warmup launches and capture a stable window. </li> <li>--set default keeps overhead manageable. </li> <li>--target-processes all helps with multi process launchers, but use it carefully since it can generate a lot of data. </li> </ul> Then open the report in the GUI or inspect from the CLI: ncu --import ncu_report.ncu-rep --page summary Look for the kernels with the highest total time. Often you will see cuBLAS GEMM kernels and cuDNN convolution kernels near the top. <h2>Step 3: Filter to the one kernel you care about</h2> Once you identify the kernel name, profile only that kernel with richer metrics. Filtering dramatically reduces overhead and noise. Example: ncu --target-processes all \ --kernel-name "regex:.*gemm.*" \ --launch-count 10 \ --set full \ -o gemm_deep \ python train_or_infer.py If you are profiling attention, you might filter by patterns like fmha, flash, softmax, layernorm, or framework specific fused kernel names. <h2>Step 4: Use the “Speed of Light” view and roofline thinking</h2> Nsight Compute often organizes key findings in a “Speed of Light” style summary. The core question is: is the kernel limited by compute or by memory? A simple roofline mindset helps: <ul> <li>If achieved FLOPs is far below peak, you might be memory bound or instruction limited. </li> <li>If memory bandwidth is near peak but math is low, you are likely memory bound. </li> <li>If both are low, you may have low occupancy, launch overhead, synchronization, or poor vectorization. </li> </ul> For deep learning kernels, common scenarios include: <ul> <li>GEMM is compute bound but not using Tensor Cores efficiently </li> <li>Softmax is memory bound with lots of reads and writes </li> <li>Layernorm is memory bound, and fusion matters more than micro tuning </li> <li>Small batch inference leads to underutilized compute due to small problem size </li> </ul> <h2>Step 5: Check Tensor Core utilization</h2> If you expect Tensor Cores to be active, verify it. Things that often prevent Tensor Core use: <ul> <li>Wrong dtype, such as FP32 path when you intended FP16 or BF16 </li> <li>Shapes not aligned to preferred tile sizes </li> <li>Operations not hitting Tensor Core capable kernels </li> <li>Conversions or layout issues causing fallbacks </li> </ul> Nsight Compute can reveal whether the kernel uses HMMA or Tensor Core instructions. If you see a GEMM that is not using Tensor Cores when it should, fix dtype, layout, and alignment first before any deeper tuning. <h2>Step 6: Diagnose occupancy and launch limits</h2> Low occupancy can be fine if the kernel is already saturating memory bandwidth or compute, but it can also be a red flag. Typical reasons occupancy is limited: <ul> <li>High register usage per thread </li> <li>Large shared memory per block </li> <li>Block size that does not map well to SM resources </li> <li>Too few blocks overall because the problem size is small </li> </ul> Nsight Compute shows achieved occupancy and the limiting factor. For deep learning inference at small batch sizes, the “problem too small” scenario is extremely common, and the fix is usually batching, fusion, or choosing kernels optimized for small shapes rather than chasing occupancy alone. <h2>Step 7: Investigate memory behavior</h2> If the kernel looks memory bound, focus on: <ul> <li>Global memory load and store efficiency </li> <li>L2 cache hit rate </li> <li>Memory coalescing and transaction sizes </li> <li>Excessive reads and writes due to unfused ops </li> <li>Use of shared memory and whether it reduces global traffic </li> </ul> In many deep learning graphs, the largest memory improvements come from fusion and avoiding intermediate tensors, not from micro optimizing a single kernel. Nsight Compute helps you prove this by showing how much bandwidth each kernel is consuming and whether the data movement is the true limiter. <h2>Step 8: Use NVTX to map kernels to model phases</h2> Deep learning runs can have hundreds of similar kernels. NVTX ranges make it easier to connect kernels to steps like forward, backward, optimizer, or specific layers. Many frameworks and libraries already emit NVTX ranges when enabled. If not, you can add NVTX ranges in your code and then use those ranges to guide what you capture and when. <h2>Practical tips and common pitfalls</h2> <ul> <li>Profile fewer kernels with deeper metrics, not many kernels with everything. </li> <li>Re run profiling. Single runs can be noisy, especially with dynamic shapes or autotuning. </li> <li>Watch for profiler overhead. The “full” set can slow kernels significantly. </li> <li>Lock clocks only if you know what you are doing. Changing GPU clocks can improve repeatability but can also misrepresent production behavior. </li> <li>For multi GPU training, start with one GPU. Then profile a single rank and isolate communication kernels separately with Nsight Systems. </li> <li>In containers, ensure you have the right permissions and the host driver supports profiling. Some environments restrict access to performance counters. </li> </ul> <h2>A simple end to end workflow</h2> <ol> <li>Identify hotspot ops with PyTorch Profiler or Nsight Systems. </li> <li>Capture a short steady state window with Nsight Compute default set. </li> <li>Filter to the top kernel and switch to a richer set like full. </li> <li>Classify the kernel as compute bound or memory bound. </li> <li>Verify Tensor Core usage where expected. </li> <li>Use occupancy and memory metrics to decide the best fix, often dtype, layout, fusion, batching, or kernel choice. </li> <li>Re benchmark with the same inputs and compare throughput and latency. </li> </ol> If you tell me whether you are profiling training or inference, your GPU model, and whether you are using FP16, BF16, or FP32, I can suggest a focused Nsight Compute command line and the highest value metric sections to collect for your specific workload.