JetPack 6. The most radical update of Jetson.

1. Introduction to Nvidia Robotics Edge Device

NVIDIA Jetson is a series of embedded computing boards from NVIDIA. These boards are designed to provide high computational power in a small form factor, making them ideal for use in autonomous systems, robotics, AI applications, and edge computing. Jetson boards are particularly known for their GPU capabilities, which are essential for machine learning and computer vision tasks.

Key features of NVIDIA Jetson include:

Powerful NVIDIA GPU: Jetson boards come with NVIDIA GPUs, which are highly efficient at parallel processing tasks, making them suitable for AI and machine learning applications.

Compact Size: Despite their power, Jetson boards are relatively small, which is beneficial for integrating into space-constrained environments like drones or compact robots.

Energy Efficiency: They are designed to deliver high performance while consuming minimal power, which is critical for battery-operated or energy-sensitive applications.

Software Support: NVIDIA provides extensive software support through its JetPack SDK, which includes AI libraries, drivers, an operating system, and more, helping developers to easily deploy AI applications on Jetson.

Versatility: These boards are used in various applications, from industrial automation and medical devices to drones and robotics.

Community and Ecosystem: NVIDIA has built a robust community and ecosystem around Jetson, offering resources, forums, and support for developers.

What is JetPack?

JetPack is a comprehensive software development kit (SDK) designed specifically for Jetson devices. It includes a range of tools and libraries to facilitate the development and deployment of AI and computer vision applications on Jetson platforms. JetPack typically encompasses the Linux operating system for Jetson boards, CUDA for GPU-accelerated computing, cuDNN for deep neural networks, TensorRT for optimized inference, as well as additional tools and APIs necessary for AI development. This SDK plays a crucial role in leveraging the hardware capabilities of Jetson devices for efficient AI and machine learning tasks.

2. What's new JetPack 5 vs JetPack 6?

JetPack 5 Features (from JetPack 5.1.2)

CUDA 11.4, cuDNN 8.6.0, and TensorRT 8.5.2: These core libraries for GPU-accelerated computing, deep learning, and optimized inference are integral parts of JetPack 5, but their versions are very old, limited to execute SoTA models.
Limited Kernel Support: JetPack 5 supports specific kernel versions tied closely to the L4T version. Is not possible to upgrade, so you miss optimizations of the new versions.
Standard Security Features: Standard security measures are included, but without some of the advanced features found in JetPack 6.
Camera Drivers: Integrated as part of the kernel source, which can complicate driver updates.
OTA Updates: Over-the-air updates are supported, though the methodology is less flexible than in JetPack 6.

JetPack 6 New Features and Improvements

Greater Flexibility for Linux Kernels: Users can now use their own kernel (version 5.15 and up), providing more control and easier integration of newer drivers. So you will be able to upgrade to version 6.7 with ARM optimizations.

Increased Linux Distribution Choices: Support extends beyond L4T to include distributions like Ubuntu, Wind River, Red Hat, Yocto, and others, with Canonical offering commercial support for Ubuntu on Jetson. Each new version has newer optimizations and libraries such as libgc a major limitation currently in ubuntu 20.04, so you can install any Linux distribution you want.

Upgradable Compute Stack: Independent upgrades for the compute stack (including CUDA 12.2, TensorRT 8.6, and cuDNN 8.9 by default) are possible without updating the entire BSP. So, you can use new features from PyTorch like python torch.compilehttps://pytorch.org/get-started/pytorch-2.0/ to speed up the inference.

ROS 2 Humble Support: Enhanced support for robotics applications with native ROS 2 Humble support in Ubuntu 22.04.

Tiered Support Structure: Different levels of support, with direct assistance for distro partners and priority customers, and a DIY-based approach for others.

Camera Drivers as Out-of-Tree Modules: Simplifies driver updates and distribution, making it more modular.

Enhanced Security Features: Introduction of new security features like deep and measured boot.

Consistent OTA Update Methodology: Maintains the approach from previous JetPack versions, with app-based OTA for developer kits and image-based OTA for production.

3. Ubuntu 20.04 vs 22.04 vs future 24.04.

Ubuntu 20.04 to 22.04 Changes

1. Support Lifespan

   • 22.04: Supported until April 2027. Source
   • 20.04: Supported until 2025.

2. Dark Mode Improvements

   • 22.04: Complete system-wide dark mode experience. Source
   • 20.04: Basic dark mode.

3. GNOME 42 and Layout Changes

   • 22.04: Uses GNOME 42, introducing visual and workflow changes. Source
   • 20.04: Uses GNOME 3.36.8 with a stable experience.

4. Multitasking Capabilities

   • 22.04: Enhanced multitasking options. Source

5. Linux Kernel Version

   • 22.04: Multiple versions, including 5.15 LTS, 5.17 and 6.2. Source
   • 20.04: Linux Kernel 5.13.

6. File Manager

   • 22.04: Refreshed look. Source

7. Software Center Improvements

   • 22.04: Provides more detailed information about software. Source

Some benchmarks

Ubuntu 24.04 (Upcoming in 2024)

1. Linux Kernel

   • Includes new 6.x (6.7 and 6.8) Linux kernel. Source

2. Toolchain Upgrades

   • Updates to GCC, Python, Perl, LLVM, Rust toolchain. Source

3. Ubuntu Desktop Changes

   • Minimal default installation, reintroduction of ZFS guided installations, TPM-backed FDE, and other enhancements. Source

4. GNOME Update and Applications

   • Updated to GNOME +46 features and fixes, updated applications including Firefox as a native Wayland application. Source

Ubuntu 24.04 LTS should end up seeing either the Linux 6.7 or 6.8 kernels, GNOME 46, Mesa +24.1, and a wealth of other package updates.

Note: The information for Ubuntu 24.04 is based on expected features as it is yet to be released. Source

4. Important features of Cuda 12.2.

CUDA 11 vs CUDA 12: Key Differences and Improvements

CUDA 11

• NVIDIA Ampere GPU Architecture Support
  • Support for A100 GPU, increased SMs, and larger, faster memory (source).
• Multi-Instance GPU (MIG)
  • Divides a single A100 GPU into multiple instances for improved GPU utilization (source).
• Memory Error Recovery
  • Enhanced memory error recovery features for the A100 GPU (source).
• Third-Generation Tensor Cores
  • Accelerated matrix operations for various data types (source).
• Asynchronous Data Movement
  • Formalized asynchronous SIMT programming model (source).
• Task Graph Acceleration
  • Hardware acceleration for task graphs to reduce kernel launch latency (source).
• C++ Language and Compiler Improvements
  • Included CUB in CUDA Toolkit, support for ISO C++17 (source).
• Enhanced CUDA Libraries
  • Performance improvements using A100 hardware features (source).
• Developer Tools
  • Enriched Nsight tools for profiling and debugging, Roofline models in Nsight Compute (source).

CUDA 12

• Support for NVIDIA Hopper and NVIDIA Ada Lovelace Architectures
  • New hardware capabilities, increased SM counts, and memory bandwidth (source).
• Lazy Loading
  • Delayed loading of kernels and CPU-side modules for memory savings (source).
• Minor Version Compatibility
  • Dynamic linking across minor versions within the same major release (source).
• JIT LTO Support
  • Introduction of nvJitLink library for JIT LTO support (source).
• C++20 Compiler Support
  • Added support for C++20 standard (source).
• Nsight Developer Tools Updates
  • New features in Nsight Systems and Nsight Compute (source).
• Math Library Updates
  • Reduced binary sizes and specific feature enhancements (source).
• Heterogeneous Memory Management (HMM)
  • Seamless sharing of data between host memory and accelerator devices (source).
• Application Prioritization with CUDA MPS
  • Better priority arbitration under CUDA MPS (source).
• Debugging Tools
  • Advanced debugging tools like NVIDIA Compute Sanitizer and CUDA-GDB (source).