<span style="font-size: 32px;">**Isaac Sim**</span>

If there are any errors in the article, please correct me, thank you.

---
[toc]

# Install Isaac Sim
## Docker runs Isaac Sim connect to Omniverse steaming client
**( Headless remote server )**

### In server

1. docker test

docker run --rm hello-world

2. check GPU

nvidia-smi

* check Nvidia-conatainer-toolkit,
If it is not installed, nothing will be displayed.

dpkg -l | grep nvidia-container-toolkit

3. Pull Container
[Images source](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/isaac-sim)

docker pull nvcr.io/nvidia/isaac-sim:{Version}

docker images

4. Run docker

docker run --name isaac-sim-test --entrypoint bash -it \
    --gpus '"device=0"' \
    -e "ACCEPT_EULA=Y" \
    --rm \
    --network=host \
    -e "PRIVACY_CONSENT=Y" \
    nvcr.io/nvidia/isaac-sim:4.5.0

docker run --name isaac-sim-test --entrypoint bash -it \
    --gpus '"device=0"' \
    -e "ACCEPT_EULA=Y" \
    --rm \
    --network=host \
    -e "PRIVACY_CONSENT=Y" \
    -v /path/to/folder/isaac-sim/cache/kit:/isaac-sim/kit/cache:rw \
    -v /path/to/folder/isaac-sim/cache/ov:/root/.cache/ov:rw \
    -v /path/to/folder/isaac-sim/cache/pip:/root/.cache/pip:rw \
    -v /path/to/folder/isaac-sim/cache/glcache:/root/.cache/nvidia/GLCache:rw \
    -v /path/to/folder/isaac-sim/cache/computecache:/root/.nv/ComputeCache:rw \
    -v /path/to/folder/isaac-sim/logs:/root/.nvidia-omniverse/logs:rw \
    -v /path/to/folder/isaac-sim/data:/root/.local/share/ov/data:rw \
    -v /path/to/folder/isaac-sim/documents:/root/Documents:rw \
    nvcr.io/nvidia/isaac-sim:4.5.0

docker run --name isaac-sim-test --entrypoint bash -it --gpus '"device=0"' -e "ACCEPT_EULA=Y" --rm --network=host \
    -e "PRIVACY_CONSENT=Y" \
    -v /m****/Pl***/{User_name}/{folder_name}/isaac-sim/cache/kit:/isaac-sim/kit/cache:rw \
    -v /m****/Pl***/{User_name}/{folder_name}/isaac-sim/cache/ov:/root/.cache/ov:rw \
    -v /m****/Pl***/{User_name}/{folder_name}/isaac-sim/cache/pip:/root/.cache/pip:rw \
    -v /m****/Pl***/{User_name}/{folder_name}/isaac-sim/cache/glcache:/root/.cache/nvidia/GLCache:rw \
    -v /m****/Pl***/{User_name}/{folder_name}/isaac-sim/cache/computecache:/root/.nv/ComputeCache:rw \
    -v /m****/Pl***/{User_name}/{folder_name}/isaac-sim/logs:/root/.nvidia-omniverse/logs:rw \
    -v /m****/Pl***/{User_name}/{folder_name}/isaac-sim/data:/root/.local/share/ov/data:rw \
    -v /m****/Pl***/{User_name}/{folder_name}/isaac-sim/documents:/root/Documents:rw \
    nvcr.io/nvidia/isaac-sim:4.5.0

`/m****/Pl***/{User_name}/{folder_name}/isaac-sim/`is the location of the local folder, so that the data after closing docker will also be stored.
Please see the [official website](https://docs.omniverse.nvidia.com/isaacsim/latest/installation/install_container.html#isaac-sim-setup-remote-headless-container) for details.

5. Start Isaac Sim with native livestream mode

# isaac sim 4.2.0
./runheadless.native.sh

./runheadless.native.sh apps/omni.create.kit \
   --enable omni.services.streaming.manager \
   --enable omni.kit.livestream.native \
   --no-window \
   --allow-root \
   --renderer RTX \
   --/app/window/dpiScaleOverride=1.0 \
   -v
   
./isaac-sim.headless.native.sh apps/omni.create.kit \
   --enable omni.services.streaming.manager \
   --enable omni.kit.livestream.native \
   --no-window \
   --allow-root \
   --renderer RTX \
   --/app/window/dpiScaleOverride=1.5 \
   
   
   
# isaac sim 4.5.0
# For omniverse streaming server  [DEPRECATED]
./isaac-sim.old_streaming.sh  apps/omni.create.kit \
   --enable omni.kit.livestream.native \
   --no-window \
   --allow-root \
   --renderer RTX \
   --/app/window/dpiScaleOverride=1.5 \
   
# For Isaac Sim WebRTC Streaming Client
./runheadless.sh
   

For isaac sim 4.2.0 :
When you see the message `Isaac Sim Headless Native App is loaded.`, it means that Isaac Sim has successfully started.
Using different `.sh` file to open Isaac Sim will obtain different results. The results here only correspond to using `./runheadless.native.sh`.
Please see the [official website](https://docs.omniverse.nvidia.com/isaacsim/latest/installation/manual_livestream_clients.html) for details.

For Isaac sim 4.5.0 :
When you see the message `Isaac Sim Full Streaming App is loaded.`, it means that Isaac Sim has successfully started.
Refer :
* [live streaming](https://docs.isaacsim.omniverse.nvidia.com/latest/installation/manual_livestream_clients.html#isaac-sim-short-webrtc-streaming-client)
* [Isaac Sim WebRTC Streaming Client Latest Release](https://docs.isaacsim.omniverse.nvidia.com/latest/installation/download.html#isaac-sim-latest-release)

### In local

* **This connection is established using a local network IP. If anyone knows how to connect using a public IP or other methods, your insights would be greatly appreciated.**

1. Install and launch the Omniverse Streaming Client from Omniverse Launcher **(deprecated)**, or use the latest Isaac Sim WebRTC Streaming Client.
* [Isaac Sim WebRTC Streaming Client Latest Release](https://docs.isaacsim.omniverse.nvidia.com/latest/installation/download.html#isaac-sim-latest-release)

2. Find server IP

hostname -I

Output: `192.168.X.XX 172.17.X.XX `

3. Enter the IP address of the machine or instance running the Isaac Sim container and click on the Connect button to begin live streaming.
In this example enter the IP as `192.168.X.XX`.

### Use python on Isaac Sim (Optional)
**On the local side, need to install VSCode along with `Dev Containers`, `Remote - SSH`, `Remote - SSH: Editing Configuration Files`, and `Remote Explorer`. On the server side, `Isaac Sim VS Code Edition` is required.
This allows you to edit Python code on the local side and execute it on the server side, with the results displayed in the GUI interface.**

# Install Isaac lab
* There seems to be other ways to use docker to start isaac lab, but I haven't read it in detail.
refer to:[Docker Guide](https://isaac-sim.github.io/IsaacLab/main/source/deployment/docker.html#directory-organization)
## If use docker to start Isaac sim
**( Headless remote server )**
### In server
* **The following is run in the started Isaac sim docker**
1. Clone the Isaac Lab

apt update && apt install -y git
git --version

git clone https://github.com/isaac-sim/IsaacLab.git

2. Creating the Isaac Sim Symbolic Link
Set up a symbolic link between the installed Isaac Sim root folder and _isaac_sim in the Isaac Lab directory. This makes it convenient to index the python modules and look for extensions shipped with Isaac Sim.

`cd /path/to/isaac-sim/IsaacLab`

ln -s path_to_isaac_sim _isaac_sim
# For example: ln -s /isaac-sim _isaac_sim

ls -l _isaac_sim

3. Installation

apt install cmake build-essential

`cd /path/to/isaac-sim/IsaacLab`

./isaaclab.sh --install # or "./isaaclab.sh -i"

4. Start Isaac Lab with native livestream mode

> In the following example, `--livestream 1` is used for Omniverse Streaming Client **(deprecated)**, while `--livestream 2` is used for Isaac Sim WebRTC Streaming Client.
> Written on 2025-02-15.

`cd /path/to/isaac-sim/IsaacLab`

./isaaclab.sh -p path/to/{aim.py_file} \
   --headless \
   --livestream 1 \
   --kit_args="--/app/window/dpiScaleOverride=1.5" \

**The paths in the following examples may change with different versions and are provided for reference only**

Example:
File path: `~/IsaacLab/scripts/tutorials/00_sim/create_empty.py`

`cd /path/to/isaac-sim/IsaacLab`

# isaac sim 4.5.0
./isaaclab.sh -p scripts/tutorials/00_sim/launch_app.py    
   --headless \
   --livestream 1 \
   --kit_args="--/app/window/dpiScaleOverride=1.5" \

Train example:
task list path: ~~`~/IsaacLab/source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/cartpole/__init__.py`~~

New path:
`~/IsaacLab/source/isaaclab_tasks/isaaclab_tasks/direct/cartpole/__init__.py`

`cd /path/to/isaac-sim/IsaacLab`

# isaac sim 4.2.0 / isaac lab 1.0
./isaaclab.sh -p source/standalone/workflows/rl_games/train.py \
   --task=Isaac-Cartpole-Direct-v0 \
   --headless \
   --livestream 1 \
   --kit_args="--/app/window/dpiScaleOverride=1.5" \
   
# new path
./isaaclab.sh -p scripts/reinforcement_learning/rl_games/train.py \
   --task=Isaac-Cartpole-Direct-v0 \
   --headless \
   --livestream 1 \
   --kit_args="--/app/window/dpiScaleOverride=1.5" \
   

Please see the [official website(RL train)](https://isaac-sim.github.io/IsaacLab/main/source/tutorials/03_envs/run_rl_training.html) for details.

Spot dog
path: `/IsaacLab/source/isaaclab_tasks/isaaclab_tasks/manager_based/locomotion/velocity/config/spot/__init__.py`
> ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py \
--task Isaac-Velocity-Flat-Spot-v0 \
--num_envs 4096 \
--headless \
--livestream 1 \
--kit_args="--/app/window/dpiScaleOverride=1.5" \

> ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/play.py \
--task Isaac-Velocity-Flat-Spot-v0 \
--num_envs 64 \
--headless \
--livestream 1 \
--kit_args="--/app/window/dpiScaleOverride=1.5" \

### In local
* Same as before

# install ROS2

* Written on 2025/03/24

## ROS2 Container Installation and Cross-Container Communication with Isaac Sim

This note documents the steps to create a ROS2 container, download and build the Isaac Sim official ROS workspace, and mount the ROS2 container into the Isaac Sim container to enable cross-container communication.

---

### 1. Creating the ROS2 Container on the Server

#### 1.1 Add the ROS package source and install rocker

sudo apt-key adv --keyserver 'hkp://keyserver.ubuntu.com:80' --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654

sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'

sudo apt-get update

sudo apt-get install python3-rocker

sudo apt autoremove

#### 1.2 Start the ROS2 Container
(We use the OSRF provided image: `osrf/ros:humble-desktop-full-jammy`)

rocker --nvidia --x11 --privileged --network host --name ROS2 --nocleanup \
osrf/ros:humble-desktop-full-jammy

> **Note:** To keep the container running long-term, you can use tmux:
> ```bash
> tmux new-session -d 'rocker --nvidia --x11 --privileged --network host --name <container name> osrf/ros:humble-desktop-full-jammy'
> ```

#### 1.3 Exit the ROS2 Container

exit

---

### 2. Downloading and Building the Isaac Sim ROS Workspace

#### 2.1 Download the official ROS workspace from Isaac Sim

cd /home/user
git clone https://github.com/isaac-sim/IsaacSim-ros_workspaces.git

#### 2.2 Copy the workspace into the running ROS2 container

docker cp /home/user/IsaacSim-ros_workspaces/humble_ws ROS2:/root/humble_ws

#### 2.3 Enter the ROS2 Container
<container name> = ROS2

docker exec -it <container name> bash

#### 2.4 Build the ROS2 Workspace

source /opt/ros/humble/setup.sh
cd /root/humble_ws
apt-get update
rosdep install --from-paths src --ignore-src --rosdistro=humble -y
source /opt/ros/humble/setup.sh
colcon build
source install/local_setup.bash

#### 2.5 Set ROS2-related environment variables and test

export ROS_DOMAIN_ID=0
export FASTRTPS_DEFAULT_PROFILES_FILE=~/.ros/fastdds.xml
source /opt/ros/humble/setup.sh

# Verify functionality
ros2 topic list
ros2 node list

---

### 3. Creating a New ROS2 Image and Mounting as a Volume

#### 3.1 Copy the ROS2 installation folder from the ROS2 container to the server

docker cp ROS2:/opt/ros/humble /home/user/docker/ROS2/humble

#### 3.2 Commit the ROS2 Container to a New Image

docker commit ROS2 new_ros2_image:3.24

#### 3.3 Stop and Remove the Original ROS2 Container

docker stop ROS2
docker rm ROS2

#### 3.4 Start a New ROS2 Container with Volume Mounting

rocker --nvidia --x11 --privileged --network host --name ROS2 \
--nocleanup \
--volume /home/user/docker/ROS2/humble:/opt/ros/humble \
-- new_ros2_image:3.24

---

### 4. Starting the Isaac Sim Container and Mounting the ROS2/humble Folder

Run the following command on the server to start the Isaac Sim container and mount the ROS2/humble folder:

docker run --name isaac-sim-robotic --entrypoint bash -it --gpus '"device=1"' -e "ACCEPT_EULA=Y" --network=host \
    -e "PRIVACY_CONSENT=Y" \
    -v /home/user/docker/isaac-sim/cache/kit:/isaac-sim/kit/cache:rw \
    -v /home/user/docker/isaac-sim/cache/ov:/root/.cache/ov:rw \
    -v /home/user/docker/isaac-sim/cache/pip:/root/.cache/pip:rw \
    -v /home/user/docker/isaac-sim/cache/glcache:/root/.cache/nvidia/GLCache:rw \
    -v /home/user/docker/isaac-sim/cache/computecache:/root/.nv/ComputeCache:rw \
    -v /home/user/docker/isaac-sim/logs:/root/.nvidia-omniverse/logs:rw \
    -v /home/user/docker/isaac-sim/data:/root/.local/share/ov/data:rw \
    -v /home/user/docker/isaac-sim/documents:/root/Documents:rw \
    -v /home/user/docker/IsaacLab:/IsaacLab:rw \
    -v /home/user/docker/ROS2/humble:/opt/ros/humble:rw \
    --env ROS_DISTRO=humble \
    --env AMENT_PREFIX_PATH=/opt/ros/humble \
    --env RMW_IMPLEMENTATION=rmw_fastrtps_cpp \
    --env LD_LIBRARY_PATH=/isaac-sim/exts/isaacsim.ros2.bridge/humble/lib:$LD_LIBRARY_PATH \
    isaac-sim-isaac-lab-robotic:3.16

---

### 5. Setting Up the ROS2 Environment Inside the Isaac Sim Container

Once inside the Isaac Sim container, execute the following steps:

1. **Set PATH and Create a Symbolic Link for Python3**

   export PATH=/isaac-sim/kit/python/bin:$PATH
   ln -s /isaac-sim/kit/python/bin/python3 /usr/bin/python3

2. **Set ROS2 and ROS2 Bridge Related Environment Variables**

   export ROS_DOMAIN_ID=0
   export RMW_IMPLEMENTATION=rmw_fastrtps_cpp
   export LD_LIBRARY_PATH=/opt/ros/humble/lib:$LD_LIBRARY_PATH
   export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/isaac-sim/exts/omni.isaac.ros2.bridge/humble/lib
   export AMENT_PREFIX_PATH=/opt/ros/humble
   export ROS_DISTRO=humble

3. **Source the ROS2 Environment**

   source /opt/ros/humble/setup.bash

4. **(Option)Install Missing tinyxml2 Packages**

   apt-get update
   apt-get install -y libtinyxml2-9 libtinyxml2-dev

   # Check installation:
   ls -l /usr/lib/x86_64-linux-gnu/libtinyxml2.so.9*

---

### 6. Final Checks

Inside the Isaac Sim container, verify that the ROS2 Bridge extension starts without libtinyxml2 errors:

cd /isaac-sim
./runheadless.sh

output:

[13.071s] [ext: isaacsim.ros2.bridge-4.1.15] startup
[13.226s] Attempting to load system rclpy
[13.245s] rclpy loaded

---

### Note

- Ensure both containers are using `--network host` or have a properly configured Docker network to allow cross-container communication.
- Adjust paths, container names, and image tags as needed based on your environment.
- The paths used above (`/home/user` and `/home/user/docker`) are for public documentation purposes; you can modify them as appropriate for your setup.

This completes the process for installing the ROS2 container and enabling cross-container communication with Isaac Sim.