# Isaac Sim 5.1 + Isaac Lab 2.3 + LeIsaac on Windows
**My working setup (with the fixes that actually matter).**

I built Isaac Sim 5.1 from source, wired it to Isaac Lab 2.3, added LeIsaac, and controlled an **SO101 Leader** arm. Below is exactly what I ran, in the order I ran it. If you follow this, you should end up with a setup that opens, runs, and records without mystery errors.

---

## Final layout I aim for

C:\Projects\IsaacStack\
├─ isaacsim\_build\windows-x86_64\release\     ← Isaac Sim 5.1 build
├─ IsaacLab_23\                                ← Isaac Lab 2.3
│  └─ _isaac_sim\ → (symlink to build)         ← Critical symlink
├─ leisaac\                                    ← Lightwheel repo
│  └─ assets\ → (symlink to assets)            ← Assets symlink
├─ assets\                                     ← Centralized assets
│  ├─ robots\so101_follower.usd
│  └─ scenes\kitchen_with_orange\
│     ├─ scene.usd
│     ├─ assets\
│     └─ objects\
└─ datasets\                                   ← Teleop recordings

---
## 0) Remove conflicting package

https://github.com/isaac-sim/IsaacLab/discussions/2046

 Remove "OpenCL™, OpenGL®, and Vulkan® Compatibility Pack". It can be removed from Add and Remove Programs in Windows Settings.

## 1) Build Isaac Sim 5.1

**What I used**
- Windows 11 (22H2 or 24H2)
- NVIDIA GPU (driver **576+** recommended)
- Visual Studio 2022 with **Desktop development with C++**
- Include **Windows 11 SDK (10.0.22621.0)**
- Git + Git LFS
- ~100 GB free disk
- [Enabled Developer Mode](https://learn.microsoft.com/en-us/windows/advanced-settings/developer-mode)

**Clone + build** (Using Command Prompt -- Not PowerShell)

cd C:\Projects\IsaacStack
git clone https://github.com/isaac-sim/IsaacSim.git isaacsim
cd isaacsim
git lfs install && git lfs pull
build.bat -r

**Smoke test the build**

cd _build\windows-x86_64\release
isaac-sim.bat --reset-user

If it opens and closes cleanly, you’re good.
Build lives at: `C:\Projects\IsaacStack\isaacsim\_build\windows-x86_64\release`

---

## 2) Isaac Lab 2.3 (linked to the build)

**Clone the right branch**

cd C:\Projects\IsaacStack
git clone --branch release/2.3.0 --depth 1 https://github.com/isaac-sim/IsaacLab.git IsaacLab_23

**Symlink Isaac Sim into Isaac Lab** *(Run CMD as Administrator)*

mklink /D C:\Projects\IsaacStack\IsaacLab_23\_isaac_sim C:\Projects\IsaacStack\isaacsim\_build\windows-x86_64\release

**Quick check**

dir C:\Projects\IsaacStack\IsaacLab_23\_isaac_sim\*.bat

You should see `isaac-sim.bat`, `python.bat`, etc.

---

## 3) Conda env (handy shell, not the interpreter we run)

conda create -n lab23 python=3.11 -y
conda activate lab23
python -m pip install --upgrade pip

**PyTorch 2.7.0 (CUDA 12.8)**

pip install torch==2.7.0 torchvision==0.22.0 --index-url https://download.pytorch.org/whl/cu128

**CUDA sanity check**

python -c "import torch; print('CUDA:', torch.cuda.is_available(), 'Version:', torch.version.cuda)"

Should print something like: `CUDA: True Version: 12.8`

> Heads‑up: We’ll **run** everything with Isaac Sim’s Python, not conda’s. I still like having the conda env for shells and convenience scripts.

---

## 4) Install Isaac Lab **using Isaac Sim’s Python** (critical)

cd C:\Projects\IsaacStack\IsaacLab_23

rem Use Isaac Sim's Python, NOT conda's!
_isaac_sim\python.bat -m pip install --upgrade pip
_isaac_sim\python.bat -m pip install -e source/isaaclab

rem Needed for datasets
_isaac_sim\python.bat -m pip install h5py

---

## 5) LeIsaac (installed into Isaac Sim’s Python)

**Clone the repo**

cd C:\Projects\IsaacStack
git clone https://github.com/LightwheelAI/leisaac.git
#version lock
git checkout 3bb45ed

Warning: This update introduced a bug so we cannot do recording.
#https://github.com/LightwheelAI/leisaac/commit/58d56c72d88c21cab9b1b4dfbdb7b9a14e02979a#diff-5be0fc3e1d02fba8f98099ea5eecdd4dd20efc7988c9f3025724144bc2eb4030

**Editable install + hardware deps**

cd C:\Projects\IsaacStack\IsaacLab_23
_isaac_sim\python.bat -m pip install -e C:\Projects\IsaacStack\leisaac\source\leisaac
_isaac_sim\python.bat -m pip install pyserial hidapi

---

## 6) Assets (one folder, symlinked into LeIsaac)

**Download**

cd C:\Projects\IsaacStack
mkdir assets\robots
mkdir assets\scenes

rem Robot USD
curl -L -o assets\robots\so101_follower.usd ^
  https://github.com/LightwheelAI/leisaac/releases/download/v0.1.0/so101_follower.usd

rem Kitchen scene
curl -L -o kitchen_with_orange.zip ^
  https://github.com/LightwheelAI/leisaac/releases/download/v0.1.0/kitchen_with_orange.zip

tar -xf kitchen_with_orange.zip -C assets\scenes\
del kitchen_with_orange.zip

Now, copy and paste the entire assets folder to manually. Please delete or rename the older folder as `assets_backup` before copy and pasting.

#copy manually
#C:\Project\IsaacStack\assets -> C:\Projects\IsaacStack\leisaac\assets

**Check**

dir C:\Projects\IsaacStack\leisaac\assets\scenes\kitchen_with_orange\

You should see `scene.usd` plus `assets\` and `objects\`.

---

## 7) Find the SO101 COM port

mode

Look for a block like: `Status for device COM3:` — that’s your port.

---

## 8) Launch teleoperation 🚀

**Basic run**

C:\Projects\IsaacStack\IsaacLab_23\_isaac_sim\python.bat C:\Projects\IsaacStack\leisaac\scripts\environments\teleoperation\teleop_se3_agent.py ^
  --task=LeIsaac-SO101-PickOrange-v0 ^
  --teleop_device=so101leader ^
  --port=COM3 ^
  --num_envs=1 ^
  --device=cuda ^
  --enable_cameras

**Record a dataset too**
[:warning:!warning!:warning:] Please make sure `run_tel1st.hdf5` is removed from the `C:\Projects\IsaacStack\datasets` folder or this will quit because it won't be able to create dataset. If you want to resume using the dataset, please use the `--resume` flag.

Also, please check and ensure `port=COM3`, if not please change to `COM4` or others accordingly.

mkdir C:\Projects\IsaacStack\datasets 2>nul
set H5=C:\Projects\IsaacStack\datasets\run_te1st.hdf5

C:\Projects\IsaacStack\datasets
C:\Projects\IsaacStack\IsaacLab_23\_isaac_sim\python.bat
C:\Projects\IsaacStack\leisaac\scripts\environments\teleoperation\teleop_se3_agent.py ^
  --task=LeIsaac-SO101-PickOrange-v0 ^
  --teleop_device=so101leader ^
  --port=COM3 ^
  --num_envs=1 ^
  --device=cuda ^
  --enable_cameras ^
  --record ^
  --dataset_file="%H5%"

**Controls**
- **B** → start/stop control
- **N** → mark success
- **R** → mark failure

---

## 9) One‑click launchers

**`teleop.bat`**

@echo off
setlocal
cd /d C:\Projects\IsaacStack\IsaacLab_23
For /f "tokens=2-4 delims=/ " %%a in ('date /t') do (set mydate=%%c-%%a-%%b)
For /f "tokens=1-4 delims=/:/ " %%a in ('echo %TIME:~0,8%') do (set mytime=%%a-%%b-%%c)
set TS=%mydate%_%mytime%
set H5=C:\Projects\IsaacStack\datasets\run_%TS%.hdf5
if not exist C:\Projects\IsaacStack\datasets mkdir C:\Projects\IsaacStack\datasets
_isaac_sim\python.bat C:\Projects\IsaacStack\leisaac\scripts\environments\teleoperation\teleop_se3_agent.py ^
  --task=LeIsaac-SO101-PickOrange-v0 ^
  --teleop_device=so101leader ^
  --port=COM3 ^
  --num_envs=1 ^
  --device=cuda ^
  --enable_cameras ^
  --record ^
  --dataset_file="%H5%"
echo Dataset saved to: %H5%
pause
endlocal

**`teleop_vr.bat` (Quest)**

@echo off
setlocal

C:\Projects\IsaacStack\IsaacLab_23\_isaac_sim\python.bat C:\Projects\IsaacStack\leisaac\scripts\environments\teleoperation\teleop_se3_agent.py ^
  --xr ^
  --task=LeIsaac-SO101-PickOrange-v0 ^
  --teleop_device=so101leader ^
  --port=COM3 ^
  --num_envs=1 ^
  --device=cuda ^
  --enable_cameras
endlocal

---

## 10) Replay datasets

C:\Projects\IsaacStack\IsaacLab_23\_isaac_sim\python.bat C:\Projects\IsaacStack\leisaac\scripts\environments\teleoperation\replay.py ^
  --task=LeIsaac-SO101-PickOrange-v0 ^
  --num_envs=1 ^
  --device=cuda ^
  --enable_cameras ^
  --replay_mode=action ^
  --dataset_file=C:\Projects\IsaacStack\datasets\run_test.hdf5 ^
  --select_episodes 1 2

---

## 11) VR (Quest) checklist

1) In Meta Quest Link → **Settings → General → Set as Active OpenXR Runtime**
2) Launch with `--xr` (or use `teleop_vr.bat`)
3) In Isaac Sim: **Window → Rendering → VR** ✔
4) **Window → VR → VR Panel → Start VR**

---

## 12) Common “gotchas” I hit (and fixes)

| Symptom | Fix |
|---|---|
| `TypeError: 'NoneType' object is not callable` | You ran the wrong Python. Use `_isaac_sim\python.bat` (or `run.bat`). |
| `ModuleNotFoundError: No module named 'isaacsim'` | Re‑install Isaac Lab with `_isaac_sim\python.bat -m pip install -e source/isaaclab`. |
| `ModuleNotFoundError: No module named 'h5py'` | `_isaac_sim\python.bat -m pip install h5py`. |
| `ModuleNotFoundError: No module named 'leisaac'` | `_isaac_sim\python.bat -m pip install -e C:\Projects\IsaacStack\leisaac\source\leisaac`. |
| `Failed to open layer @...scene.usd@` | Create the assets symlink: `mklink /D leisaac\assets assets` (Admin). |
| `'run.bat' is not recognized` | Create it: `echo @echo off > run.bat && echo _isaac_sim\python.bat %%* >> run.bat`. |
| First launch is slow with scary warnings | Normal — shader/material caches warming. |
| `usd-core` / `lxml` warnings | Harmless in this flow. |

---

## 13) Quick sanity checks

rem Isaac Sim
_isaac_sim\python.bat -c "from isaacsim import SimulationApp; print('Isaac Sim OK')"

rem Isaac Lab
run.bat -c "import isaaclab; print('Isaac Lab OK')"

rem LeIsaac
run.bat -c "import leisaac; print('LeIsaac OK')"

rem HW libs
run.bat -c "import h5py, serial, hid; print('Hardware libs OK')"

rem Asset present
dir C:\Projects\IsaacStack\leisaac\assets\scenes\kitchen_with_orange\scene.usd

---

## 14) What’s different from most guides (and why)

**The big ones**
1. **Always use `_isaac_sim\python.bat` (or `run.bat`)** — don’t mix in `isaaclab.bat`.
2. **Install everything into Isaac Sim’s Python** — not conda’s Python.
3. **Centralize assets under** `C:\Projects\IsaacStack\assets\` **and symlink** `leisaac\assets` **to it**.
4. **Install `h5py` explicitly** with Isaac Sim’s Python.
5. **Create symlinks as Administrator** (`_isaac_sim` and `assets`).

**Folder reality check**

✅ CORRECT:
leisaac\assets\ → symlink to → C:\Projects\IsaacStack\assets\
                                ├─ robots\so101_follower.usd
                                └─ scenes\kitchen_with_orange\scene.usd

❌ WRONG:
leisaac\assets\scenes\kitchen_with_orange\ (actual files here)

---

## 15) Pre‑flight checklist

- [ ] Isaac Sim builds and runs with `--reset-user`
- [ ] `_isaac_sim` symlink points to the build
- [ ] Conda env `lab23` exists (for shell convenience)
- [ ] Isaac Lab installed **with** `_isaac_sim\python.bat`
- [ ] `h5py` installed **with** `_isaac_sim\python.bat`
- [ ] LeIsaac installed **with** `_isaac_sim\python.bat`
- [ ] Assets live in `C:\Projects\IsaacStack\assets\`
- [ ] `leisaac\assets` → `assets` symlink created
- [ ] `run.bat` exists and works
- [ ] SO101 shows up on the correct `COM` port
- [ ] Quick test prints `SUCCESS`

---

## 16) Golden Path

Here you can try running this to create a run_tes1st.hdf5 dataset

C:\Projects\IsaacStack\IsaacLab_23\_isaac_sim\python.bat C:\Projects\IsaacStack\leisaac\scripts\environments\teleoperation\teleop_se3_agent.py ^
  --task=LeIsaac-SO101-PickOrange-v0 ^
  --teleop_device=so101leader ^
  --port=COM4 ^
  --num_envs=1 ^
  --device=cuda ^
  --enable_cameras ^
  --record ^
  --dataset_file="C:\Projects\IsaacStack\datasets\run_te1st.hdf5" 

Press B to start the recording and interaction, press N to save, and press R to discard.

Now, check making sure the run_te1st.hdf5 is there. And use this to playback.

C:\Projects\IsaacStack\IsaacLab_23\_isaac_sim\python.bat C:\Projects\IsaacStack\leisaac\scripts\environments\teleoperation\replay.py ^
  --task=LeIsaac-SO101-PickOrange-v0 ^
  --num_envs=1 ^
  --device=cuda ^
  --enable_cameras ^
  --replay_mode=action ^
  --dataset_file=C:\Projects\IsaacStack\datasets\run_te1st.hdf5 ^
  --select_episodes 1 2

Finally, we will test out the VR setup.

C:\Projects\IsaacStack\IsaacLab_23\_isaac_sim\python.bat C:\Projects\IsaacStack\leisaac\scripts\environments\teleoperation\teleop_se3_agent.py ^
  --task=LeIsaac-SO101-PickOrange-v0 ^
  --teleop_device=so101leader ^
  --port=COM4 ^
  --num_envs=1 ^
  --device=cuda ^
  --enable_cameras ^
  --record --xr ^
  --dataset_file="C:\Projects\IsaacStack\datasets\run_te1st_vr.hdf5" 

The only difference this time is we will have VR enabled. To enable VR (it stated AR in the app...), you have to to go the tab on the right, and press the start AR button. After that you have to scroll down to use the recenter, and also use the 'active camera'. This will help you to calibrate your arm position next to you.

Remember we need to delete the older dataset "C:\Projects\IsaacStack\datasets" or use '--resume flag'. Pick whichever path makes most sense to you.

Finally, this is the script we should use for final demo, this will generate a new dataset with timestamp up to seconds.

@echo off
setlocal
cd /d C:\Projects\IsaacStack\IsaacLab_23
For /f "tokens=2-4 delims=/ " %%a in ('date /t') do (set mydate=%%c-%%a-%%b)
For /f "tokens=1-4 delims=/:/ " %%a in ('echo %TIME:~0,8%') do (set mytime=%%a-%%b-%%c)
set TS=%mydate%_%mytime%
set H5=C:\Projects\IsaacStack\datasets\run_%TS%.hdf5
echo Saving to %H5%
if not exist C:\Projects\IsaacStack\datasets mkdir C:\Projects\IsaacStack\datasets
_isaac_sim\python.bat C:\Projects\IsaacStack\leisaac\scripts\environments\teleoperation\teleop_se3_agent.py ^
  --xr ^
  --task=LeIsaac-SO101-PickOrange-v0 ^
  --teleop_device=so101leader ^
  --port=COM4 ^
  --num_envs=1 ^
  --device=cuda ^
  --enable_cameras ^
  --record ^
  --dataset_file="%H5%"
echo Dataset saved to: %H5%
pause
endlocal

## 17) Data Conversion and Post-processing

Now, you have a huge file sitting in 'C:\Projects\IsaacStack\datasets\run_te1st.hdf5' that is often 2-3GB for a 30-40seconds recording. The next step we need is to turn the simulation recording into LeRobot format. Here is how.

cd C:\Projects\IsaacStack\leisaac\scripts\convert
python -m venv venv
venv\Scripts\activate.bat
git clone https://github.com/huggingface/lerobot.git
cd lerobot

pip install -e .
pip install h5py
#now here is the fun part, first go back to the convert directory
cd .. 

#Here you have to edit isaaclab2lerobot.py to fix the bug 
#as lerobotdataset is updated

#second, you have to enter the parameters here in the main

Fix the bug, and replace the process_single_arm_data function with this.

def process_single_arm_data(dataset: LeRobotDataset, task: str, demo_group: h5py.Group, demo_name: str) -> bool:
    try:
        actions = np.array(demo_group['actions'])
        joint_pos = np.array(demo_group['obs/joint_pos'])
        front_images = np.array(demo_group['obs/front'])
        wrist_images = np.array(demo_group['obs/wrist'])
    except KeyError:
        print(f'Demo {demo_name} is not valid, skip it')
        return False

    if actions.shape[0] < 10:
        print(f'Demo {demo_name} has less than 10 frames, skip it')
        return False

    # preprocess actions and joint pos
    actions = preprocess_joint_pos(actions)
    joint_pos = preprocess_joint_pos(joint_pos)

    assert actions.shape[0] == joint_pos.shape[0] == front_images.shape[0] == wrist_images.shape[0]
    total_state_frames = actions.shape[0]
    # skip the first 5 frames
    for frame_index in tqdm(range(5, total_state_frames), desc='Processing each frame'):
        frame = {
            "action": actions[frame_index],
            "observation.state": joint_pos[frame_index],
            "observation.images.front": front_images[frame_index],
            "observation.images.wrist": wrist_images[frame_index],
	    "task": str(task)
        }
        dataset.add_frame(frame=frame)

    return True

We have to pre-enter parameters. This will convert the file inside the datasets folder, and create a folder ray/ray_so101_test_orange_pick inside
'C:\Users\nvidia\.cache\huggingface\lerobot' (for my case with nvidia username)

def convert_isaaclab_to_lerobot():
    """NOTE: Modify the following parameters to fit your own dataset"""
    repo_id = 'ray/ray_so101_test_orange_pick'
    robot_type = 'so101_follower'  # so101_follower, bi_so101_follower
    fps = 30
    hdf5_root = 'C:\Projects\IsaacStack\datasets'
    hdf5_files = [os.path.join(hdf5_root, 'run_te1st.hdf5')]
    task = {"grap_orange"}
    push_to_hub = False

    """parameters check"""
    assert robot_type in ['so101_follower', 'bi_so101_follower'], 'robot_type must be so101_follower or bi_so101_follower'

    """convert to LeRobotDataset"""
    now_episode_index = 0
    dataset = LeRobotDataset.create(
        repo_id=repo_id,
        fps=fps,
        robot_type=robot_type,
        features=SINGLE_ARM_FEATURES if robot_type == 'so101_follower' else BI_ARM_FEATURES,
    )

    for hdf5_id, hdf5_file in enumerate(hdf5_files):
        print(f'[{hdf5_id+1}/{len(hdf5_files)}] Processing hdf5 file: {hdf5_file}')
        with h5py.File(hdf5_file, 'r') as f:
            demo_names = list(f['data'].keys())
            print(f'Found {len(demo_names)} demos: {demo_names}')

            for demo_name in tqdm(demo_names, desc='Processing each demo'):
                demo_group = f['data'][demo_name]
                if "success" in demo_group.attrs and not demo_group.attrs["success"]:
                    print(f'Demo {demo_name} is not successful, skip it')
                    continue

                if robot_type == 'so101_follower':
                    valid = process_single_arm_data(dataset, task, demo_group, demo_name)
                elif robot_type == 'bi_so101_follower':
                    valid = process_bi_arm_data(dataset, task, demo_group, demo_name)

                if valid:
                    now_episode_index += 1
                    dataset.save_episode()
                    print(f'Saving episode {now_episode_index} successfully')

    if push_to_hub:
        dataset.push_to_hub()
        

At the end you run the script

python .\isaaclab2lerobot.py 

And run this to visualize the output

 lerobot-dataset-viz.exe --repo-id=ray/ray_so101_test_orange_pick --episode-index 0


### That’s it.
This is the exact combo that’s been reliable for me on Windows. The two rules that keep everything sane:
- **Run with `_isaac_sim\python.bat` (or `run.bat`).**
- **Keep assets centralized and symlink into LeIsaac.**