>[!Warning]
>***This is part two, part one down below
>https://hackmd.io/@NYTCEE/rJjOk_9c-g***

CUDA_VISIBLE_DEVICES=1 python -m scripts.eval     --policy_type lerobot     --policy_path outputs/train/act_top_long/checkpoints/last/pretrained_model     --garment_type "top_long"     --dataset_root Datasets/example/top_long_merged     --num_episodes 5     --enable_cameras     --device cpu        --save_video        --model_name "act_top_long"

## π₀.₅

### 5-1. top_long_merged

Garment Type: ==top_long==
Success Rate: 3.33%

python -m scripts.eval     --policy_type lerobot     --policy_path outputs/train/pi05_four_types/checkpoints/last/pretrained_model     --garment_type "top_long"     --dataset_root Datasets/example/top_long_merged     --num_episodes 5     --enable_cameras     --device cpu        --save_video

2026-04-01 17:03:09 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO - Overall Summary
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 17:03:09 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-01 17:03:09 - scripts.utils.eval_utils - INFO - Evaluation Results Summary
2026-04-01 17:03:09 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-01 17:03:09 - scripts.utils.eval_utils - INFO - Total Episodes: 60
2026-04-01 17:03:09 - scripts.utils.eval_utils - INFO - Average Return: 107.55 ± 18.43
2026-04-01 17:03:09 - scripts.utils.eval_utils - INFO - Success Rate: 3.33%
2026-04-01 17:03:09 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO - Per-Garment Summary
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO -   Top_Long_Seen_0: Success Rate = 0.00%, Avg Return = 103.78
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO -   Top_Long_Seen_1: Success Rate = 0.00%, Avg Return = 116.55
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO -   Top_Long_Seen_2: Success Rate = 0.00%, Avg Return = 94.18
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO -   Top_Long_Seen_3: Success Rate = 0.00%, Avg Return = 101.19
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO -   Top_Long_Seen_4: Success Rate = 20.00%, Avg Return = 130.49
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO -   Top_Long_Seen_5: Success Rate = 20.00%, Avg Return = 119.48
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO -   Top_Long_Seen_6: Success Rate = 0.00%, Avg Return = 114.09
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO -   Top_Long_Seen_7: Success Rate = 0.00%, Avg Return = 100.17
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO -   Top_Long_Seen_8: Success Rate = 0.00%, Avg Return = 91.21
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO -   Top_Long_Seen_9: Success Rate = 0.00%, Avg Return = 111.28
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO -   Top_Long_Unseen_0: Success Rate = 0.00%, Avg Return = 111.06
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO -   Top_Long_Unseen_1: Success Rate = 0.00%, Avg Return = 97.12
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO - Evaluation completed successfully
2026-04-01 17:03:09 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01T09:03:09Z [3,020,623ms] [Error] [omni.kit.renderer.plugin] advanceCurrentFrame: backbuffers are not initialized!
[3023.947s] Simulation App Shutting Down
/home/nytcee/.local/share/uv/python/cpython-3.11.14-linux-x86_64-gnu/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
(lehome) nytcee@idlab1:/mnt/train-data-1-hdd/naomi/lehome-challenge$ 

## XVLA

### 5-1. top_long_merged

Garment Type: ==top_long==
Success Rate: 1.33%

CUDA_VISIBLE_DEVICES=1 python -m scripts.eval     --policy_type lerobot     --policy_path outputs/train/xvla_four_types/checkpoints/last/pretrained_model     --garment_type "top_long"     --dataset_root Datasets/example/top_long_merged     --num_episodes 5     --enable_cameras     --device cpu        --save_video

2026-04-01 12:42:03 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO - Overall Summary
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 12:42:03 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-01 12:42:03 - scripts.utils.eval_utils - INFO - Evaluation Results Summary
2026-04-01 12:42:03 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-01 12:42:03 - scripts.utils.eval_utils - INFO - Total Episodes: 60
2026-04-01 12:42:03 - scripts.utils.eval_utils - INFO - Average Return: 115.85 ± 26.25
2026-04-01 12:42:03 - scripts.utils.eval_utils - INFO - Success Rate: 1.67%
2026-04-01 12:42:03 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO - Per-Garment Summary
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO -   Top_Long_Seen_0: Success Rate = 0.00%, Avg Return = 118.30
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO -   Top_Long_Seen_1: Success Rate = 0.00%, Avg Return = 150.64
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO -   Top_Long_Seen_2: Success Rate = 0.00%, Avg Return = 84.90
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO -   Top_Long_Seen_3: Success Rate = 0.00%, Avg Return = 96.07
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO -   Top_Long_Seen_4: Success Rate = 0.00%, Avg Return = 118.70
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO -   Top_Long_Seen_5: Success Rate = 0.00%, Avg Return = 120.36
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO -   Top_Long_Seen_6: Success Rate = 20.00%, Avg Return = 130.63
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO -   Top_Long_Seen_7: Success Rate = 0.00%, Avg Return = 112.38
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO -   Top_Long_Seen_8: Success Rate = 0.00%, Avg Return = 100.08
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO -   Top_Long_Seen_9: Success Rate = 0.00%, Avg Return = 121.13
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO -   Top_Long_Unseen_0: Success Rate = 0.00%, Avg Return = 114.86
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO -   Top_Long_Unseen_1: Success Rate = 0.00%, Avg Return = 122.16
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO - Evaluation completed successfully
2026-04-01 12:42:03 - scripts.utils.evaluation - INFO - ============================================================
[2433.010s] Simulation App Shutting Down
/home/nytcee/.local/share/uv/python/cpython-3.11.14-linux-x86_64-gnu/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
(lehome) nytcee@idlab1:/mnt/train-data-1-hdd/naomi/lehome-challenge$ 

### 5-2. top_short_merged

Garment Type: ==top_short==
Success Rate: 0.00%

CUDA_VISIBLE_DEVICES=1 python -m scripts.eval     --policy_type lerobot     --policy_path outputs/train/xvla_four_types/checkpoints/last/pretrained_model     --garment_type "top_short"     --dataset_root Datasets/example/top_short_merged     --num_episodes 5     --enable_cameras     --device cpu        --save_video

2026-04-01 13:45:51 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO - Overall Summary
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 13:45:51 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-01 13:45:51 - scripts.utils.eval_utils - INFO - Evaluation Results Summary
2026-04-01 13:45:51 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-01 13:45:51 - scripts.utils.eval_utils - INFO - Total Episodes: 60
2026-04-01 13:45:51 - scripts.utils.eval_utils - INFO - Average Return: 140.68 ± 52.63
2026-04-01 13:45:51 - scripts.utils.eval_utils - INFO - Success Rate: 0.00%
2026-04-01 13:45:51 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO - Per-Garment Summary
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO -   Top_Short_Seen_0: Success Rate = 0.00%, Avg Return = 136.44
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO -   Top_Short_Seen_1: Success Rate = 0.00%, Avg Return = 118.15
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO -   Top_Short_Seen_2: Success Rate = 0.00%, Avg Return = 114.30
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO -   Top_Short_Seen_3: Success Rate = 0.00%, Avg Return = 226.95
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO -   Top_Short_Seen_4: Success Rate = 0.00%, Avg Return = 211.95
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO -   Top_Short_Seen_5: Success Rate = 0.00%, Avg Return = 95.18
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO -   Top_Short_Seen_6: Success Rate = 0.00%, Avg Return = 119.18
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO -   Top_Short_Seen_7: Success Rate = 0.00%, Avg Return = 120.32
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO -   Top_Short_Seen_8: Success Rate = 0.00%, Avg Return = 118.37
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO -   Top_Short_Seen_9: Success Rate = 0.00%, Avg Return = 111.35
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO -   Top_Short_Unseen_0: Success Rate = 0.00%, Avg Return = 125.23
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO -   Top_Short_Unseen_1: Success Rate = 0.00%, Avg Return = 190.76
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO - Evaluation completed successfully
2026-04-01 13:45:51 - scripts.utils.evaluation - INFO - ============================================================
[2474.182s] Simulation App Shutting Down
/home/nytcee/.local/share/uv/python/cpython-3.11.14-linux-x86_64-gnu/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
(lehome) nytcee@idlab1:/mnt/train-data-1-hdd/naomi/lehome-challenge$ 

### 5-3. pant_long_merged

Garment Type: ==pant_long==
Success Rate: 3.33%

CUDA_VISIBLE_DEVICES=1 python -m scripts.eval     --policy_type lerobot     --policy_path outputs/train/xvla_pant_long/checkpoints/last/pretrained_model     --garment_type "pant_long"     --dataset_root Datasets/example/pant_long_merged     --num_episodes 5     --enable_cameras     --device cpu        --save_video

2026-04-01 16:50:33 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO - Overall Summary
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 16:50:33 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-01 16:50:33 - scripts.utils.eval_utils - INFO - Evaluation Results Summary
2026-04-01 16:50:33 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-01 16:50:33 - scripts.utils.eval_utils - INFO - Total Episodes: 60
2026-04-01 16:50:33 - scripts.utils.eval_utils - INFO - Average Return: 107.98 ± 32.83
2026-04-01 16:50:33 - scripts.utils.eval_utils - INFO - Success Rate: 3.33%
2026-04-01 16:50:33 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO - Per-Garment Summary
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_0: Success Rate = 0.00%, Avg Return = 114.23
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_1: Success Rate = 40.00%, Avg Return = 141.79
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_2: Success Rate = 0.00%, Avg Return = 122.96
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_3: Success Rate = 0.00%, Avg Return = 101.98
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_4: Success Rate = 0.00%, Avg Return = 109.61
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_5: Success Rate = 0.00%, Avg Return = 93.64
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_6: Success Rate = 0.00%, Avg Return = 82.86
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_7: Success Rate = 0.00%, Avg Return = 117.84
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_8: Success Rate = 0.00%, Avg Return = 90.82
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_9: Success Rate = 0.00%, Avg Return = 115.47
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO -   Pant_Long_Unseen_0: Success Rate = 0.00%, Avg Return = 119.48
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO -   Pant_Long_Unseen_1: Success Rate = 0.00%, Avg Return = 85.14
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO - Evaluation completed successfully
2026-04-01 16:50:33 - scripts.utils.evaluation - INFO - ============================================================
[5000.283s] Simulation App Shutting Down
/home/nytcee/.local/share/uv/python/cpython-3.11.14-linux-x86_64-gnu/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
(lehome) nytcee@idlab1:/mnt/train-data-1-hdd/naomi/lehome-challenge$ 

### 5-4. pant_short_merged

Garment Type: ==pant_short==
Success Rate: 28.33%

CUDA_VISIBLE_DEVICES=1 python -m scripts.eval     --policy_type lerobot     --policy_path outputs/train/xvla_four_types/checkpoints/last/pretrained_model     --garment_type "pant_short"     --dataset_root Datasets/example/pant_short_merged     --num_episodes 5     --enable_cameras     --device cpu        --save_video

2026-04-01 18:04:19 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO - Overall Summary
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 18:04:19 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-01 18:04:19 - scripts.utils.eval_utils - INFO - Evaluation Results Summary
2026-04-01 18:04:19 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-01 18:04:19 - scripts.utils.eval_utils - INFO - Total Episodes: 60
2026-04-01 18:04:19 - scripts.utils.eval_utils - INFO - Average Return: 243.61 ± 95.14
2026-04-01 18:04:19 - scripts.utils.eval_utils - INFO - Success Rate: 28.33%
2026-04-01 18:04:19 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO - Per-Garment Summary
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_0: Success Rate = 0.00%, Avg Return = 198.61
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_1: Success Rate = 0.00%, Avg Return = 259.63
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_2: Success Rate = 20.00%, Avg Return = 187.09
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_3: Success Rate = 0.00%, Avg Return = 324.11
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_4: Success Rate = 40.00%, Avg Return = 238.19
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_5: Success Rate = 20.00%, Avg Return = 311.36
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_6: Success Rate = 40.00%, Avg Return = 245.93
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_7: Success Rate = 60.00%, Avg Return = 273.75
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_8: Success Rate = 40.00%, Avg Return = 304.20
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_9: Success Rate = 80.00%, Avg Return = 179.50
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO -   Pant_Short_Unseen_0: Success Rate = 0.00%, Avg Return = 210.82
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO -   Pant_Short_Unseen_1: Success Rate = 40.00%, Avg Return = 190.11
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO - ============================================================
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO - Evaluation completed successfully
2026-04-01 18:04:19 - scripts.utils.evaluation - INFO - ============================================================

2026-04-10 16:48:35 - scripts.utils.evaluation - INFO - ============================================================
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO - Overall Summary
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO - ============================================================
2026-04-10 16:48:35 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-10 16:48:35 - scripts.utils.eval_utils - INFO - Evaluation Results Summary
2026-04-10 16:48:35 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-10 16:48:35 - scripts.utils.eval_utils - INFO - Total Episodes: 60
2026-04-10 16:48:35 - scripts.utils.eval_utils - INFO - Average Return: 184.98 ± 79.77
2026-04-10 16:48:35 - scripts.utils.eval_utils - INFO - Success Rate: 41.67%
2026-04-10 16:48:35 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO - ============================================================
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO - Per-Garment Summary
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO - ============================================================
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO -   Top_Short_Seen_0: Success Rate = 60.00%, Avg Return = 171.12
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO -   Top_Short_Seen_1: Success Rate = 20.00%, Avg Return = 170.90
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO -   Top_Short_Seen_2: Success Rate = 80.00%, Avg Return = 160.57
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO -   Top_Short_Seen_3: Success Rate = 80.00%, Avg Return = 188.67
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO -   Top_Short_Seen_4: Success Rate = 60.00%, Avg Return = 219.83
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO -   Top_Short_Seen_5: Success Rate = 80.00%, Avg Return = 121.95
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO -   Top_Short_Seen_6: Success Rate = 60.00%, Avg Return = 161.42
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO -   Top_Short_Seen_7: Success Rate = 0.00%, Avg Return = 263.26
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO -   Top_Short_Seen_8: Success Rate = 60.00%, Avg Return = 159.46
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO -   Top_Short_Seen_9: Success Rate = 0.00%, Avg Return = 212.47
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO -   Top_Short_Unseen_0: Success Rate = 0.00%, Avg Return = 153.30
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO -   Top_Short_Unseen_1: Success Rate = 0.00%, Avg Return = 236.79
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO - ============================================================
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO - Evaluation completed successfully
2026-04-10 16:48:35 - scripts.utils.evaluation - INFO - ============================================================
2026-04-10T08:48:35Z [2,109,325ms] [Error] [omni.kit.renderer.plugin] advanceCurrentFrame: backbuffers are not initialized!
[2112.387s] Simulation App Shutting Down
/home/nytcee/.local/share/uv/python/cpython-3.11.14-linux-x86_64-gnu/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
(lehome) nytcee@idlab1:/mnt/train-data-1-hdd/naomi/lehome-challenge$ CUDA_VISIBLE_DEVICES=0 python -m scripts.eval     --policy_type lerobot     --policy_path outputs/train/smolvla_top_short/checkpoints/last/pretrained_model     --garment_type "top_short"     --dataset_root Datasets/example/top_short_merged     --num_episodes 5     --enable_cameras     --device cpu        --save_video        --model_name "smolvla_top_short"

2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - ============================================================
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - Overall Summary
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - ============================================================
2026-04-10 16:35:55 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-10 16:35:55 - scripts.utils.eval_utils - INFO - Evaluation Results Summary
2026-04-10 16:35:55 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-10 16:35:55 - scripts.utils.eval_utils - INFO - Total Episodes: 60
2026-04-10 16:35:55 - scripts.utils.eval_utils - INFO - Average Return: 145.87 ± 61.00
2026-04-10 16:35:55 - scripts.utils.eval_utils - INFO - Success Rate: 71.67%
2026-04-10 16:35:55 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - ============================================================
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - Per-Garment Summary
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - ============================================================
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - Top_Long_Seen_0: Success Rate = 60.00%, Avg Return = 160.13
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - Top_Long_Seen_1: Success Rate = 100.00%, Avg Return = 137.22
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - Top_Long_Seen_2: Success Rate = 80.00%, Avg Return = 130.81
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - Top_Long_Seen_3: Success Rate = 80.00%, Avg Return = 147.52
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - Top_Long_Seen_4: Success Rate = 40.00%, Avg Return = 168.62
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - Top_Long_Seen_5: Success Rate = 60.00%, Avg Return = 174.32
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - Top_Long_Seen_6: Success Rate = 60.00%, Avg Return = 174.76
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - Top_Long_Seen_7: Success Rate = 60.00%, Avg Return = 175.12
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - Top_Long_Seen_8: Success Rate = 100.00%, Avg Return = 114.36
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - Top_Long_Seen_9: Success Rate = 80.00%, Avg Return = 138.87
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - Top_Long_Unseen_0: Success Rate = 100.00%, Avg Return = 106.22
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - Top_Long_Unseen_1: Success Rate = 40.00%, Avg Return = 122.49
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - ============================================================
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - Evaluation completed successfully
2026-04-10 16:35:55 - scripts.utils.evaluation - INFO - ============================================================
2026-04-10T08:35:55Z [1,865,364ms] [Error] [omni.kit.renderer.plugin] advanceCurrentFrame: backbuffers are not initialized!
[1867.864s] Simulation App Shutting Down
/home/nytcee/.local/share/uv/python/cpython-3.11.14-linux-x86_64-gnu/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
(lehome) nytcee@idlab1:/mnt/train-data-1-hdd/naomi/lehome-challenge$ CUDA_VISIBLE_DEVICES=0 python -m scripts.eval --policy_type lerobot --policy_path outputs/train/smolvla_top_long/checkpoints/last/pretrained_model --garment_type "top_long" --dataset_root Datasets/example/top_long_merged --num_episodes 5 --enable_cameras --device cpu --save_video --model_name "smolvla_top_long"

2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - Overall Summary
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 11:47:55 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-13 11:47:55 - scripts.utils.eval_utils - INFO - Evaluation Results Summary
2026-04-13 11:47:55 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-13 11:47:55 - scripts.utils.eval_utils - INFO - Total Episodes: 60
2026-04-13 11:47:55 - scripts.utils.eval_utils - INFO - Average Return: 122.65 ± 49.89
2026-04-13 11:47:55 - scripts.utils.eval_utils - INFO - Success Rate: 51.67%
2026-04-13 11:47:55 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - Per-Garment Summary
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - Pant_Long_Seen_0: Success Rate = 20.00%, Avg Return = 124.35
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - Pant_Long_Seen_1: Success Rate = 100.00%, Avg Return = 110.75
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - Pant_Long_Seen_2: Success Rate = 60.00%, Avg Return = 178.73
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - Pant_Long_Seen_3: Success Rate = 60.00%, Avg Return = 127.96
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - Pant_Long_Seen_4: Success Rate = 100.00%, Avg Return = 107.02
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - Pant_Long_Seen_5: Success Rate = 80.00%, Avg Return = 97.34
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - Pant_Long_Seen_6: Success Rate = 80.00%, Avg Return = 101.47
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - Pant_Long_Seen_7: Success Rate = 80.00%, Avg Return = 102.60
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - Pant_Long_Seen_8: Success Rate = 20.00%, Avg Return = 107.47
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - Pant_Long_Seen_9: Success Rate = 0.00%, Avg Return = 206.49
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - Pant_Long_Unseen_0: Success Rate = 0.00%, Avg Return = 96.07
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - Pant_Long_Unseen_1: Success Rate = 20.00%, Avg Return = 111.60
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - Evaluation completed successfully
2026-04-13 11:47:55 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13T03:47:55Z [2,000,479ms] [Error] [omni.kit.renderer.plugin] advanceCurrentFrame: backbuffers are not initialized!
[2002.960s] Simulation App Shutting Down
/home/nytcee/.local/share/uv/python/cpython-3.11.14-linux-x86_64-gnu/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
(lehome) nytcee@idlab1:/mnt/train-data-1-hdd/naomi/lehome-challenge$ CUDA_VISIBLE_DEVICES=0 python -m scripts.eval --policy_type lerobot --policy_path outputs/train/smolvla_pant_long/checkpoints/last/pretrained_model --garment_type "pant_long" --dataset_root Datasets/example/pant_long_merged --num_episodes 5 --enable_cameras --device cpu --save_video --model_name "smolvla_pant_long"

2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - Overall Summary
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 11:39:09 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-13 11:39:09 - scripts.utils.eval_utils - INFO - Evaluation Results Summary
2026-04-13 11:39:09 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-13 11:39:09 - scripts.utils.eval_utils - INFO - Total Episodes: 60
2026-04-13 11:39:09 - scripts.utils.eval_utils - INFO - Average Return: 139.67 ± 58.79
2026-04-13 11:39:09 - scripts.utils.eval_utils - INFO - Success Rate: 83.33%
2026-04-13 11:39:09 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - Per-Garment Summary
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_0: Success Rate = 100.00%, Avg Return = 106.05
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_1: Success Rate = 100.00%, Avg Return = 106.49
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_2: Success Rate = 100.00%, Avg Return = 108.58
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_3: Success Rate = 80.00%, Avg Return = 160.10
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_4: Success Rate = 100.00%, Avg Return = 91.14
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_5: Success Rate = 100.00%, Avg Return = 123.26
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_6: Success Rate = 100.00%, Avg Return = 133.46
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_7: Success Rate = 100.00%, Avg Return = 130.41
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_8: Success Rate = 100.00%, Avg Return = 159.16
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_9: Success Rate = 100.00%, Avg Return = 134.47
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - Pant_Short_Unseen_0: Success Rate = 0.00%, Avg Return = 232.16
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - Pant_Short_Unseen_1: Success Rate = 20.00%, Avg Return = 190.73
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - Evaluation completed successfully
2026-04-13 11:39:09 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13T03:39:10Z [1,299,657ms] [Error] [omni.kit.renderer.plugin] advanceCurrentFrame: backbuffers are not initialized!
[1302.415s] Simulation App Shutting Down
/home/nytcee/.local/share/uv/python/cpython-3.11.14-linux-x86_64-gnu/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
(lehome) nytcee@idlab1:/mnt/train-data-1-hdd/naomi/lehome-challenge$ CUDA_VISIBLE_DEVICES=0 python -m scripts.eval --policy_type lerobot --policy_path outputs/train/smolvla_pant_short/checkpoints/last/pretrained_model --garment_type "pant_short" --dataset_root Datasets/example/pant_short_merged --num_episodes 5 --enable_cameras --device cpu --save_video --model_name "smolvla_pant_short"

2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - Overall Summary
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 16:36:10 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-13 16:36:10 - scripts.utils.eval_utils - INFO - Evaluation Results Summary
2026-04-13 16:36:10 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-13 16:36:10 - scripts.utils.eval_utils - INFO - Total Episodes: 60
2026-04-13 16:36:10 - scripts.utils.eval_utils - INFO - Average Return: 138.93 ± 67.31
2026-04-13 16:36:10 - scripts.utils.eval_utils - INFO - Success Rate: 36.67%
2026-04-13 16:36:10 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - Per-Garment Summary
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - Pant_Long_Seen_0: Success Rate = 0.00%, Avg Return = 123.45
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - Pant_Long_Seen_1: Success Rate = 40.00%, Avg Return = 143.82
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - Pant_Long_Seen_2: Success Rate = 60.00%, Avg Return = 139.51
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - Pant_Long_Seen_3: Success Rate = 20.00%, Avg Return = 134.12
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - Pant_Long_Seen_4: Success Rate = 100.00%, Avg Return = 119.22
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - Pant_Long_Seen_5: Success Rate = 20.00%, Avg Return = 207.03
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - Pant_Long_Seen_6: Success Rate = 40.00%, Avg Return = 169.28
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - Pant_Long_Seen_7: Success Rate = 60.00%, Avg Return = 142.75
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - Pant_Long_Seen_8: Success Rate = 40.00%, Avg Return = 119.13
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - Pant_Long_Seen_9: Success Rate = 0.00%, Avg Return = 131.19
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - Pant_Long_Unseen_0: Success Rate = 40.00%, Avg Return = 146.98
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - Pant_Long_Unseen_1: Success Rate = 20.00%, Avg Return = 90.68
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - Evaluation completed successfully
2026-04-13 16:36:10 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13T08:36:10Z [2,080,328ms] [Error] [omni.kit.renderer.plugin] advanceCurrentFrame: backbuffers are not initialized!
[2083.012s] Simulation App Shutting Down
/home/nytcee/.local/share/uv/python/cpython-3.11.14-linux-x86_64-gnu/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
(lehome) nytcee@idlab1:/mnt/train-data-1-hdd/naomi/lehome-challenge$ CUDA_VISIBLE_DEVICES=0 python -m scripts.eval --policy_type lerobot --policy_path outputs/train/smolvla_four_types/checkpoints/last/pretrained_model --garment_type "pant_long" --dataset_root Datasets/example/pant_long_merged --num_episodes 5 --enable_cameras --device cpu --save_video --model_name "smolvla_4_pant_long"

2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - Overall Summary
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 16:44:09 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-13 16:44:09 - scripts.utils.eval_utils - INFO - Evaluation Results Summary
2026-04-13 16:44:09 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-13 16:44:09 - scripts.utils.eval_utils - INFO - Total Episodes: 60
2026-04-13 16:44:09 - scripts.utils.eval_utils - INFO - Average Return: 159.43 ± 93.07
2026-04-13 16:44:09 - scripts.utils.eval_utils - INFO - Success Rate: 78.33%
2026-04-13 16:44:09 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - Per-Garment Summary
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_0: Success Rate = 80.00%, Avg Return = 127.56
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_1: Success Rate = 80.00%, Avg Return = 119.15
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_2: Success Rate = 100.00%, Avg Return = 92.11
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_3: Success Rate = 60.00%, Avg Return = 214.98
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_4: Success Rate = 100.00%, Avg Return = 170.44
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_5: Success Rate = 80.00%, Avg Return = 178.19
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_6: Success Rate = 80.00%, Avg Return = 176.25
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_7: Success Rate = 100.00%, Avg Return = 118.01
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_8: Success Rate = 100.00%, Avg Return = 138.26
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - Pant_Short_Seen_9: Success Rate = 100.00%, Avg Return = 123.82
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - Pant_Short_Unseen_0: Success Rate = 40.00%, Avg Return = 212.53
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - Pant_Short_Unseen_1: Success Rate = 20.00%, Avg Return = 241.90
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - Evaluation completed successfully
2026-04-13 16:44:09 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13T08:44:09Z [1,325,138ms] [Error] [omni.kit.renderer.plugin] advanceCurrentFrame: backbuffers are not initialized!
[1327.828s] Simulation App Shutting Down
/home/nytcee/.local/share/uv/python/cpython-3.11.14-linux-x86_64-gnu/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
(lehome) nytcee@idlab1:/mnt/train-data-1-hdd/naomi/lehome-challenge$ CUDA_VISIBLE_DEVICES=0 python -m scripts.eval --policy_type lerobot --policy_path outputs/train/smolvla_four_types/checkpoints/last/pretrained_model --garment_type "pant_short" --dataset_root Datasets/example/pant_short_merged --num_episodes 5 --enable_cameras --device cpu --save_video --model_name "smolvla_4_pant_short"

2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - Overall Summary
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 17:26:13 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-13 17:26:13 - scripts.utils.eval_utils - INFO - Evaluation Results Summary
2026-04-13 17:26:13 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-13 17:26:13 - scripts.utils.eval_utils - INFO - Total Episodes: 60
2026-04-13 17:26:13 - scripts.utils.eval_utils - INFO - Average Return: 158.02 ± 63.66
2026-04-13 17:26:13 - scripts.utils.eval_utils - INFO - Success Rate: 58.33%
2026-04-13 17:26:13 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - Per-Garment Summary
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - Top_Long_Seen_0: Success Rate = 80.00%, Avg Return = 126.63
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - Top_Long_Seen_1: Success Rate = 60.00%, Avg Return = 182.59
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - Top_Long_Seen_2: Success Rate = 40.00%, Avg Return = 183.13
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - Top_Long_Seen_3: Success Rate = 80.00%, Avg Return = 207.71
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - Top_Long_Seen_4: Success Rate = 40.00%, Avg Return = 136.31
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - Top_Long_Seen_5: Success Rate = 80.00%, Avg Return = 121.22
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - Top_Long_Seen_6: Success Rate = 80.00%, Avg Return = 125.13
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - Top_Long_Seen_7: Success Rate = 0.00%, Avg Return = 194.24
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - Top_Long_Seen_8: Success Rate = 100.00%, Avg Return = 128.79
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - Top_Long_Seen_9: Success Rate = 80.00%, Avg Return = 172.34
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - Top_Long_Unseen_0: Success Rate = 40.00%, Avg Return = 167.29
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - Top_Long_Unseen_1: Success Rate = 20.00%, Avg Return = 150.91
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - Evaluation completed successfully
2026-04-13 17:26:13 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13T09:26:13Z [2,237,279ms] [Error] [omni.kit.renderer.plugin] advanceCurrentFrame: backbuffers are not initialized!
[2241.587s] Simulation App Shutting Down
/home/nytcee/.local/share/uv/python/cpython-3.11.14-linux-x86_64-gnu/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
(lehome) nytcee@idlab1:/mnt/train-data-1-hdd/naomi/lehome-challenge$ CUDA_VISIBLE_DEVICES=0 python -m scripts.eval --policy_type lerobot --policy_path outputs/train/smolvla_four_types/checkpoints/last/pretrained_model --garment_type "top_long" --dataset_root Datasets/example/top_long_merged --num_episodes 5 --enable_cameras --device cpu --save_video --model_name "smolvla_4_top_long"

2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - Overall Summary
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 17:33:18 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-13 17:33:18 - scripts.utils.eval_utils - INFO - Evaluation Results Summary
2026-04-13 17:33:18 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-13 17:33:18 - scripts.utils.eval_utils - INFO - Total Episodes: 60
2026-04-13 17:33:18 - scripts.utils.eval_utils - INFO - Average Return: 191.77 ± 66.99
2026-04-13 17:33:18 - scripts.utils.eval_utils - INFO - Success Rate: 15.00%
2026-04-13 17:33:18 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - Per-Garment Summary
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - Top_Short_Seen_0: Success Rate = 20.00%, Avg Return = 185.75
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - Top_Short_Seen_1: Success Rate = 0.00%, Avg Return = 170.39
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - Top_Short_Seen_2: Success Rate = 40.00%, Avg Return = 210.23
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - Top_Short_Seen_3: Success Rate = 20.00%, Avg Return = 238.36
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - Top_Short_Seen_4: Success Rate = 20.00%, Avg Return = 289.22
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - Top_Short_Seen_5: Success Rate = 0.00%, Avg Return = 211.37
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - Top_Short_Seen_6: Success Rate = 40.00%, Avg Return = 162.41
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - Top_Short_Seen_7: Success Rate = 0.00%, Avg Return = 178.77
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - Top_Short_Seen_8: Success Rate = 0.00%, Avg Return = 160.07
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - Top_Short_Seen_9: Success Rate = 20.00%, Avg Return = 155.52
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - Top_Short_Unseen_0: Success Rate = 0.00%, Avg Return = 188.15
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - Top_Short_Unseen_1: Success Rate = 20.00%, Avg Return = 150.96
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - Evaluation completed successfully
2026-04-13 17:33:18 - scripts.utils.evaluation - INFO - ============================================================
2026-04-13T09:33:19Z [2,606,610ms] [Error] [omni.kit.renderer.plugin] advanceCurrentFrame: backbuffers are not initialized!
[2609.447s] Simulation App Shutting Down
/home/nytcee/.local/share/uv/python/cpython-3.11.14-linux-x86_64-gnu/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
(lehome) nytcee@idlab1:/mnt/train-data-1-hdd/naomi/lehome-challenge$ CUDA_VISIBLE_DEVICES=0 python -m scripts.eval --policy_type lerobot --policy_path outputs/train/smolvla_four_types/checkpoints/last/pretrained_model --garment_type "top_short" --dataset_root Datasets/example/top_short_merged --num_episodes 5 --enable_cameras --device cpu --save_video --model_name "smolvla_4_top_short"

2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - Overall Summary
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 14:36:31 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-14 14:36:31 - scripts.utils.eval_utils - INFO - Evaluation Results Summary
2026-04-14 14:36:31 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-14 14:36:31 - scripts.utils.eval_utils - INFO - Total Episodes: 60
2026-04-14 14:36:31 - scripts.utils.eval_utils - INFO - Average Return: 135.63 ± 45.36
2026-04-14 14:36:31 - scripts.utils.eval_utils - INFO - Success Rate: 31.67%
2026-04-14 14:36:31 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - Per-Garment Summary
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - Top_Long_Seen_0: Success Rate = 80.00%, Avg Return = 131.55
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - Top_Long_Seen_1: Success Rate = 20.00%, Avg Return = 152.18
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - Top_Long_Seen_2: Success Rate = 20.00%, Avg Return = 88.35
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - Top_Long_Seen_3: Success Rate = 0.00%, Avg Return = 132.22
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - Top_Long_Seen_4: Success Rate = 80.00%, Avg Return = 138.23
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - Top_Long_Seen_5: Success Rate = 0.00%, Avg Return = 134.94
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - Top_Long_Seen_6: Success Rate = 20.00%, Avg Return = 150.08
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - Top_Long_Seen_7: Success Rate = 20.00%, Avg Return = 121.73
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - Top_Long_Seen_8: Success Rate = 0.00%, Avg Return = 119.36
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - Top_Long_Seen_9: Success Rate = 80.00%, Avg Return = 175.77
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - Top_Long_Unseen_0: Success Rate = 60.00%, Avg Return = 169.53
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - Top_Long_Unseen_1: Success Rate = 0.00%, Avg Return = 113.63
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - Evaluation completed successfully
2026-04-14 14:36:31 - scripts.utils.evaluation - INFO - ============================================================
[3458.254s] Simulation App Shutting Down
/home/nytcee/.local/share/uv/python/cpython-3.11.14-linux-x86_64-gnu/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
(lehome) nytcee@idlab1:/mnt/train-data-1-hdd/naomi/lehome-challenge$ CUDA_VISIBLE_DEVICES=1 python -m scripts.eval --policy_type lerobot --policy_path outputs/train/act_four_types/checkpoints/last/pretrained_model --garment_type "top_long" --dataset_root Datasets/example/top_long_merged --num_episodes 5 --enable_cameras --device cpu --save_video --model_name "act_4_top_long"

2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - Overall Summary
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 14:38:35 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-14 14:38:35 - scripts.utils.eval_utils - INFO - Evaluation Results Summary
2026-04-14 14:38:35 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-14 14:38:35 - scripts.utils.eval_utils - INFO - Total Episodes: 60
2026-04-14 14:38:35 - scripts.utils.eval_utils - INFO - Average Return: 170.94 ± 64.63
2026-04-14 14:38:35 - scripts.utils.eval_utils - INFO - Success Rate: 16.67%
2026-04-14 14:38:35 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - Per-Garment Summary
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - Top_Short_Seen_0: Success Rate = 0.00%, Avg Return = 190.10
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - Top_Short_Seen_1: Success Rate = 0.00%, Avg Return = 228.74
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - Top_Short_Seen_2: Success Rate = 0.00%, Avg Return = 184.06
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - Top_Short_Seen_3: Success Rate = 20.00%, Avg Return = 179.08
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - Top_Short_Seen_4: Success Rate = 60.00%, Avg Return = 198.22
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - Top_Short_Seen_5: Success Rate = 20.00%, Avg Return = 143.19
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - Top_Short_Seen_6: Success Rate = 0.00%, Avg Return = 160.31
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - Top_Short_Seen_7: Success Rate = 20.00%, Avg Return = 145.26
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - Top_Short_Seen_8: Success Rate = 40.00%, Avg Return = 141.22
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - Top_Short_Seen_9: Success Rate = 0.00%, Avg Return = 207.08
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - Top_Short_Unseen_0: Success Rate = 0.00%, Avg Return = 108.37
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - Top_Short_Unseen_1: Success Rate = 40.00%, Avg Return = 165.65
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - Evaluation completed successfully
2026-04-14 14:38:35 - scripts.utils.evaluation - INFO - ============================================================
[3606.993s] Simulation App Shutting Down
/home/nytcee/.local/share/uv/python/cpython-3.11.14-linux-x86_64-gnu/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
(lehome) nytcee@idlab1:/mnt/train-data-1-hdd/naomi/lehome-challenge$ CUDA_VISIBLE_DEVICES=1 python -m scripts.eval --policy_type lerobot --policy_path outputs/train/act_four_types/checkpoints/last/pretrained_model --garment_type "top_short" --dataset_root Datasets/example/top_short_merged --num_episodes 5 --enable_cameras --device cpu --save_video --model_name "act_4_top_short"

2026-04-14 16:28:20 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO - Overall Summary
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 16:28:20 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-14 16:28:20 - scripts.utils.eval_utils - INFO - Evaluation Results Summary
2026-04-14 16:28:20 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-14 16:28:20 - scripts.utils.eval_utils - INFO - Total Episodes: 60
2026-04-14 16:28:20 - scripts.utils.eval_utils - INFO - Average Return: 165.06 ± 102.73
2026-04-14 16:28:20 - scripts.utils.eval_utils - INFO - Success Rate: 75.00%
2026-04-14 16:28:20 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO - Per-Garment Summary
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_0: Success Rate = 20.00%, Avg Return = 278.81
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_1: Success Rate = 80.00%, Avg Return = 125.58
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_2: Success Rate = 80.00%, Avg Return = 134.66
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_3: Success Rate = 80.00%, Avg Return = 175.79
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_4: Success Rate = 100.00%, Avg Return = 78.28
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_5: Success Rate = 100.00%, Avg Return = 124.76
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_6: Success Rate = 100.00%, Avg Return = 133.70
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_7: Success Rate = 100.00%, Avg Return = 120.26
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_8: Success Rate = 80.00%, Avg Return = 184.62
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO -   Pant_Short_Seen_9: Success Rate = 80.00%, Avg Return = 167.47
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO -   Pant_Short_Unseen_0: Success Rate = 0.00%, Avg Return = 296.05
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO -   Pant_Short_Unseen_1: Success Rate = 80.00%, Avg Return = 160.72
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO - Evaluation completed successfully
2026-04-14 16:28:20 - scripts.utils.evaluation - INFO - ============================================================
[1669.296s] Simulation App Shutting Down
/home/nytcee/.local/share/uv/python/cpython-3.11.14-linux-x86_64-gnu/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
(lehome) nytcee@idlab1:/mnt/train-data-1-hdd/naomi/lehome-challenge$ CUDA_VISIBLE_DEVICES=1 python -m scripts.eval     --policy_type lerobot     --policy_path outputs/train/act_four_types/checkpoints/last/pretrained_model     --garment_type "pant_short"     --dataset_root Datasets/example/pant_short_merged     --num_episodes 5     --enable_cameras     --device cpu        --save_video        --model_name "act_4_pant_short"

2026-04-14 16:42:05 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO - Overall Summary
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 16:42:05 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-14 16:42:05 - scripts.utils.eval_utils - INFO - Evaluation Results Summary
2026-04-14 16:42:05 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-14 16:42:05 - scripts.utils.eval_utils - INFO - Total Episodes: 60
2026-04-14 16:42:05 - scripts.utils.eval_utils - INFO - Average Return: 138.13 ± 70.30
2026-04-14 16:42:05 - scripts.utils.eval_utils - INFO - Success Rate: 33.33%
2026-04-14 16:42:05 - scripts.utils.eval_utils - INFO - ==================================================
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO - Per-Garment Summary
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_0: Success Rate = 20.00%, Avg Return = 114.24
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_1: Success Rate = 40.00%, Avg Return = 145.57
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_2: Success Rate = 20.00%, Avg Return = 211.35
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_3: Success Rate = 0.00%, Avg Return = 158.45
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_4: Success Rate = 80.00%, Avg Return = 123.74
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_5: Success Rate = 60.00%, Avg Return = 125.46
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_6: Success Rate = 100.00%, Avg Return = 92.87
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_7: Success Rate = 20.00%, Avg Return = 214.23
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_8: Success Rate = 20.00%, Avg Return = 133.18
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO -   Pant_Long_Seen_9: Success Rate = 40.00%, Avg Return = 143.23
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO -   Pant_Long_Unseen_0: Success Rate = 0.00%, Avg Return = 94.64
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO -   Pant_Long_Unseen_1: Success Rate = 0.00%, Avg Return = 100.65
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO - ============================================================
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO - Evaluation completed successfully
2026-04-14 16:42:05 - scripts.utils.evaluation - INFO - ============================================================
[2592.057s] Simulation App Shutting Down
/home/nytcee/.local/share/uv/python/cpython-3.11.14-linux-x86_64-gnu/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
(lehome) nytcee@idlab1:/mnt/train-data-1-hdd/naomi/lehome-challenge$ CUDA_VISIBLE_DEVICES=1 python -m scripts.eval     --policy_type lerobot     --policy_path outputs/train/act_four_types/checkpoints/last/pretrained_model     --garment_type "pant_long"     --dataset_root Datasets/example/pant_long_merged     --num_episodes 5     --enable_cameras     --device cpu        --save_video        --model_name "act_4_pant_long"

(base) [nytcee@slurm-ui02 IDLAB]$ cd Naomi
(base) [nytcee@slurm-ui02 Naomi]$ tmux new -s lehome
-bash: tmux: command not found
(base) [nytcee@slurm-ui02 Naomi]$ srun --partition=a100_long-al9 --nodes=1 --ntasks=1 --cpus-per-task=8 --gres=gpu:1 --mem=256G --pty bash
(base) [nytcee@hp-teslaa01 Naomi]$ module load singularity/4.1.2
Loading singularity/4.1.2
  Loading requirement: golang/1.21.7
(base) [nytcee@hp-teslaa01 Naomi]$ singularity shell --nv -B /ceph/work/IDLAB/Naomi:/workspace /ceph/work/IDLAB/Naomi/isaac-sim_sandbox.sif
Singularity> cd /workspace/lehome-challenge
Singularity> source .venv/bin/activate
(lehome) Singularity> nvidia-smi
Wed Apr 22 07:32:08 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 560.35.03              Driver Version: 560.35.03      CUDA Version: 12.6     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA A100-SXM4-80GB          Off |   00000000:48:00.0 Off |                    0 |
| N/A   24C    P0             59W /  400W |       1MiB /  81920MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
(lehome) Singularity> tmux new -s lehome
bash: tmux: command not found
(lehome) Singularity> cd lehome-challenge
bash: cd: lehome-challenge: No such file or directory
(lehome) Singularity> ls
Assets	Datasets  LICENSE  README.md  configs  docs  export  hf_login.py  logs	outputs  ov_temp  pyproject.toml  scripts  source  third_party	uv.lock
(lehome) Singularity> lerobot-train --config_path=configs/train_act.yaml --dataset.video_backend pyav

lerobot-train \
    --config_path=configs/train_act.yaml \
    --dataset.repo_id=local_mix_dataset \
    --dataset.root=Datasets/Mix_Top_Long_Dataset/final_training_data_top_long \
    --dataset.video_backend=pyav \
    --batch_size=8 \
    --num_workers=0