使用NVIDIA Isaac Lab-Arena與LeRobot在模擬環境中評估通用型機器人策略

Generalist Robot Policy Evaluation in Simulation with NVIDIA Isaac Lab-Arena and LeRobot

Robots must perceive unstructured environments, reason and plan under uncertainty, and execute actions safely in real time on physical systems. To do this, they need to develop an understanding of the physical world through three forms of computational intelligence: training, simulation, and on-robot inference. Each stage relies on distinct hardware and software, and each plays a critical role in the end-to-end robotics development pipeline.

NVIDIA and Hugging Face are integrating NVIDIA’s open Isaac and GR00T technologies, including robot simulation and learning frameworks, models, and hardware systems, across these three computers into the LeRobot library.

This will accelerate open-source physical AI development by bringing NVIDIA's 2 million robotics developers and Hugging Face’s more than 13 million AI builders together.

Developers get access to open-source pre-trained Isaac GR00T N vision language action (VLA) models, physical AI datasets, evaluation frameworks like NVIDIA Isaac Lab-Arena, and hardware platforms such as the Reachy 2 humanoid running on NVIDIA Jetson Thor.

In this blog, we show how to evaluate VLA policies using Isaac Lab-Arena in LeRobot EnvHub, making robot environments easier to author, share, and reuse across the community.

NVIDIA Isaac Lab-Arena and LeRobot Integration

Isaac Lab-Arena is an open source framework for efficient and scalable robotic policy evaluation in simulation, with the evaluation and task layers designed in close collaboration with Lightwheel.

Hugging Face’s LeRobot EnvHub feature enables developers to share simulation environments, and easily load them for training, evaluation, or teleoperation, directly from the LeRobot framework.

Isaac Lab-Arena is now integrated into Hugging Face’s LeRobot Environment Hub, making it easier for developers to build, experiment with, and collaborate on robotics simulation. Developers can now easily prototype complex and diverse simulation environments using Isaac Lab-Arena, register these environments on LeRobot EnvHub and seamlessly use them to train and evaluate robot policies, such as GR00T N, Pi, SmolVLA, among other policies available in LeRobot. All of this happens within the LeRobot ecosystem, providing streamlined access to powerful simulation and training tools.

How to Evaluate a VLA on Isaac Lab-Arena Environments available on LeRobot

Prerequisites

Setup

Evaluate SmolVLA

In the first stage, install SmolVLA on your machine following the command below. Please also remember to install numpy to 1.26.0

When the installation is completed, we can run the LeRobot evaluation on Isaac Lab-Arena with the command below

If you want to go more in-depth and also evaluate PI0.5 follow the documentation on the LeRobot documentation.

How to create and register new Isaac Lab-Arena Environments on LeRobot EnvHub

While the previous section demonstrated a sample workflow using Isaac Lab-Arena sample environments already available on EnvHub, developers are encouraged to share custom environments with the community.

Lightwheel Robocasa and LIBERO Task Suites on Isaac Lab-Arena

Lightwheel has adopted the Isaac Lab-Arena framework to create and open-source 250+ tasks through the Lightwheel-RoboCasa-Tasks and Lightwheel-LIBERO-Tasks suites -
LeRobot developers can now seamlessly use them thanks to this integration work.

Once a policy has been evaluated in simulation, it can be deployed on robot systems such as Reachy 2, powered by NVIDIA Jetson Thor⚡

Get Started

Here are some additional resources to explore how to work with Isaac Lab-Arena and LeRobot:

*This blog was co-authored with Steven Palma, Jade Choghari of Hugging Face

Community

·
Sign up or
log in to comment