At CES 2025, NVIDIA announced key updates to NVIDIA Isaac, a platform of accelerated libraries, application frameworks, and AI models that accelerate the development of AI robots.NVIDIA Isaac streamlines the development of robotic systems from simulation to real-world deployment. In this post, we discuss all the new advances in NVIDIA Isaac:Isaac SimIsaac LabIsaac ManipulatorIsaac PerceptorWhat’s new in Isaac Sim 4.5NVIDIA Isaac Sim is a reference application built on NVIDIA Omniverse that enables you to develop, simulate, and test AI-driven robots in physically based virtual environments.The new Isaac Sim 4.5 will offer a number of significant changes, including the following:A reference application templateImproved URDF import and setupImproved physics simulation and modelingNew joint visualization toolSimulation accuracy and statisticsNVIDIA Cosmos world foundation modelReference application templateIsaac Sim has been redesigned as a customizable reference application. A minimal template for faster startup and a full template with complete functionality along with all dependencies. This enables you to tailor the application to your specific needs, whether for headless applications or full Isaac Sim experiences.Improved URDF import and setupSignificant improvements have been made to the URDF importer. The User Interface has been simplified to provide a more streamlined process and align with how other formats are imported. You can now individually configure joint drives, making the robot ready to use immediately after import. To assist with joint drive configuration, a natural frequency-based tuning option is provided. Improved physics simulation and modelingIsaac Sim 4.5 features significant advancements in physics modeling and simulation. You can define and configure various joint types between robot components, setting parameters like stiffness and damping to fine-tune joint behavior. A new joint visualization tool enables you to inspect the physics properties of selected prims, including their position, rotation, linear and angular velocities, and accelerations. You can now review and optimize various scene parameters such as deformable surfaces or memory usage, before running simulations. Figure 1. Various attributes of selected objects and their association with the other objectsSimulation accuracy and statisticsSimulation accuracy is greatly improved with a new implementation of full momentum conservation for rigid bodies and articulations. You can now also visualize the simulation statistics of objects and scenes that can either interact with each other or are completely independent. You can review different parameters from deformable surfaces to overall memory used in the buffer. This provides a way to troubleshoot and optimize your scene prior to running the simulation. Figure 2. Simulation statistics associated inside a given sceneNVIDIA Cosmos world foundation modelAlso announced at CES, the NVIDIA Cosmos world foundation model platform can be used to generate massive amounts of controllable synthetic data to train perception robots when paired with Isaac Sim. In Isaac Sim, you compose SimReady 3D scenes by unifying diverse data inputs including CAD, lidar-to-point cloud scans, and generated 3D objects from AI models such as Edify 3D. Then, you compose and stage the scenario to reflect the specific task the robot must perform, and render images or videos. Cosmos can ingest images and videos and output photoreal video clips to then retrain policy models.Video 1. NVIDIA Cosmos: A World Foundation Model Platform for Physical AIWhat’s new in Isaac Lab 2.0NVIDIA Isaac Lab is an open-source unified framework for robot learning to train robot policies. Isaac Lab is built on top of NVIDIA Isaac Sim, helping developers and researchers more efficiently build intelligent, adaptable robots with robust, perception-enabled, simulation-trained policies.A new 2.0 version of Isaac Lab includes performance and usability improvements:Tiled rendering: Up to a 1.2x boost in tiled rendering speed, which combines outputs from simultaneous simulations into a single, large image rather than processing numerous smaller images from individual cameras.Quality of life improvements: Simplified installation process using Python package managers. Isaac Lab will also be available as a container, enabling the movement of workloads across systems without underlying dependencies.Previewed at CES, humanoid robot developers can also take advantage of the NVIDIA Isaac GR00T Blueprint for building custom data pipelines for generating vast amounts of synthetic trajectory data from just a small number of human demonstrations. The GR00T blueprint is currently in invite-only early access. Join the NVIDIA Humanoid Developer Program when it becomes widely available in beta.Video 2. Streamline Data Collection With NVIDIA Isaac GR00TWhat’s new in Isaac ManipulatorNVIDIA Isaac Manipulator, built on ROS 2, is a collection of NVIDIA CUDA-accelerated libraries, AI models, and reference workflows. It enables you to build AI-enabled robot arms, or manipulators, that can perceive, understand, and interact with their environments. Isaac Manipulator now includes new end-to-end reference workflows for pick-and-place and object-following, enabling you to quickly get started on fundamental industrial robot arm tasks:Object-following: Shows the robot gripper’s ability to maintain a consistent position relative to a moving object, while maneuvering around obstacles.Pick-and-place: Shows how a robot can pick up an object and release it in a predetermined region while avoiding obstacles (Figure 3). Figure 3. Perception-driven pick-and-placeThese reference workflows are now supported in Isaac Sim, enabling rapid testing without the need for physical hardware setup. Developer support and other enhancements include the following:What’s new in Isaac PerceptorNVIDIA Isaac Perceptor, built on ROS 2, is a collection of NVIDIA CUDA-accelerated libraries, AI models, and reference workflows for the development of autonomous mobile robots (AMRs). It enables AMRs to perceive, localize, and operate in unstructured environments such as warehouses or factories. Isaac Perceptor encompasses CUDA-accelerated libraries such as nvblox for 3D scene reconstruction and cuVSLAM for stereo-visual-inertial SLAM (simultaneous localization and mapping), which you can integrate into existing AMR workflows.Isaac Perceptor’s latest updates bring significant improvements to AMR’s environmental awareness and operational efficiency in dynamic settings such as warehouses. Key new features and improvements include:These updates significantly improve 3D scene reconstruction, leading to higher accuracy and robustness in 3D scene capture and mapping performance in real-world scenarios and complex, dynamic environments.Figure 4. 3D scene reconstruction using multiple camerasNVIDIA ecosystem partners such as Orbbec, LIPS (Realsense), StereoLabs (Zed) offer compatible cameras. Developer support also includes the Mapping and Localization with Isaac Perceptor tutorial for offline mapping capabilities with the cuVGL and cuVSLAM libraries using Nova sensors.Ecosystem adoption Multiple industry partners have announced integrations of NVIDIA Isaac into their platforms and solutions:Boston Dynamics is using Isaac Lab and NVIDIA Jetson AGX Orin to enable simulated policies to be directly deployed for inference, simplifying the deployment process.To train their GR-1 and GR-2 humanoid robots, the Fourier team turned to NVIDIA Isaac Gym (now deprecated) for reinforcement learning. They are currently porting their workflows to NVIDIA Isaac Lab. Foxglove has developed an extension in Isaac Sim that enables the real-time visualization of robotics simulation data directly in Foxglove.Main Street Autonomy’s Calibration Anywhere software automates sensor calibration improving sensor fusion for robotics using Isaac Perceptor.Miso Robotics automates kitchen tasks using advanced robotics, Isaac Manipulator, and NVIDIA Isaac ROS, enhancing efficiency, consistency, and customer satisfaction in commercial kitchens.RGo Robotics and NVIDIA are transforming mobile robotics with advanced AI and perception technologies using Isaac Perceptor and Isaac ROS.Scaled Foundations, an NVIDIA inception member, has developed General Robot Intelligence Development (GRID), an advanced cloud-based platform that accelerates robot AI solution development. GRID seamlessly integrates Isaac Sim and Isaac Lab technologies, offering an end-to-end platform for robotics developers and researchers to train, simulate, and deploy their robotics applications. For more information, see NVIDIA Isaac Sim on GRID. Virtual Incision is using NVIDIA platforms involving Holoscan, IGX, Sensor Bridge, Isaac Sim. They are exploring Cosmos to train, simulate, and test, and also exploring inference AI features for the next generation of assisted robotic surgery devices. NVIDIA Inception member and deep tech startup Wandelbots is building custom robot simulations with their operating system Wandelbots NOVA, which is seamlessly integrated with Isaac Sim. Get started developing your own robotics solutionsSign up for the NVIDIA Developer Program for updates on additional resources and reference architectures to support your development goals.NVIDIA Cosmos is a platform that helps you build custom world models for physical AI systems and includes pretrained world foundation models for robotics applications.NVIDIA Isaac Lab is an open-source unified framework for robot learning to train robot policies.NVIDIA Isaac ROS, built on the open-source ROS 2 software framework, is a collection of accelerated computing packages and AI models, bringing NVIDIA-acceleration to ROS developers everywhere.NVIDIA Isaac Sim, built on NVIDIA Omniverse, lets you build your own OpenUSD-based applications to design, simulate, test, and train AI-based robots and machines in a physically based virtual environment.Stay up-to-date on LinkedIn, Instagram, X, and Facebook. For more information, explore our documentation and join the Robotics community on the developer forums and YouTube channels. Follow our self-paced training and webinars (Isaac ROS and Isaac Sim).
