Spatial computing experiences are transforming how we interact with data, connecting the physical and digital worlds through technologies like extended reality (XR) and digital twins. These advancements are enabling more intuitive and immersive ways to analyze and understand complex datasets. This post explains how developers can now engage with Universal Scene Description (OpenUSD)-based NVIDIA Omniverse datasets in immersive environments, bringing new levels of understanding and insights. A unique hybrid rendering solutionCreating immersive spatial experiences poses unique challenges. Often, XR devices cannot render complex datasets at the level of detail needed for realistic and engaging experiences. Hardware limitations can result in reduced fidelity, making it difficult to preserve intricate details. To address these challenges, NVIDIA announced at CES 2025 that the Omniverse platform now includes the spatial streaming for Omniverse digital twins workflow that enables XR experiences with unmatched visual fidelity and performance. Partners Accenture, Innoactive, and Katana have begun to integrate this workflow into their existing spatial workflows.This hybrid rendering workflow combines local and cloud-based rendering using NVIDIA RTX GPUs and NVIDIA Graphics Delivery Network (GDN). This approach ensures high-quality immersive experiences while reducing the strain on local devices. Developers can now stream detailed, high-fidelity OpenUSD-based Omniverse digital twins to Apple Vision Pro.
Figure 1. The spatial streaming for Omniverse digital twins workflow enables XR experiences with unmatched visual fidelity and performanceSetting up spatial streamingFollow the steps below to build your own hybrid rendering application and stream immersive experiences using the Omniverse platform and Apple Vision Pro.1. Create an Omniverse Kit ApplicationBegin by creating an Omniverse Kit Application. This will act as the server for your dataset, handling computationally intensive rendering tasks.2. Add XR extensionsEnable spatial streaming by including the XR extensions bundle. This simplifies setup and ensures all necessary components are included.3. Build and run the Kit ApplicationBuild and run the Kit Application to verify it processes and streams data correctly.4. Set up the client projectOpen Xcode on your Mac and configure the Client project. This will render the final visuals on Apple Vision Pro.5. Test with Apple Vision Pro simulatorUse Apple Vision Pro simulator in Xcode to test and debug without needing a physical headset.6. Configure the sample projectAdjust the provided sample project to work with your specific dataset. The default configuration uses a Purse USD dataset as a reference.7. Connect to the Omniverse ServerFinally, link your Client project to the Omniverse Server. Communication between the two is managed through ActionGraph Logic in Omniverse.Key considerations for setting up spatial streamingThere are several components needed in order to set up your spatial streaming environment, detailed below. Kit Application as the server: Handles rendering tasks to reduce local hardware strain, ensuring smooth operation.XR extensions: Simplifies development and minimizes setup errors.Build and run early: Identifies potential issues in the server setup before proceeding.Xcode client project: Manages seamless communication between the server and local device.Simulator testing: Speeds up development by eliminating dependency on physical hardware.Sample project: Provides a robust starting point to customize for specific datasets.Best practices for performanceOptimizing renderingEnsure optimal performance by minimizing latency and optimizing network conditions. This maximizes the responsiveness and quality of spatial experiences.Lighting and contentAdhere to best practices for lighting and content design to achieve visually stunning, realistic environments that enhance user engagement.
Figure 2. View of a wind turbine through Apple Vision Pro, powered by NVIDIA Omniverse. Image courtesy of InnoactiveBenefits of hybrid renderingStreaming immersive experiences with the NVIDIA hybrid rendering workflow delivers significant advantages:High-fidelity viewing: Maintain full dataset fidelity without compromising quality. Now a large, enterprise dataset can be rendered with intricate detail and streamed to the high resolution displays of the Apple Vision Pro.Performance and responsiveness: Cloud GPUs can render more complex datasets, handle more AI workflows, and scale to reach many simultaneous users.Cost and energy savings: Leverage scalable cloud resources to reduce costs and energy consumption. Accessibility: Enable immersive digital twin experiences without requiring expensive, powerful workstations. This democratizes access to high-quality XR applications.Sustainability: Reduce material waste and production costs by relying on digital models instead of physical prototypes. For example, digital rendering can save up to $9,000 per project.Leveraging Apple Vision Pro: Fully utilize advanced features like high-resolution displays and spatial audio, delivering unmatched user experiences.Next stepsGet started with spatial streaming for Omniverse digital twins today. To learn more, check out these resources:Stay up-to-date by subscribing to NVIDIA news, and following NVIDIA Omniverse on Discord, YouTube, and Medium. Watch the keynote from NVIDIA CEO Jensen Huang at CES 2025.
