Deploying NVIDIA Omniverse Nucleus on Amazon EC2

<section class="blog-post-content"><h2>Introduction</h2>This post aims to get users up and running with NVIDIA Omniverse Enterprise Nucleus Server on <a href="https://aws.amazon.com/ec2/">Amazon Elastic Compute Cloud (Amazon EC2)</a>. Here I’ll outline the requirements for Enterprise Nucleus Server deployment and dive deep into the technical steps for getting Nucleus running in your <a href="https://aws.amazon.com/">Amazon Web Services (AWS)</a> account.<h2>What is Omniverse?</h2>NVIDIA Omniverse is a scalable, multi-GPU, real-time platform for building and operating metaverse applications, based on Pixar’s Universal Scene Description (USD) and NVIDIA RTX technology.NVIDIA Omniverse Nucleus is the database and collaboration engine of Omniverse. With Omniverse Nucleus, teams can have multiple live users connected using different applications at once. This allows people to use the application they are most comfortable with and opens a lot of doors for rapid iteration. <a href="https://docs.omniverse.nvidia.com/prod_nucleus/prod_nucleus/overview.html">Learn more on the NVIDIA Omniverse Introduction Documentation</a>.Nucleus operates under a publish-and-subscribe model and enables efficient live synchronization between NVIDIA Omniverse applications. Changes to USD scenes are transmitted in real-time between connected Omniverse clients. Clients connect using the publish-and-subscribe pattern, which makes it possible for them to receive changes submitted in near real-time.Other Nucleus features include users and group management, assets access control lists (ACLs) for fine-grained access control, versioning with checkpoints, single sign-on (SSO) with SAML Authentication, and TLS encryption support.<h2>Why AWS?</h2>There are multiple reasons to deploy Nucleus on the AWS Global Cloud Infrastructure. With AWS you can connect distributed users all over the globe. Our security, identity, and access management controls allow you to retain complete control over your data. Also, with the variety of compute instance types and storage solutions AWS offers, you can right size your infrastructure and fine tune performance as needed.<h2>Solution Overview</h2>The following steps outline a solution that implements the basic components of a Nucleus deployment. To handle communication from end users an Amazon EC2 instance configured as a NGINX reverse proxy is deployed in a public subnet. The reverse proxy accepts TLS traffic and has a TLS certificate from <a href="https://aws.amazon.com/certificate-manager/">Amazon Certificates Manager (ACM)</a>. Typically, this component would be an <a href="https://aws.amazon.com/elasticloadbalancing/">Elastic Load Balancer (ELB)</a>, but the Nucleus Server requires path rewrites in the request which is not currently supported by an ELB.The Enterprise Nucleus Server is an Amazon EC2 instance deployed to a private subnet that only accepts traffic from the reverse proxy subnet. The Enterprise Nucleus Server is running the Nucleus Enterprise Stack, which is deployed as a Docker Compose Stack. The Nucleus instance will need a NAT Gateway and Internet Gateway to communicate with the NVIDIA NGC. This procedure uses the basic Nucleus stack with TLS support, and not SSO.<h2>Prerequisites</h2><h2>Deploying Omniverse Nucleus on Amazon EC2</h2><img class="aligncenter wp-image-63 size-large" src="https://d2908q01vomqb2.cloudfront.net/a17554a0d2b15a664c0e73900184544f19e70227/2022/09/22/Deploying-NVIDIA-Omniverse-Nucleus-on-Amazon-EC2-architecture-1024x687.png" alt="Omniverse Enterprise Nucleus on Amazon EC2 Architecture" width="1024" height="687" /><h3>Register a domain and create a hosted zone with Amazon Route 53</h3>First, you will need a hosted zone and a domain for the Nucleus Server. <a href="https://aws.amazon.com/route53/">Amazon Route 53 (Route 53)</a> allows registration of a domain, such as my-omniverse.com, and creation of a subdomain, such as nucleus.my-omniverse.com, for the Nucleus Server. When registering a domain, communication occurs with the domain registrar. It is best to do this step manually and then reference the Hosted Zone ID, created by Route 53, in the subsequent configuration steps.See this page for more information on registering a domain and creating a hosted zone: <a href="https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/domain-register.html">Registering a new domain</a>.<h3>Configure the CDK Stack</h3>Next, you will configure a CDK stack with basic resources for the Nucleus deployment.Step 1. Open a terminal and create a project folder for your CDK appThe name of the folder will become the application name. For this procedure, nucleus-app is the name used.Step 2. Change directory into the folder created in Step 1 and initialize your CDK app with the following command:Now your project structure should be the following:<code>nucleus-app.ts</code> is the main entry point for the app and the file that subsequent CDK commands will reference. When viewing this file, you can see it imports <code>lib/nucleus-app-stack.ts</code>, which is where you’ll put custom code for your deployment.Step 3. Run a basic “Hello World” testDeploy the starter CDK stack with <code>cdk deploy</code>. This will produce a basic stack and confirm your CLI and CDK are properly configured.Step 4. Set default account and AWS Region environment valuesOpen <code>bin/nucleus-app.ts</code> and set the default account and Region environment (env) values. The contents of the file should look like the following:Step 5. Remove sample resourcesOpen <code>lib/nucleus-app-stack.ts</code> and remove the sample <a href="https://aws.amazon.com/sqs/">Amazon Simple Queue Service (SQS)</a> and <a href="https://aws.amazon.com/sns/">Amazon Simple Notification Service (SNS)</a> resources. Your file should now look like the following:Step 6. Add the below CDK libraries, as these are required in subsequent steps<h3>Define Stack Resources</h3>Next, you will define custom infrastructure resources required for the deployment. Code samples in this section need to be added inside the constructor of the NucleusAppStack classStep 1. Create an <a href="https://docs.aws.amazon.com/s3/?id=docs_gateway">Amazon Simple Storage Service (Amazon S3)</a> bucket for artifactsFirst, create a simple Amazon S3 bucket that will be used to transfer artifacts from our local client to Amazon EC2 instances. As per security best practices, enable encryption, enforce SSL, and block public access. Then create an <a href="https://aws.amazon.com/iam/">AWS Identity and Access Management (IAM)</a> policy that allows access to list bucket and get objects from the bucket. This policy will be attached to our Amazon EC2 instance profile role.Step 2. Add an <a href="https://aws.amazon.com/vpc/">Amazon Virtual Private Cloud (VPC)</a> configurationSpecify the private subnet that contains the NAT gateway with a route to the internet. Then provision two security groups that the proxy server and the Nucleus Server will use.Step 3. Add security group ingress rulesConfigure the proxy and Nucleus security groups to allow traffic on required ports. The Nucleus security group only allows traffic from the proxy security group. The proxy security group allows traffic from a specific CIDR range. You’ll want to set this to a range you will use to connect to the server. For example, you can use the IP address of the client machine you plan to connect from. Then, you enter that IP appended with a network mask as the CIDR range. For this solution, the recommended network mask is /32.Step 4. Add TLS Certificate and set the domain from Step 1 for validationNote: the root-domain variable must be set to the domain registered with the Route 53 hosted zone from Step 1.Note: Currently there is no additional management of this CNAME record. Meaning when you no longer require it, you’ll have to remove it manually from your Route 53 Hosted Zone.Step 5. Add reverse proxy resourcesFor the reverse proxy, configure it with Nitro Enclaves enabled. Enclaves provides features to create isolated compute environments to protect and securely process highly sensitive data. In this case that’s our TLS certificate. On top of that, Nitro Enclaves has support for integration with Amazon Certificates Manager. This means Certificates Manager can automatically handle the rotation of the certificate. For more information, see <a href="https://docs.aws.amazon.com/enclaves/latest/user/nitro-enclave.html">AWS Nitro Enclaves User Guide</a>.Starting from the Certificate Manager for Nitro Enclaves AMI, create a c5.xlarge instance with 32GB of storage. In this case c5.xlarge was chosen as one of the smallest available instances required for the Nitro Enclaves AMI. Configure a basic instance role with the AmazonSSMManagedInstanceCore policy. This allows you to connect to the instance with <a href="https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent.html">AWS Systems Manager (SSM)</a> and avoid opening the instance to SSH traffic over the internet.Finally, attach a “dummy” IAM policy to the reverse proxy. This is an empty policy which will get updated with the configuration scripts.Note, if your Region is not in the list of Regions below, review the AMI listing on the AWS Marketplace <a href="https://aws.amazon.com/marketplace/pp/prodview-f4gcl7narsmle">AWS Certificate Manager for Nitro Enclaves</a> or the AWS Documentation for finding the correct AMI ID, <a href="https://docs.aws.amazon.com/managedservices/latest/userguide/find-ami.html">Finding AMI IDs</a>.<div class="hide-language"><pre class="lang-ts">// AWS Certificate Manager for Nitro Enclaves AMI const proxyServerAMI = new ec2.GenericLinuxImage({ 'us-west-1': 'ami-0213075968e811ea7', //california 'us-west-2': 'ami-01c4415fd6c2f0927', //oregon 'us-east-1': 'ami-00d96e5ee00daa484', //virginia 'us-east-2': 'ami-020ea706ac260de21', //ohio 'ca-central-1': 'ami-096dd1150b96b6125', //canada 'eu-central-1': 'ami-06a2b19f6b97762cb', //frankfurt 'eu-west-1': 'ami-069e205c9dea19322', //ireland 'eu-west-2': 'ami-069b79a2d7d0d9408' //london }) const proxyInstanceRole = new iam.Role(this, 'proxyInstanceRole', { assumedBy: new iam.ServicePrincipal('ec2.amazonaws.com'), description: 'EC2 Instance Role', managedPolicies: [ iam.ManagedPolicy.fromAwsManagedPolicyName('AmazonSSMManagedInstanceCore') ], inlinePolicies: { getBucketObjectsPolicy: getBucketObjectsPolicy } }); const cfnInstanceProfile = new iam.CfnInstanceProfile(this, 'proxyInstanceProfile', { roles: [proxyInstanceRole.roleName], }); // using CfnInstance because it exposes enclaveOptions const cfnInstance = new ec2.CfnInstance(this, 'reverseProxyServer', { blockDeviceMappings: [{ deviceName: '/dev/xvda', ebs: { encrypted: true, volumeSize: 32, } }], enclaveOptions: { enabled: true, }, imageId: proxyServerAMI.getImage(this).imageId, instanceType: 'c5.xlarge', securityGroupIds: [proxySG.securityGroupId], subnetId: vpc.selectSubnets({subnetGroupName: 'publicSubnetNatGateway'}).subnetIds[0], tags: [{ key: 'Name', value: 'Nucleus-ReverseProxy', }], iamInstanceProfile: cfnInstanceProfile.ref }) new route53.CnameRecord(this, CnameApiRecord, { recordName: fullDomain, zone: hostedZone, domainName: cfnInstance.attrPublicDnsName, }); const revProxyCertAssociationPolicy = new iam.ManagedPolicy(this, 'revProxyCertAssociationPolicy', { statements: [ new iam.PolicyStatement({ actions: ["s3:GetObject"], resources: ["*"] }) ] }) proxyInstanceRole.addManagedPolicy(revProxyCertAssociationPolicy)</pre></div>Step 6. Add Nucleus Server resourcesNext, configure the Nucleus Server. Start with the Ubuntu, 20.04 LTS AMI with c5.4xlarge as the instance type. C5 instances are optimized for compute-intensive workloads and deliver cost-effective high performance at a low price per compute ratio. The instance has 16 vCPUs and 32GB of Memory. An <a href="https://aws.amazon.com/ebs/">Amazon Elastic Block Store (EBS)</a> volume is attached to the instance with 512GB of storage. These specs were chosen to be sufficiently large for a proof of concept.The instance user data script is configured to install docker, docker-compose, and the AWS CLI.Step 7. Configure stack outputsNext, add output values so you can easily reference them later.Step 8. Deploy the stackOnce this is complete, you will have the basic resources required and next you will configure them.If you encounter the following CDK deploy error:Check that you have the correct domain specified and that your hosted zone exists in the Route 53 console <a href="https://us-east-1.console.aws.amazon.com/route53/v2/hostedzones">Route 53 Hosted zones</a>.Step 9. Note the stack output values. You’ll use them in the future<h3>Configure The Reverse Proxy Server</h3>Step 1. Associate Enclave certificate with proxy instance IAM roleThe first thing you have to do with the reverse proxy is associate your certificate with the IAM role that the Nitro Enclave uses. In the following code, please replace <code>tls-certificate-arn</code>, <code>proxy-instance-role-arn</code>, <code>proxy-cert-association-policy-arn</code>, and <code>region</code> in the below script with stack output values from above.Note: The following script was written in Python 3.9. If you have issues with conflicting python versions. It’s recommended that you set a local <code>virtualenv</code>. For more information, see <a href="https://docs.python.org/3/tutorial/venv.html">Python Tutorial Virtual Environments and Packages</a>.This script associates an identity and IAM role with an <a href="https://aws.amazon.com/certificate-manager/">AWS Certificate Manager (ACM)</a> certificate. This enables the certificate to be used by the ACM for Nitro Enclaves application inside an enclave. For more information, see <a href="https://docs.aws.amazon.com/enclaves/latest/user/nitro-enclave-refapp.html">Certificate Manager for Nitro Enclaves</a> in the Amazon Web Services Nitro Enclaves User Guide. The script then updates the IAM role policy with permissions to get its own role, download the certificate, and decrypt it.Save the script to a file and run it from the terminal:Step 2. Configure Nginx confNVIDIA provides a sample Nginx config for the Nucleus deployment. It is packaged within a provided archive file. At the time of writing this, the latest was <code>nucleus-stack-2022.1.0+tag-2022.1.0.gitlab.3983146.613004ac.tar.gz</code>Open the archive and look for: ssl/nginx.ingress.router.confThis file needs to be updated and then placed at /etc/nginx/nginx.conf on the reverse proxy instance.First, you need to update the config with configuration outlined in the AWS Certificate Manager for Nitro Enclaves guide: <a href="http://%20https//docs.aws.amazon.com/enclaves/latest/user/nitro-enclave-refapp.html">Nitro Enclaves application: AWS Certificate Manager for Nitro Enclaves</a>.At the top of the file, in the main context add the following:After the line, # Configure your SSL options as required by your security practices, add the below snippet:Next, update the config file with the Nucleus Server private DNS address and the fully qualified domain for your server. Replace instances of <code>my-ssl-nucleus.my-company.com</code> with your domain. Then, replace instances of <code>BASE_STACK_IP_OR_HOST</code> with the <code>nucleusServerPrivateDnsName</code> from the stack outputs above.Step 3. Copy the .conf file to Amazon S3Step 4. Connect to the proxy instanceFrom your web browser, navigate to the <a href="https://console.aws.amazon.com/ec2/v2/home">EC2 Dashboard</a> in the AWS Console, select the Nucleus-ReverseProxy instance, and press the Connect button.Select the Session Manager tab, then press the Connect button.<img class="aligncenter wp-image-62" src="https://d2908q01vomqb2.cloudfront.net/a17554a0d2b15a664c0e73900184544f19e70227/2022/09/22/Deploying-NVIDIA-Omniverse-Nucleus-on-Amazon-EC2-image-1.png" alt="Session Manager tab of the Nucleus Reverse Proxy instance" width="768" height="315" />Step 5. In the terminal, copy the <code>nginx.conf</code> file path from Amazon S3 to <code>/etc/nginx/</code>Step 6. While still in the proxy server terminal, rename the sample ACM for Nitro Enclaves configuration file from <code>/etc/nitro_enclaves/acm.example.yaml</code> to <code>/etc/nitro_enclaves/acm.yaml</code> using the following command:Step 7. Update the <code>acm.yaml certificate_arn</code> valueUsing your preferred text editor, open <code>/etc/nitro_enclaves/acm.yaml</code>. In the ACM section, update <code>certificate_arn</code>, with the ARN of the certificate from our stack. This is the <code>tls-certificate-arn</code> from the stack outputs above. Save and close the file.Step 8. Start the Nginx serverStep 9. Confirm the server is accepting TLS requests to your domainYou’ll see a generic HTML template as output.<h3>Configure Nucleus Server</h3>Much of the following comes from NVIDIA’s documentation on deploying a Nucleus Server. Review these docs for more information: <a href="https://docs.omniverse.nvidia.com/prod_nucleus/prod_nucleus/enterprise/installation/quick_start_tips.html#enterprise-nucleus-server-quick-start-tips">Enterprise Nucleus Server Quick Start Tips</a>.Step 1. From your local computer using the AWS CLI, copy the Nucleus Stack archive to Amazon S3Step 2. Connect to the Nucleus Server with EC2 Session ManagerWith your web browser, navigate to the EC2 Dashboard in the AWS Console, select the Nucleus-Server instances, press the Connect button, and then press the Connect button again on the Session Manager tab.Step 3. In the Nucleus-Server terminal, change directory to the home directory, and then copy the Nucleus Stack from S3Step 4. Unpack the archive to an appropriate directory, then <code>cd</code> into that directoryStep 5. Update nucleus-stack.envWith your preferred text editor, review the nucleus-stack.env file. It is recommended that you review this file in its entirety. You will use this file to confirm that you accept the NVIDIA Omniverse end user license agreement.Then update the following <code>nucleus-stack.env</code> variables as neededStep 6. Generate secrets required for authenticationNote the following is required because you are not using SSO integration at this time. See the security notes in nucleus-stack.env for more information.Step 7. Pull the Nucleus docker imagessudo docker-compose –env-file ${omniverse_root}/base_stack/nucleus-stack.env -f ${omniverse_root}/base_stack/nucleus-stack-ssl.yml pullStep 8. Start the Nucleus stack<h3>Usage</h3>Back on your local machine, test a connection to your Nucleus Server by pointing your web browser to the domain you specified in the <code>.env</code> file. You should be greeted with the following login dialog:<img class="aligncenter wp-image-64 size-medium" src="https://d2908q01vomqb2.cloudfront.net/a17554a0d2b15a664c0e73900184544f19e70227/2022/09/22/Deploying-NVIDIA-Omniverse-Nucleus-on-Amazon-EC2-image-2-263x300.png" alt="Omniverse Login Window" width="263" height="300" />Here you can use the Master or Service Username and Password configured in the <code>nucleus-stack.env</code>, or press Create Account. Then you’ll be presented with a navigator view of your Nucleus Server content<img class="aligncenter wp-image-65 size-full" src="https://d2908q01vomqb2.cloudfront.net/a17554a0d2b15a664c0e73900184544f19e70227/2022/09/22/Deploying-NVIDIA-Omniverse-Nucleus-on-Amazon-EC2-image-3.png" alt="Omniverse Nucleus Server content window" width="936" height="210" /><h3>Cleanup</h3>Step 1. Disassociate the Nitro Enclave certificate by running the dissacociate_enclave_cert.py scriptStep 2. Delete the stack by running <code>cdk desktroy</code> from the nucleus-app application folder.<h3>Conclusion</h3>This post provides the basics to get up and running with NVIDIA Omniverse Nucleus on Amazon EC2 using the Docker Compose container. This post walked through the setup procedures of the Amazon EC2 Nucleus and reverse proxy servers, implemented S3 for storage and retrieval of configuration files, and Route 53 private hosted zones for secure, private access to your Omniverse data.This deployment of Nucleus on Amazon EC2 allows your teams, no matter where they are located, to collaborate and interact in real-time while building 3D products, applications, and experiences.To learn more about spatial computing at AWS, continue following along here on the Spatial Computing Blog channel.<h2>Additional Reading</h2>This information may also be found on the AWS GitHub repository, <a href="https://github.com/aws-samples/nvidia-omniverse-nucleus-on-amazon-ec2">NVIDIA Omniverse Nucleus on Amazon EC2</a>.<h3>AWS Services</h3><a href="https://aws.amazon.com/ec2/">Amazon EC2</a>, secure and resizable compute capacity for virtually any workload<a href="https://aws.amazon.com/route53/">Amazon Route 53</a>, a reliable and cost-effective way to route end users to Internet applications<a href="https://aws.amazon.com/s3/">Amazon S3</a>, object storage built to retrieve any amount of data from anywhere<a href="https://aws.amazon.com/ebs/">Amazon EBS</a>, easy to use, high performance block storage at any scale<a href="https://docs.aws.amazon.com/enclaves/latest/user/nitro-enclave-refapp.html">AWS Certificates Manager for Nitro Enclaves</a>, public and private TLS certificates with web servers running on Amazon EC2 instances<h3>NVIDIA Omniverse Nucleus</h3><a href="https://docs.omniverse.nvidia.com/prod_nucleus/prod_nucleus/overview.html">Nucleus Overview</a><a href="https://docs.omniverse.nvidia.com/prod_nucleus/prod_nucleus.html">Nucleus Documentation</a></section>

Fish AI Reader

FishAI

联系邮箱 441953276@qq.com

相关标签