The NVIDIA AI Blueprint for RAG provides developers with a foundational starting point for building scalable and customizable retrieval pipelines that deliver high-accuracy and throughput. Use the blueprint to create a RAG application that delivers context-aware responses, connecting LLMs to large corpora of enterprise data, to enable actionable insights grounded in relevant data. The blueprint can be used as is, or combined with other NVIDIA Blueprints to address more advanced use cases including digital humans and AI assistants for customer service. Get started with this reference architecture to enhance decision-making and productivity.Architecture DiagramKey FeaturesOpenAI-compatible APIsMulti-turn conversationsMulti-collectionMulti-session supportMultilingual and cross-lingual retrievalOptimized data storageConfigurability options for NIM selection and NIM endpointsReranking usageMinimum System RequirementsHardware RequirementsThe blueprint by default uses API endpoints, making it very easy to experience without needing GPUs.It is expected that the NIM microservices will need to be self-hosted as you progress in your RAG development. For self-hosting the blueprint with these microservices locally deployed, the recommended system requirement is 8XH100-80GB or 8XA100-80GB with the Llama 3.1 70b NIM, the NeMo Retriever embedding and reranking NIM, and the Milvus database accelerated with NVIDIA cuVS.OS RequirementsDeployment OptionsSoftware used in this blueprintNIM microservicesNVIDIA Technology3rd Party SoftwareEthical ConsiderationsNVIDIA believes Trustworthy AI is a shared responsibility, and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their supporting model team to ensure the models meet requirements for the relevant industry and use case and address unforeseen product misuse. For more detailed information on ethical considerations for the models, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards. Please report security vulnerabilities or NVIDIA AI concerns here.LicenseUse of the models in this blueprint is governed by the NVIDIA AI Foundation Models Community License.Terms of UseGOVERNING TERMS: The software and materials are governed by the NVIDIA Software License Agreement and the Product-Specific Terms for NVIDIA AI Products, except that models are governed by the AI Foundation Models Community License Agreement and the NVIDIA RAG dataset is governed by the NVIDIA Asset License Agreement. ADDITIONAL INFORMATION: for Meta/llama-3.1-70b-instruct model the Llama 3.1 Community License Agreement, for nvidia/llama-3.2-nv-embedqa-1b-v2model the Llama 3.2 Community License Agreement, and for the nvidia/llama-3.2-nv-rerankqa-1b-v2 model the Llama 3.2 Community License Agreement. Built with Llama.
