Magpie-ultra 是由 Argilla 团队发布的一个用于监督微调的新数据集，包含 50,000 个指令-响应对。这个合成生成的数据集利用了先进的 Llama 3.1 405B-Instruct 模型和其他 Llama 模型，如 Llama-Guard-3-8B 和 Meta-Llama-3.1-8B-Instruct。该数据集涵盖了各种任务，包括编码、数学、数据分析、创意写作、寻求建议和头脑风暴，提供了具有挑战性的指令和响应以增强 AI 模型训练。

🤖 该数据集是使用 distilabel 创建的，其创建遵循了 Magpie 食谱，如论文“Magpie：通过提示对齐的 LLM 使用无物从头开始的对齐数据合成”中所述。与最初的 Magpie 版本不同，此版本采用了新的 Llama 3.1 模型系列，并生成了一个更集中的 50,000 个指令-响应对，相比之前的 100 万个。

💡 该管道利用了各种模型来生成指令、创建响应、评估质量和进行安全分类。生成过程涉及一台 8xH100 机器，指令-响应对的创建大约需要 60 个小时。其他步骤，如使用基础模型生成响应、计算嵌入、评估质量和难度以及对指令进行分类，总共需要大约 51 个小时。

📝 该数据集的结构包括各种列，提供了有关每个指令-响应对的丰富信息。关键列包括指令本身、来自指令和基础模型的响应、意图、所需知识、难度级别、质量评估和类别分类。此外，该数据集还使用 Llama-Guard-3-8B 进行安全检查，并为每个指令提供嵌入信息。

🚀 该数据集的优势之一在于其潜在的应用。它可以用于监督微调 (SFT) 或直接偏好优化 (DPO)，具体取决于指令和基础模型响应之间的分数差异。这种灵活性允许研究人员和开发人员将数据集调整到他们在 AI 模型训练和优化中的特定需求。

⚠️ 尽管此版本标志着 AI 训练数据向前迈出的重要一步，但重要的是要注意其局限性。此版本未经过滤，计划在将来发布经过过滤的版本。此外，该数据集可能需要更加平衡，这个问题将在即将发布的版本中得到解决。尽管存在这些局限性，Magpie-ultra 仍然是推进各个领域 AI 能力的宝贵资源。

Magpie-ultra, a new dataset by the Argilla team for supervised fine-tuning, has been released, featuring 50,000 instruction-response pairs. This synthetically generated dataset utilizes the advanced Llama 3.1 405B-Instruct model and other Llama models like Llama-Guard-3-8B and Meta-Llama-3.1-8B-Instruct. The dataset covers various tasks, including coding, mathematics, data analysis, creative writing, advice-seeking, and brainstorming, offering challenging instructions and responses to enhance AI model training.

This dataset is created with distilabel, and the dataset’s creation follows the Magpie recipe, as outlined in the paper “Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing.” This iteration differs from the original Magpie release by employing the new Llama 3.1 family of models and generating a more focused set of 50,000 instruction-response pairs, compared to the previous 1 million. The pipeline utilizes various models for instruction generation, response creation, quality assessment, and safety classification.

The generation process involved a single 8xH100 machine, with the instruction-response pair creation taking approximately 60 hours. Additional steps, such as generating responses with the base model, computing embeddings, assessing quality and difficulty, and classifying instructions, required about 51 hours combined. This efficient process resulted in a comprehensive dataset with multiple data points for each entry.

The dataset’s structure includes various columns providing rich information about each instruction-response pair. Key columns include the instruction itself, responses from both instruct and base models, intent, required knowledge, difficulty level, quality assessment, and category classification. Also, the dataset incorporates safety checks using Llama-Guard-3-8B and provides embedding information for each instruction.

One of the dataset’s strengths lies in its potential applications. It can be used for Supervised Fine-Tuning (SFT) or Direct Preference Optimization (DPO), depending on the score difference between instruct and base model responses. This flexibility allows researchers and developers to tailor the dataset to their specific needs in AI model training and optimization.

While this release marks a significant step forward in AI training data, it’s important to note its limitations. This version is unfiltered, with a filtered version planned for future release. Also, the dataset may need to be more balanced, an issue that will be addressed in upcoming iterations. Despite these limitations, Magpie-ultra represents a valuable resource for advancing AI capabilities across various domains.

---

Check out the Pipeline and Dataset. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter..

Don’t Forget to join our 47k+ ML SubReddit

Find Upcoming AI Webinars here

---

Arcee AI Released DistillKit: An Open Source, Easy-to-Use Tool Transforming Model Distillation for Creating Efficient, High-Performance Small Language Models

The post Magpie-Ultra Dataset Released: Harnessing Llama 3.1 405B for Diverse AI Instruction-Response Pairs appeared first on MarkTechPost.