As the need for high-quality training data grows, synthetic data generation has become essential for improving LLM performance. Instruction-tuned models are commonly used for this task, but they often struggle to generate diverse outputs, which is crucial for model generalization. Despite efforts such as prompting techniques that encourage variation—like conditioning on past outputs or assuming different personas—the diversity remains limited. In contrast, base models, which lack post-training biases, generate more diverse responses but tend to be lower in quality. Studies show that base models produce outputs with lower pairwise cosine similarity, indicating greater diversity, while instruct-tuned models risk mode collapse.
Synthetic data is widely used in training state-of-the-art models for reasoning, coding, and problem-solving tasks. Still, its overuse can lead to issues such as iterative degradation, where models generate increasingly homogenized outputs. Existing approaches to enhance diversity—such as temperature scaling, nucleus sampling, and multi-stage generation—offer partial solutions but often require significant manual effort. While downstream performance is the standard metric for evaluating synthetic data, embedding-based measures like BERTScore provide better insights into semantic diversity. Additionally, assessing the quality of individual synthetic samples remains a challenge, necessitating more robust evaluation frameworks.
Researchers from UC Berkeley, Stanford, Foundry, Microsoft Research, and Princeton propose a synthetic data generation method that integrates base and instruct-tuned models to balance diversity and quality. Their approach, Base-Refine (BARE), follows a two-stage process where base model outputs are refined using instruct-tuned models, enhancing dataset quality while preserving diversity. Fine-tuning with just 1,000 BARE-generated samples achieves performance comparable to top models on LiveCodeBench and improves GSM8K accuracy by 101% over instruct-only data. BARE also boosts RAFT-based fine-tuning by 18.4%, demonstrating its effectiveness in generating high-quality, diverse data for various machine-learning tasks.
BARE is a synthetic data generation method that enhances dataset quality by refining diverse base model outputs with instruct-tuned models. The process begins with a base model generating an initial dataset with minimal few-shot examples. Then, an instruct-tuned model improves each sample by correcting errors and enhancing clarity while preserving diversity. This two-stage approach ensures high-quality yet varied data, making BARE particularly effective in data-scarce domains. With only three few-shot examples and general prompts, BARE minimizes human effort while maximizing flexibility. Experimental results show its potential to generate more accurate and diverse synthetic datasets for machine learning tasks.
The evaluation of BARE focuses on diversity, data quality, and downstream performance across the same domains and baselines discussed earlier. Implementing Llama-3.1-70B-Base for initial generation and Llama-3.1-70B-Instruct for refinement, BARE maintains data diversity while improving generation quality. Fine-tuning experiments show BARE outperforms base and instruct models, enhancing model accuracy across multiple datasets. Notably, refining with GPT-4o further boosts performance. Ablation studies confirm that using a base model is essential for diversity, as refining instruct-only outputs lowers accuracy. Overall, BARE effectively integrates base and instruct-tuned models to generate high-quality synthetic data for improved downstream tasks.
In conclusion, the study quantitatively examines synthetic data generation methods, revealing that base models ensure diversity while instruct-tuned models enhance quality. BARE integrates both to generate high-quality, diverse data. Extensive experiments validate its effectiveness, improving downstream tasks like GSM8K, LiveCodeBench, and RAFT, setting a new state-of-the-art. Future work could refine the process through fine-tuned refiners, additional stages, or alternative training objectives. Beyond synthetic training data, BARE can also create diverse evaluation datasets. As synthetic data becomes essential for model training, BARE offers a scalable solution that balances diversity and quality, outperforming existing methods in various domains.
Check out the Paper and GitHub Page. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. Don’t Forget to join our 75k+ ML SubReddit.
Recommended Open-Source AI Platform: ‘IntellAgent is a An Open-Source Multi-Agent Framework to Evaluate Complex Conversational AI System’ (Promoted)The post BARE: A Synthetic Data Generation AI Method that Combines the Diversity of Base Models with the Quality of Instruct-Tuned Models appeared first on MarkTechPost.
