MarkTechPost@AI 2024年08月13日
Outperforming Existing Models with Multi-Pass Refinement: This AI Paper from Amazon Unveils a New Era in Code Suggestion Tools
index_new5.html
../../../zaker_core/zaker_tpl_static/wap/tpl_guoji1.html

 

亚马逊和牛津大学的研究者提出新方法,用于处理含潜在错误的部分代码,以提升AI编程工具的可靠性和效率

🏷亚马逊和牛津大学的研究者面临处理含潜在错误的部分代码这一挑战,当前方法存在不足,他们提出新的解决方案。该方案将部分代码视为‘实施提示’,通过一次生成和多次迭代改进的策略,生成正确且功能完整的代码

🏷此方法的核心技术进步在于对包括InCoder、CodeGen和StarCoder等先进的代码语言模型进行微调。通过在专门构建的数据集上训练,这些模型能够有效处理有错误和干净的代码

🏷该方法的性能通过多个基准进行评估,实验表明微调后的模型在从有错误的前缀生成功能代码方面优于基线方法,特别是多次迭代改进策略,在各种性能指标上实现了更高的准确性

Handling partial code with potential bugs presents a significant challenge in developing real-time code suggestion systems. Incomplete code snippets often exhibit errors, necessitating accurate completion that also addresses embedded bugs to enhance the reliability and efficiency of AI-driven programming tools. The primary challenge involves developing models capable of generating code completions while simultaneously correcting potential errors within the partial code, thereby producing fully functional and accurate code.

Current approaches to code completion generate code based on a given prefix or problem description but struggle with partial code that contains potential bugs. Previous methodologies, such as “buggy-code completion,” aim to complete code with undesirable elements but often produce non-functional outputs due to an inability to effectively correct errors. Models like CodeGen and InCoder typically rely on straightforward, linear completion strategies that do not accommodate the complexities of debugging and code correction. The main limitations of these methods are their computational intensity and inadequacy for real-time applications, where rapid and accurate code correction is necessary.

Researchers from Amazon and the University of Oxford propose a novel approach that fine-tunes large language models of code (CodeLLMs) for the dual task of rewriting and completing partial code. This proposed method treats partial code as “implementation hints,” allowing the model to deviate from the provided code to generate a correct and functional completion. The innovation lies in the application of two strategies: one-pass generation and multi-pass iterative refinement. One-pass generation attempts to create a complete program from the partial code in a single step, while the multi-pass iterative refinement strategy generates an initial solution and then refines it iteratively, progressively fixing potential bugs to enhance code accuracy.

The core technical advancement in this approach involves fine-tuning state-of-the-art CodeLLMs, including InCoder, CodeGen, and StarCoder, on datasets specifically constructed for this task. These datasets incorporate semantic-altering transformations that introduce potential bugs into clean code snippets, enabling the models to learn to handle both buggy and clean code effectively. The one-pass generation method trains the model to predict the entire program in a single forward pass from the partial code, while the multi-pass iterative refinement method generates a solution and then iteratively refines it. The performance of these models is evaluated using pass rates on multiple benchmarks, such as the newly created b-HumanEval and b-FixEval datasets, which focus on the model’s ability to manage buggy code prefixes.

Experiments reveal that the fine-tuned models consistently outperform baseline methods in generating functional code from buggy prefixes. The multi-pass iterative refinement strategy proves particularly effective, achieving higher accuracy across various performance metrics, including pass@1, pass@10, and pass@100. These models demonstrate a significant improvement in handling partial code with potential bugs, offering more reliable and accurate code completions compared to previous approaches. The results underscore the practical effectiveness of the proposed method in real-world scenarios, where it successfully addresses the challenges of real-time code suggestion by not only completing but also correcting buggy code snippets. This advancement enhances the robustness and reliability of AI-driven programming tools, making them better suited for real-time application in diverse coding environments.

In conclusion, This proposed method significantly advances the field of AI-driven code completion by enabling CodeLLMs to jointly rewrite and complete partial code with potential bugs. The introduction and evaluation of both one-pass and multi-pass iterative refinement strategies substantially improve the functional accuracy of generated code, particularly in the presence of bugs. This development promises to make AI programming assistants more robust and reliable, especially in handling real-world, in-progress code.


Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter..

Don’t Forget to join our 48k+ ML SubReddit

Find Upcoming AI Webinars here


The post Outperforming Existing Models with Multi-Pass Refinement: This AI Paper from Amazon Unveils a New Era in Code Suggestion Tools appeared first on MarkTechPost.

Fish AI Reader

Fish AI Reader

AI辅助创作,多种专业模板,深度分析,高质量内容生成。从观点提取到深度思考,FishAI为您提供全方位的创作支持。新版本引入自定义参数,让您的创作更加个性化和精准。

FishAI

FishAI

鱼阅,AI 时代的下一个智能信息助手,助你摆脱信息焦虑

联系邮箱 441953276@qq.com

相关标签

亚马逊 代码建议 AI编程
相关文章