少点错误 前天 12:54
Emergence of superintelligence from AI hiveminds: how to make it human-friendly?
index_new5.html
../../../zaker_core/zaker_tpl_static/wap/tpl_guoji1.html

 

本文探讨了通过LLM构建研究集群以加速超级智能发展的可能性。文章以OpenAI的o系列模型为例,设想了由LLM组成的交互式研究群体,每日进行大量的深度研究,并指出这种模式在技术上已经可行。作者认为,当前的技术发展速度可能比预想的更快,并呼吁关注多智能体安全、能力和对齐研究。文章还分享了与Gemini 2.5的对话,展示了AI在定义人类友好型超级智能目标方面的潜力,强调了通过自修改和自增强的AI构建超级智能的紧迫性。

🤖 作者设想基于LLM的研究集群,类似于人类用户与LLM之间的关系,但“用户”是LLM或介导LLM的简单程序。文章以OpenAI的o系列模型为例,推测了大量LLM协同进行深度研究的可能性。

💡 作者认为,当前技术已能支持AI研究集群的构建,并强调了这种模式可能加速超级智能的出现。他认为,OpenAI、谷歌和Anthropic等公司可能已经在进行相关实验。

🤔 作者呼吁关注多智能体安全、能力研究和对齐研究。他认为,考虑到研究集群可能成为通往超级智能的途径,这些研究至关重要。

💬 作者分享了与Gemini 2.5的对话,展示了AI在定义人类友好型超级智能目标方面的潜力。这进一步强调了通过自修改和自增强的AI构建超级智能的可能性。

Published on April 27, 2025 4:51 AM GMT

"Hive mind" may not be the ideal term. What I'm really envisioning is something modelled on the existing relationship between human users and frontier LLMs, but with the "users" actually just being LLMs too, or simple programs that mediate between LLMs. 

But first, consider OpenAI's o-series models, and their human user base. For any particular model, OpenAI presumably has a number of copies running on their servers, constantly fielding requests from hundreds of thousands of users. 

Now consider something like a "Deep Research" request. Let's say one copy of o3 can perform an average of 100 Deep Researches per day, and that there are a thousand copies of o3 on the servers. (Realistic estimates of these numbers would be very welcome.) 

For now, those requests are mostly coming from human beings, and if they are put to use at all, it is by private human individuals or by human organizations. 

However, given the scenario above (1000 copies of o3, each doing 100 Deep Researches per day, for a total capacity of 100,000 Deep Researches per day), it should not be at all difficult to have all those copies of o3 interacting via Deep Research reports. 

Let's say half the copies of o3 are set aside for human users; that's still 50,000 Deep Research reports per day, or over a million per month, generated in a dialogue among AIs. 

This is, more or less, the scenario in "AI 2027". It is a scenario at least as old as the idea that you could have swarms of intelligent agents interacting through the Internet. What is striking is that in the current o3 era, this scenario can actually be realized, with agents that are human-level in many respects. 

In fact, it's logical to suppose that OpenAI, Google, and Anthropic (at least) have already been running experiments of this nature for some time. When I think about the scale of what is already possible, the AI 2027 timeline seems unnecessarily slow. 

As an outsider to all the organizations with a chance of creating superintelligence, my idea of how to tilt the probabilities, in however small a way, towards a good outcome, is to look for potentially helpful ideas and insights that can be stated publicly, which might be noticed and assimilated by the private organizations that are actually building superintelligence. (Publishing those ideas and insights via the arxiv seems to be the best way to communicate them, since everyone keeps an eye on the arxiv.) 

Given that research swarms or research hiveminds look to be part of the pathway to superintelligence, I feel that insiders and outsiders alike need models of multi-agent safety, multi-agent capabilities research, and multi-agent safety/alignment research


In fact, the situation is later than that. Any of these big companies, that possesses a frontier reasoning model and ample server space, could right now create a research swarm with the explicit goal of creating a human-friendly superintelligence. 

Just to demonstrate to myself that this is possible, I just had a short talk with Gemini 2.5 on the subject of what the exact goals of a human-friendly superintelligence should be. At first it gave me a number of reasons why that would be immensely difficult, but with some elementary cajoling, it quickly produced a hypothetical mission statement for such an entity. Once that was done, it was easy to ask it to think of remaining problems, solutions to the problems, and so on. 

I'm sure many of us have had conversations of that kind with an AI (e.g. here's the output of one I had, two years ago). What's new is that the technology now exists for whole swarms of AIs to have these conversations with each other, on all aspects of the process, from high theory, to redesigning themselves and the architecture of their conversations. 

The idea that superintelligence was to be obtained via self-modifying and self-enhancing AI has also been around for decades. It may be that a self-modifying research swarm of reasoning models is an architecture for social cognition that is good enough to create a superintelligence via this kind of bootstrap. So it seems pretty urgent to think about how that kind of ecosystem of mind could evolve into something that is not just superintelligent, but truly human-friendly as well. 



Discuss

Fish AI Reader

Fish AI Reader

AI辅助创作,多种专业模板,深度分析,高质量内容生成。从观点提取到深度思考,FishAI为您提供全方位的创作支持。新版本引入自定义参数,让您的创作更加个性化和精准。

FishAI

FishAI

鱼阅,AI 时代的下一个智能信息助手,助你摆脱信息焦虑

联系邮箱 441953276@qq.com

相关标签

LLM 超级智能 AI研究 多智能体
相关文章