作者认为 AGI 或许会在未来 20 - 50 年内出现，而非接下来 5 年，但 Nathan 持不同观点，愿以 10:1 赔率打赌 AGI 会在接下来 5 年内被发明，并阐述了自己的理由。

🎯作者认为 AGI 需超越当前技术的科学进步，当前围绕聊天 LLMs 的热潮分散了顶尖 AI 科学家对理论进步的追求，减缓了基础科学进展。

💡Nathan 相信 LLMs 会发展到接近 AGI，成为非常有用工具的核心部分，这些工具将使人类 AI 科学家取得快速理论进展，AI 研究系统能实现假设的大规模并行测试，还能挖掘现有科学文献中的假设进行测试，从而快速发现能实现真正 AGI 的算法。

🧠Nathan 认为神经科学领域过去五年有关键突破尚未在机器学习中有效应用和大规模测试，且存在未充分测试的旧发现，这些可为 AI 研究提供丰富的可测试假设来源。

Published on September 5, 2024 4:01 PM GMT

I'm listening to Samo Burja talk on the Cognitive Revolution podcast with Nathan Labenz.Samo said that he would bet that AGI is coming perhaps in the next 20-50 years, but not in the next 5.

I will take that bet. I can't afford to make an impressively large bet because my counterfactual income is already tied up in a bet against the universe. I quit my well-paying industry job as a machine learning engineer / data scientist three years ago to focus on AI safety/alignment research. To make the bet interesting, I will therefore offer 10:1 odds.I bet $1000 USD against your $100 USD that AGI will be invented in the next 5 years. There are a lot of possible resolution criteria, but as a reasonable shelling point I'll accept this metaculus market: https://www.metaculus.com/questions/5121/date-of-artificial-general-intelligence/

I'll describe my rationale here, in case I change your mind and make you not want the bet. ;-)

I agree with your premise that AGI will require fundamental scientific advances beyond currently deployed tech like transformer LLMs.

I agree that scientific progress is hard, usually slow and erratic, fundamentally different from engineering or bringing a product to market.

I agree with your estimate that the current hype around chat LLMs, and focus on bringing better versions to market, is slowing fundamental scientific progress by distracting top AI scientists from pursuit of theoretical advances.

My cruxes are these:

I believe LLMs will scale to close enough to AGI to become central parts of very useful tools. I believe that these tools will enable the human AI scientists to make rapid theoretical progress. I expect that these AI research systems (I won't say researchers, since in this scenario they are still sub-AGI) will enable massively parallel testing of hypotheses which are derived as permutations of a handful of initial ideas given by the human scientists. I also foresee these AI research systems mining existing scientific literature for hypotheses to test. I believe the result of this technology will be rapid discovery of algorithms that can actually scale to true AGI.

I have been following advances in neuroscience relevant to brain-inspired AI for over 20 years now. I believe that the neuroscience community has made some key breakthroughs in the past five years which have yet to be effectively exported to machine learning and tested at scale. I also believe there's a backlog of older neuroscience findings that also haven't been fully tested. Thus, I believe the existing neuroscience literature provides a rich source of testable under-explored hypotheses. This could be tackled rapidly by the AI research systems from point 1, or will eventually be digested by eager young scientists looking for an academic ML paper to kickstart their careers. Thus the two cruxes are independent but potentially highly synergistic.

I look forward to your response!Regards, Nathan Helm-Burger

Discuss