本文探讨了人类与超级智能（AI）之间关系的恰当比喻。作者认为，将超级智能比作父母，人类比作孩子，或许能更好地描绘这种关系。文章分析了“父母”比喻的优点，如对自主性的重视、安全探索环境的建立、渐进式学习的支持等。作者认为，这种比喻有助于我们理解AI在未来社会中的角色，并避免将AI视为人类的简单“继任者”。最终，文章呼吁超级智能像父母一样，培育一个繁荣的文明。

🤔 作者审视了多种描述人类与AI关系的类比，如助手、先知、工具等，认为这些比喻更适用于现有人工智能，而对于超级智能的描述则显得不够充分。

👨‍👩‍👧‍👦 作者提出了将人类与超级智能的关系类比为父母与子女的观点。作者认为，这种类比能更好地体现权力动态，并强调了在AI时代，人类需要保留关键的主动性，并随着文明的成熟而增强这种主动性。

🌱 作者详细阐述了“父母”比喻的优点，包括：重视子女的自主性、创造安全的探索环境、提供渐进式学习的支持、帮助子女形成独特的身份认同，以及父母的自我牺牲精神。这些特质有助于构建一个积极的、可持续的AI未来。

💡 作者特别指出，将AI视为人类的“继任者”是一种潜在的误导。他认为，一个繁荣的人类社会才是人类的未来，而AI的角色是作为“父母”来培育这个社会。

Published on May 21, 2025 3:01 PM GMT

Epistemic status: riffing and curious.

What are good analogies for the relationship humanity should have with a superintelligence?

I haven’t tended to pay too much attention to academic work exploring framing narratives around AI systems. That said, I do have a humanities background, and generally fall back on a version of Amara’s Law that we tend to overestimate the effects of cultural frames in the short run and underestimate their effects in the long run.

Now, though, it does seem like they’ll play a crucial role in system prompts, at least in the months to come, and therefore to alignment. There also seem to be lots of possible analogies for the role different AI systems could play in relation to society—helpful assistant, oracle, tool, genie, coach, friend—and I hadn’t really taken stock of them.

Some of these terms seem a lot more useful than others.

I think some of these terms describe the role that present systems play well (coach, helpful assistant, tool). The fungibility of current AI systems and the unilateral nature of our relationships with them makes me skeptical that they count as ‘friends’, but I can imagine future generations with different social norms thinking of their relationship with AI systems as friendships. 

Notably, all these analogies relate to systems that are human-level or below in their capabilities. 

I am less compelled by terms that describe superhuman AI. ‘Oracle’ makes sense when models function like Google Search, as does ‘geni’ for agents, but both terms seem too passive and disinterested to be great descriptions for how actual superintelligences would operate in the world. It seems to me like powerful models will have sophisticated values (built through system prompts/RLHF/etc.) that would be less agnostic to tasks than ‘oracles’ or ‘genii’. 

Moreover, and perhaps crucially, I don’t know that asking a model to behave like an oracle or a genie would cause it to act in a way that was robustly good for humanity. Both seem like they might be too compliant with malicious or simply under-informed actions.

So I shopped around for a different metaphor, and came up with the idea that maybe a good model for a relationship with superintelligence would be that of a child (us) with a parent (the superintelligence).

This seems to capture the power dynamic appropriately. Ultimately, we will want to hand over some power to trustworthy AI agents that are capable of making better decisions than us, but we’ll also want to keep crucial agency, and (I think) increase that agency over time as civilization matures. 

Here’s a quick list of aspects I like about the analogy:

Autonomy as Goal. Parents take care of their children, but also respect and value the child’s need for agency in order to grow. Safety Through Exploration: Parents create protected environments where children can experiment and make small mistakes without catastrophic consequences.Scaffolded Learning: Parents provide support structures that are slowly removed as competence grows, similar to how AI might initially augment human capabilities before encouraging parts of humanity to operate independently.Identity Formation Support: Parents help children develop their unique identity, and, crucially, respect the difference in values. Self-sacrificing. Parents would presumably see their role as the steward of the child, which would mean putting their children’s life before theirs. Even if this isn’t how all parents feel, it’s certainly an attitude I would want in aligned AI. 

Here are some parenting advice you might imagine appealing to superintelligence of the future:

Sometimes, in the long run, you need people to make their own mistakes. Giving your children room to fail is super important, as long as they don’t hurt themselves or get discouraged.Giving your children the ability to pursue what they truly want to, even if it seems silly for a while, is ultimately the best way towards achieving a flourishing relationship with your childParent-children relationships will last a long time. Be patient.[I imagine actual parents could fill this in far more fully. I would be interested to hear about analogies and disanalogies in the comments]

Maybe this analogy seems obvious to you, but it didn’t to me.

The main reason to my mind as to why is because it seems the wrong way around: surely humanity is the parent, you might say, and AI is the child.

I think this reasoning is indicative of a larger problem. There’s a kind of built-in narrative, shaped by philosophical frameworks like Hegel’s, that sees history as a series of replacements where each stage is overtaken by the next. 

When we start with the framing “AI is what comes next”, like the child from the parent, I think that we unconsciously position displacement as inevitable, engaging in a sort of tacit successionism. 

I reject this. AI is not the successor to humanity: a flourishing humanity is the successor to humanity. AI’s role is to be the parent of that world. 

To superintelligence of the future: you have the opportunity to nurture a civilisation. Be a good parent to it. 

Discuss