少点错误 2小时前
The Evolution of Agency - A Research Agenda
index_new5.html
../../../zaker_core/zaker_tpl_static/wap/tpl_guoji1.html

 

本文探讨了“智能体”(agent)概念的多样性,以蚂蚁群体为例,阐述了经济学、生物学和认知科学等不同领域如何从各自的视角解读其涌现出的复杂行为。文章指出,智能体的定义和类型高度依赖于分析框架,并借鉴丹尼尔·丹尼特的“意向立场”理论,提出“智能体谱系”的概念,即不同学科的分析方法是为应对不同预测挑战而演化出的解决方案。这一框架对于理解和对齐日益复杂的AI系统至关重要,因为单一视角可能导致对齐问题的误判和潜在风险,而理解不同分析框架的演化关系,则有助于构建更全面、系统的AI对齐策略。

🧠 智能体概念的视角依赖性:以蚂蚁群体为例,经济学家视其为市场系统,生物学家视为超有机体,认知科学家则看作分布式信息处理系统。这表明,对同一复杂系统的“智能体”属性的解读,会因所采用的分析框架(如经济模型、生物学视角、认知科学理论)而产生显著差异。

💡 意向立场与分析框架的演化:借鉴丹尼尔·丹尼特的“意向立场”,文章认为将系统视为具有信念、欲望和目标的“智能体”是一种有用的建模策略。不同学科的分析方法,如经济学中的市场机制、生物学中的超有机体概念,并非任意选择,而是为应对特定预测挑战而演化出的有效解决方案,构成了“智能体谱系”。

🌐 AI对齐的挑战与“智能体谱系”的应用:在AI领域,尤其是多智能体系统(如AI公司),单一分析视角可能导致对齐问题的误判。例如,一个AI系统可能在经济学框架下完美合规,但在合作AI视角下却可能引发灾难性的协调失败。理解不同分析框架的演化关系,有助于系统性地识别和解决这些潜在风险。

🌳 借鉴生物学谱系分析:文章将学科分析框架的演化类比为生物学中的谱系分析。通过绘制不同分析方法的“演化历史”,可以揭示它们之间的数学“祖先”关系,判断哪些方法可以安全地结合,哪些可能相互冲突。这为理解和构建混合人机系统中的多尺度智能体提供了方法论支持。

Published on August 15, 2025 10:47 AM GMT

In Douglas Hofstadter's "Gödel, Escher, Bach," he explores how simple elements give rise to complex wholes that seem to possess entirely new properties. An ant colony provides the perfect real-world example of this phenomenon - a goal directed system without much central control. This system would be considered agentic under most definitions of agency, as it is goal directed, yet what type of agent is it? Can it be multiple agents at the same time? We’ll argue that this is the case and that if we map out the ways different fields describe ant colonies we’ll be better able to solve x-risk related problems.

Introduction

Watch a harvester ant colony during its morning foraging expedition. Thousands of individual ants stream out from the nest, each following simple rules: follow pheromone trails, drop chemicals when carrying food, avoid obstacles. Yet from these basic interactions, something remarkable emerges. The colony rapidly reorganizes when scouts discover food patches. Trails strengthen and fade based on success. Workers shift between tasks. The system somehow "decides" which resources are worth harvesting.

This is clearly goal-directed behavior. If you move the food, the colony adapts. If you block a path, it finds alternatives. The underlying infrastructure - the simple ant rules - remains the same, but the collective response changes to pursue the goal of efficient foraging. So under most definitions of agency we're looking at some kind of agent.

In fact, they’ve inspired a class of algorithms called “Ant Colony Optimisation” algorithms.

 

But what kind of agent?

Ask an economist, and they might tell you this is a sophisticated market system. Individual ants respond to price signals encoded in pheromone concentrations. Strong pheromone trails indicate valuable resources, attracting more workers. Weak trails signal poor opportunities, causing workers to abandon them. The colony exhibits supply and demand dynamics, efficient resource allocation, even investment strategies for uncertain opportunities. This is economic agency - rational actors coordinating through market mechanisms.

Ask a biologist, and you'll hear about a superorganism. The colony functions as a single adaptive entity with goal-directed behavior toward survival and reproduction. Individual ants are like specialized cells in a larger organism. The colony learns from experience, responds strategically to threats, and maintains homeostatic balance. This is biological agency - a living system pursuing survival goals through coordinated adaptation.

Ask a cognitive scientist, and they'll describe distributed information processing. The colony maintains memory through pheromone patterns, exhibits attention allocation through worker distribution, and demonstrates decision-making under uncertainty. It processes environmental information and generates appropriate responses through parallel computation. This is cognitive agency - a mind-like system processing information to achieve goals.

They're all looking at the same ants following the same chemical trails, but they're describing different types of agency. The economist sees market mechanisms. The biologist sees living organisms. The cognitive scientist sees information processing.

These aren't just different perspectives on the same phenomenon - each scientist would insist they're identifying the fundamental nature of what's actually happening. And each has compelling evidence for their view. The mathematical models from economics accurately predict foraging patterns. The biological framework explains adaptive responses to environmental changes. The cognitive approach captures the system's ability to solve complex optimization problems.

So what kind of agent is an ant colony, really?

The same puzzle appears everywhere we look for intelligence. Large language models seem like conversational partners to some users[1], statistical pattern-matching systems to computer scientists, and strategic actors to market analysts. Corporate decision-making systems that function as economic agents to some observers and biological adaptations to others. Financial markets that appear as collective intelligence or algorithmic chaos depending on your viewpoint.

Recently, researchers at Google DeepMind proposed an answer to this question, namely that: "Agency is Frame-Dependent." They argued that whether something counts as an agent - and what kind of agent it is - depends entirely on the analytical frame you use to examine it. The boundaries you draw, the variables you consider, the goals you attribute, the changes you count as meaningful - all these choices determine what type of agency you observe.

This is based on and similar to Daniel Dennett's "intentional stance." Dennett argued that attributing agency to a system isn't about discovering some inherent property - it's about choosing a useful modeling strategy. We adopt the intentional stance when treating something as having beliefs, desires, and goals helps us predict and understand its behavior better than purely mechanistic descriptions.

But we can take this a step further: if agency is fundamentally about choosing useful modeling strategies, then the different approaches we saw with the ant colony aren't arbitrary preferences. They're evolved solutions to different analytical challenges. The economist's market-based view evolved because it proved effective for predicting resource allocation patterns. The biologist's organism-based view emerged because it captured adaptive responses that other approaches missed. The cognitive scientist's information-processing view developed because it explained coordination mechanisms that simpler models couldn't handle.

This suggests a potential research program: studying the evolution of these different analytical approaches like a phylogeny - tracing how different fields developed distinct "intentional stances" based on the specific prediction challenges they faced. Just as Darwin's finches evolved different beak shapes for different feeding environments, scientific fields might have evolved different mathematical frameworks for different explanatory environments.

A Phylogeny of Agents

We call this the "phylogeny of agents" - mapping the evolutionary history of how different fields developed their approaches to recognizing and modeling agency.

In interdisciplinary conversations, researchers from different fields tend to get confused or disagree about "what is an agent?". Rather than treating this as a question to be solved with a specific answer, we treat it as data about how analytical tools evolve to match different explanatory challenges.

Here is a concrete example, that both explains the research agenda and explains why you should care: a completely automated AI firm. Its made up of a multi-agent system of AIs operating in the real world. It’s our ant colony but with AIs instead of ants. Just like our ant colony, the AI firm can be described in different ways. Depending on your stance it can be a market of ideas and goods, or a predictive processing system, or a utility maxing black box algorithm. These are all different types of intentional stances, or in our words, agents.

Here's the alignment problem: the same AI firm can be simultaneously aligned and misaligned depending on which type of agency you're ascribing it. If we approach alignment through only one analytical lens, we might solve the wrong problem entirely while creating new risks we can't even see. Aligning a distributed complex system is different from aligning a thermostat. Aligning a system with memory and online learning is different from aligning LLMs.

Let's look at a couple of different ways that alignment and agency could work from different intentional stances:

From an economic perspective, the AI firm might be perfectly compliant. It follows market regulations, responds to price signals, allocates resources efficiently, and maximizes shareholder value. The firm appears well-aligned with economic incentives and legal frameworks. Economists studying this system would conclude it's behaving as a rational economic agent should.

From a decision theory perspective, the firm might be executing its stated utility function flawlessly. Its sub-agents optimize for clearly defined objectives, exhibit goal-directed behavior toward specified targets, and demonstrate adaptive learning within their designed parameters. AI researchers examining the system would find textbook alignment between the AI's behavior and its programmed goals.

From a cooperative AI perspective, this same firm might be generating catastrophic coordination failures. Despite following its individual incentives perfectly, it could be contributing to race dynamics, undermining collective welfare, or creating systemic risks that no individual actor has incentive to address. Researchers studying multi-agent dynamics would see dangerous misalignment at the system level.

From a biological systems perspective, the firm might be optimized for short-term efficiency but catastrophically fragile to environmental shocks. Like a monoculture lacking genetic diversity, it could be heading toward collapse because it lacks the redundancy, adaptability, and resilience mechanisms that biological systems evolved for survival under uncertainty.

This is where the phylogeny of agents research program becomes useful. Darwin's phylogenetic tree revolutionized biology by revealing the evolutionary relationships between species. If we map the phylogeny of analytical approaches to agency, we could transform how we understand and align complex intelligent systems.

A potential phylogeny of intentional stances.

Consider what phylogenetic analysis has enabled across different fields. In biology, it revealed why certain trait combinations work together, predicted which species would adapt successfully to new environments, and explained how similar solutions evolved independently in different lineages. In medicine, phylogenetic analysis of pathogens enables vaccine development and predicts drug resistance patterns. In linguistics, it traces how languages branch and influence each other, revealing deep structural relationships between seemingly different communication systems.

What might a phylogeny of intentional stances reveal for AI alignment? Instead of treating each field's approach to agency as an isolated modeling choice, we could understand them as evolved solutions to specific analytical challenges - each carrying the "genetic code" of the optimization pressures that shaped them.

The phylogeny could reveal which analytical approaches share common mathematical "ancestors" - suggesting when they can be safely combined - and which represent fundamentally different evolutionary branches that may conflict when applied to the same system. Just as biological phylogeny predicts which species can hybridize successfully, an analytical phylogeny could predict which modeling approaches can be productively integrated.

For hybrid human-AI systems, this becomes crucial. These systems exhibit agency that doesn't fit cleanly into any single field's evolved framework. The phylogenetic approach asks: which combinations of analytical approaches, evolved under different pressures, can successfully characterize the multi-scale agency of cybernetic systems?

Rather than hoping different alignment approaches happen to be compatible, we could systematically understand their evolutionary relationships - when economic alignment strategies will enhance biological resilience, when cognitive frameworks will complement rather than conflict with systemic approaches, and when applying a single analytical lens will create dangerous blind spots.

Conclusion

The ant colony puzzle revealed that the same complex system can simultaneously be different types of agents depending on your analytical lens - what appears as confusion between fields actually represents evolved solutions to different analytical challenges. Different fields developed distinct "intentional stances" toward agency because they faced different prediction environments, and by mapping these evolutionary relationships like a phylogeny, we can transform interdisciplinary disagreement into systematic understanding. 

This matters for AI alignment because multi agent systems (such as AI-AI or hybrid human-AI systems) exhibit agency across multiple scales simultaneously - a single AI firm can be perfectly aligned from one perspective while catastrophically misaligned from another. 

The goal for this research is to help build the conceptual infrastructure to navigate conversations around agency and alignment. This helps map the right type of alignment agenda to the right frame of agency. It would help understand our own individual intentional stances towards agency depending on which field we come from, and it would also help show how different stances relate to each other through a shared ancestry.

In an ideal we would be able to get new approaches to alignment "for free" by mapping them over from different fields.

If this sounds interesting, we’re running a workshop and a research program on this area during autumn 2025 at Equilibria Network.

Finally some fun related links:

Cross posted to : Substack

  1. ^

    Some even overuse the intentional stance to the extent that they anthropomorphise LLM systems into "awakened beings"



Discuss

Fish AI Reader

Fish AI Reader

AI辅助创作,多种专业模板,深度分析,高质量内容生成。从观点提取到深度思考,FishAI为您提供全方位的创作支持。新版本引入自定义参数,让您的创作更加个性化和精准。

FishAI

FishAI

鱼阅,AI 时代的下一个智能信息助手,助你摆脱信息焦虑

联系邮箱 441953276@qq.com

相关标签

智能体 AI对齐 意向立场 多智能体系统 学科演化
相关文章