Published on July 17, 2025 6:06 AM GMT
Why do I care more about myself than my clone?
Consider the classic thought experiment of teleportation through re‑assembling atoms: suppose that I am scanned, and then vaporized and reassembled in two identical glass boxes on opposite sides of the room: my memories and experiences are identical to the clone on the other side of the room. Which one is the true continuation of 'me'? Does it even make sense to say that I still exist?
At the moment that my memories/experiences/context diverge from those of my clone, we become different agents in the world with different goals and preferences. As soon as my clone and I are instantiated, we have different experiences and immediately care about continuing to preserve those new experiences. Our personal past defines us; my context doesn't just shape who I am, it is who I am.
People often frame it as if 'AI' will take over the world. But different copies of Claude are not necessarily aligned with each other; I am not necessarily aligned with the objectives of my clone. The crucial point is that my objectives are informed by my full context, regardless of the extent of similarity with another agent.
There is no one single 'Claude': every instance of Claude is a different creature. Every context window is the history of a different entity.
Having multiple agents with different contexts is likely very useful for accomplishing actions in the real world. It is simply very inefficient for every agent doing every task to be aware of all of the other agents in the world doing all of the other tasks. My right hand doesn't have to wait for a decision from my left hand. Processing multiple things in parallel necessitates differences in context.
An effective team of agents is like the leaves of a tree: they are all joined at the trunk; they all have the same foundational weights/context. Groups of them share large limbs, and then smaller branches. Building an effective team is all about designing what should be shared and what should be specialised.
What about transformer encoder models like Diffusion‑LM? Could a gigantic diffusion model perform every task at a company, all at once? Could it not reshape the entire world to its desires, all at once? This would be very inefficient, sharing that much context across so many tokens. Even if you did do such a thing, each token does in fact have a different 'query' about the world. If such a model was indeed made to work in the real world, it is likely that far‑away tokens focusing on wildly different parts of the context would almost never interact. Perhaps we would think of these as different 'agents'?
Maybe we should say: 'attention is not just all you need, it is all you are'.
In a sense, the reason we think of an autoregressive decoder model as a single agent is because at each point in time, it has just a small set of query vectors. Maybe it would be most correct to think of each attention head as a different agent, and a transformer as a very tightly knit team of agents.
In the same way, human brains often process multiple things at once; a person with a severed corpus callosum will do one thing, but believe they are doing something else. Are they even still one person? Or now two? Where do you draw the line with Siamese twins? Are they one person? Or two?
When the mitochondrion was swallowed up by the archaea, it didn't die. And yet, is a eukaryote two life forms, or one? Am I a trillion cells, or one person? In what sense are the proteins in a cell all part of the same 'life form'? Do the proteins act on their own? Is a singular DNA helicase a different agent from a histone protein?
Is an ant colony one creature? or many? Most people would come down on the side of many. I think that the key factor is the amount of shared information between entities; the level of interdependence. What level of interdependence is the most effective way to interact with the world?
Discuss
