本文探讨了“目的论”的概念，即探究事物“目的”或“功能”的哲学领域。作者提出了一种基于“认知-收敛-佐证”的分析方法，通过一系列问题，逐步解析“目的论”的本质。文章以“心脏的目的是泵血”和“铅笔的功能是写作”为例，分析了人们如何将“目的”和“功能”归因于事物，并探讨了这些归因背后的逻辑和模式。

🤔 **认知阶段：** 文章首先从主观角度出发，探讨了人们如何将“目的”或“功能”归因于事物。作者认为，当我们赋予事物“目的”或“功能”时，实际上是在将其视为经过优化以实现特定目标的产物。这一观点与“LessWrong”中关于“优化”的概念相一致。 例如，当我们认为“心脏的目的是泵血”时，我们实际上是在将其视为经过自然选择优化后的产物，其结构和功能都适应了泵血这一目标。这一认知模式并非凭空产生，而是源于我们对事物的观察和分析，以及我们对“优化”这一概念的理解。 在这个阶段，作者主要关注的是我们对“目的论”的认知过程，而非对“目的论”本身的本质进行直接定义。

🤝 **收敛阶段：** 文章接着指出，不同的人对于事物的“目的”或“功能”的认知往往具有高度的一致性。例如，即使人们对生物进化的机制并不了解，他们也能普遍认同生物体是被“优化”过的。这一现象表明，在我们的认知过程中，存在着一些共同的模式，引导着我们对事物进行类似的归因。 作者认为，这种“收敛”现象背后的原因在于，我们都能够识别出环境中的一些特定模式，并将这些模式与“优化”的概念联系起来。例如，我们能够观察到生物体具有高度的复杂性和适应性，而这些特征恰好是“优化”的结果。 因此，文章提出了一个关键问题：什么样的模式能够被不同的人所识别，并被他们认为是“优化”的标志？作者给出了一个可能的答案：一个系统包含许多处于非典型状态的组件，但这些组件在某种程度上“方向一致”。换句话说，这些组件似乎都朝着同一个方向偏离了统计上的典型状态。

🧪 **佐证阶段：** 文章最后强调，我们需要对“优化”的模式进行数学上的定义和证明，并验证其是否能够解释我们对“目的论”的直觉。 作者认为，如果我们能够证明“方向一致的非典型组件”这一模式确实存在于被“优化”过的系统中，那么我们就能够将这一模式与“目的论”的概念联系起来。 这一阶段的关键在于，我们需要通过数学和科学的方法来验证我们的认知模式，并确保它能够准确地描述现实世界。同时，我们也需要检验我们的直觉是否与我们的认知模式相一致，并根据需要对我们的直觉进行修正。 总结而言，文章提出的“认知-收敛-佐证”方法，为探寻“目的论”的本质提供了一种新的思路。通过这一方法，我们可以更加深入地理解“目的论”的认知基础、收敛模式和数学定义，并最终找到“目的论”的“真名”。

Published on July 3, 2024 1:06 AM GMT

Opening Example: Teleology

When people say “the heart’s purpose is to pump blood” or “a pencil’s function is to write”, what does that mean physically? What are “purpose” or “function”, not merely in intuitive terms, but in terms of math and physics? That’s the core question of what philosophers call teleology - the study of “telos”, i.e. purpose or function or goal.

This post is about a particular way of approaching conceptual/philosophical questions, especially for finding “True Names” - i.e. mathematical operationalizations of concepts which are sufficiently robust to hold up under optimization pressure. We’re going to apply the method to teleology as an example. We’ll outline the general approach in abstract later; for now, try to pay attention to the sequence of questions we ask in the context of teleology.

Cognition

We start from the subjective view: set aside (temporarily) the question of what “purpose” or “function” mean physically. Instead, first ask what it means for me to view a heart as “having the purpose of pumping blood”, or ascribe the “function of writing” to a pencil. What does it mean to model things as having purpose or function?

Proposed answer: when I ascribe purpose or function to something, I model it as having been optimized (in the sense usually used on LessWrong) to do something. That’s basically the standard answer among philosophers, modulo expressing the idea in terms of the LessWrong notion of optimization.

(From there, philosophers typically ask about “original teleology” - i.e. a hammer has been optimized by a human, and the human has itself been optimized by evolution, but where does that chain ground out? What optimization process was not itself produced by another optimization process? And then the obvious answer is “evolution”, and philosophers debate whether all teleology grounds out in evolution-like phenomena. But we’re going to go in a different direction, and ask entirely different questions.)

Convergence

Next: I notice that there’s an awful lot of convergence in what things different people model as having been optimized, and what different people model things as having been optimized for. Notably, this convergence occurs even when people don’t actually know about the optimization process - for instance, humans correctly guessed millenia ago that living organisms had been heavily optimized somehow, even though those humans were totally wrong about what process optimized all those organisms; they thought it was some human-like-but-more-capable designer, and only later figured out evolution.

Why the convergence?

Our everyday experience implies that there is some property of e.g. a heron such that many different people can look at the heron, convergently realize that the heron has been optimized for something, and even converge to some degree on which things the heron (or the parts of the heron) have been optimized for - for instance, that the heron’s heart has been optimized to pump blood. (Not necessarily perfect convergence, not necessarily everyone, but any convergence beyond random chance is a surprise to be explained if we’re starting from a subjective account.) Crucially, it’s a property of the heron, and maybe of the heron’s immediate surroundings, not of the heron’s whole ancestral environment - because people can convergently figure out that the heron has been optimized just by observing the heron in its usual habitat.

So now we arrive at the second big question: what are the patterns out in the world which different people convergently recognize as hallmarks of having-been-optimized? What is it about herons, for instance, which makes it clear that they’ve been optimized, even before we know all the details of the optimization process?

Candidate answer (underspecified and not high confidence, but it will serve for an example): the system has lots of parts which are all in unusual/improbable states, but all “in a consistent direction” in some sense. So it looks like all the parts were pushed away from what’s statistically typical, in “the same way”.^[1]

Ideally, we could operationalize that intuitive answer in a way which would make convergence provable; it has the right flavor for a natural latent style convergence argument.

Corroboration

Imagine, now, that we have a full mathematical operationalization of “parts which are all in unusual/improbable states, but all ‘in a consistent direction’”. Imagine also that we are able to prove convergence. What else would we want from this operationalization of teleology?

Well, I look at a heron, I notice that it has a bunch of parts which are all in unusual/improbable states, but all ‘in a consistent direction’ - i.e. all its parts are in whatever unusual configurations they need to be in for the heron to survive; random configurations would not do that. I conclude that the heron has been optimized. Insofar as my intuition picks up on “parts which are all in unusual/improbable states, but all ‘in a consistent direction’” and interprets that pattern as a hallmark of optimization, and my intuition is correct… then it should be a derivable/provable fact about the external world that “parts which are all in unusual/improbable states, but all ‘in a consistent direction’” occur approximately if-and-only-if a system has been optimized.

More generally: insofar as we have some intuitions about how teleology works, we should be able to prove that our operationalization/characterization indeed works that way. (Or, insofar as the operationalization doesn’t work the way we intuitively expect, we should be able to propagate the counterexamples back to our intuitions and conclude that our intuitions were wrong or required additional assumptions, as opposed to the operationalization being wrong.)

Cognition -> Convergence -> Corroboration

Let’s go back over the teleology example, with an emphasis on what questions we’re asking and why.

We start with questions about my cognition:

…first ask what it means for me to view a heart as “having the purpose of pumping blood”, or ascribe the “function of writing” to a pencil. What does it mean to model things as having purpose or function?

Two things to emphasize: first, these are questions about my cognition (or, more generally, one person’s cognition); the answers may or may not generalize to other people. Second, they are questions about my cognition; they’re not asking about how the external world “actually is” (at least not directly).

Some nice things about starting from questions about my cognition:

I can get lots of relevant data by introspecting.I can get lots of relevant data from my background models of cognition, and leverage abstract models of cognition (like e.g. Bayesianism or relaxation-based search) to formulate my understanding mathematically.

The downside is that introspection is notoriously biased and error-prone, and this is all not-very-legible and hard to test/prove. That’s fine for now; (some) legible falsifiability will enter in the next steps.

From cognition, we move on to questions about convergence: 

Next: I notice that there’s an awful lot of convergence in what things different people model as having been optimized, and what different people model things as having been optimized for. [...]

Why the convergence?

The standard answer of interest, which generalizes well beyond teleology, is: people pick up on the same patterns in the environment, and convergently model/interpret them in similar ways. Then the generalizable question is: what are those patterns? Or, in the context of teleology:

… what are the patterns out in the world which different people convergently recognize as hallmarks of having-been-optimized? What is it about herons, for instance, which makes it clear that they’ve been optimized, even before we know all the details of the optimization process?

At this point, we start to have space for falsifiable predictions and/or mathematical proof: if we have a candidate pattern, then we should be able to demonstrate/prove that it is, in fact, convergently recognized (in some reasonable sense, under some reasonable conditions) by many minds. Such a proof is where a natural latent style argument would typically come in (though of course there may be other ways to establish convergence).

Once convergence is established, we know that we’ve characterized some convergently-recognized pattern. The last step is that it’s the convergently-recognized pattern we’re looking for. For instance, maybe dogs are a convergently-recognized pattern in our environment, and having-been-optimized is also a convergently-recognized pattern in our environment. If we’ve established that “parts which are all in unusual/improbable states, but all ‘in a consistent direction’” is a convergently-recognized pattern in our environment, how do we argue that that pattern is the-thing-humans-call-“teleology”, as opposed to the-thing-humans-call-“dogs”?

Well, we show that the pattern has some of the other properties we expect of teleology.

More generally, this is the corroboration step. We want to prove/demonstrate some further consequences of the pattern identified in the previous step (including how it interfaces with other patterns we think we’ve identified), in order to make sure it’s the pattern we intended to find, as opposed to some other convergently-recognized pattern. This is where all your standard math (and maybe science) would come in.

Cognition -> Convergence -> Corroboration. That’s the pipeline.

Examples are Confusing, Let’s Make it Really Abstract!

The Cognition -> Convergence -> Corroboration Algorithm:

Cognition: Guess at a Cognitive Operationalization. 
Cognitive Model Selection: Choose a framework to model your own mind. Bayesianism is one often-fruitful place to start.Cognitive Operationalization: Within that framework, operationalize the intuition itself. E.g. within my bayesian world model, what is a “dog.” Within my bayesian world model, what is “purpose” or what is going on in my bayesian world model when I ascribe “purpose” to some part of it? 
Convergence: Guess at a Pattern + Prove Pattern is Convergent.
Operationalization: (Somehow) use the Cognitive Operationalization to intuit an Environmental Operationalization of the concept in terms of the external world, possibly using methods which include investigating instances which many people point to and agree is “the thing.” If you fail, return to step 1.Convergability: Check that your operationalization is in fact a candidate member of “patterns which minds tend to convergently recognize” (which we usually operationalize using natural latents). If you fail, return to step 2.a
Corroboration: Derive further properties about the candidate external pattern (operationalization) and check if those further properties are consistent with the original intuitive concept. If they aren’t, return to step 2.

Upon failure sending you back to step 1, three things could be wrong. Use magic to figure out which it is:

Cognitive Model is fine, Intuition is fine, Cognitive Operationalization needs to update. -> Update the Cognitive Operationalization and return to step 2.Cognitive Model is fine, Intuition needs to update. -> Update the intuition and return to step 1.b.Intuition is fine, Cognitive Model needs to update. -> Update the cognitive model and return to step 1.b.

Also, obviously, if you’re caught in a loop (like, e.g., failing step 3 and going back to step 2.a over and over, jump back a bit further, e.g. step 1.)

When is the Cognition -> Convergence -> Corroboration Pipeline Useful?

The central use case is:

There’s some concept…which different humans can successfully communicate about well enough that basically-the-same concept seems to show up in their different heads (as evidenced by e.g. systematically pointing at stuff and calling it the same thing)…and we want a robust mathematical operationalization of that concept (i.e. a “True Name”).

Most topics studied in philosophy are in-scope. Most (but importantly not all) “deconfusion” work is in-scope.

Beyond just a useful process to follow for such use-cases, we’ve also found the Cognition -> Convergence -> Corroboration structure useful for organizing thoughts/arguments: it’s useful to explicitly distinguish a cognitive characterization from a convergent pattern characterization from a consequence. For instance, we’ve often found it useful to explain some problem we’re thinking about as “What are the patterns/structures in the world which people convergently recognize as X?”.

Some use-cases for which this pipeline is probably not the right tool:

Operationalizing some concept which humans are not able to communicate about at all. (You know which I’m talking about. (He’s lying, you don’t.))Doing math or engineering with concepts which have already been operationalized.Punditry, publishing ML papers, and other use cases for which being robustly correct or understanding what one is doing is not a particularly central objective.

If you want to see more examples where we apply this methodology, check out the Tools post, the recent Corrigibility post, and (less explicitly) the Interoperable Semantics post.

1. ^{^}

   If “unusual/improbable” still sounds too subjective, then you can think of operationalizing it in the Solomonoff/Kolmogorov sense, i.e. in terms of compressibility using a simple Turing machine.

Discuss