本文探讨了研究能力与战略思维之间的关系，指出人们常常误将两者混为一谈，导致过度依赖研究人员的战略判断。文章强调，虽然良好的研究记录可以作为一定程度的证据，但并不能充分证明一个人具备出色的战略思维能力。战略思维需要不同的技能，尤其是在缺乏反馈的环境下，更难以验证其正确性。作者呼吁读者在评估战略观点时，应审慎考察提出者的实际战略思维能力，而非仅仅基于其研究成就。

🤔 研究能力与战略思维是不同的技能组合：文章的核心观点是，虽然研究能力和战略思维都重要，但它们是相对独立的技能。擅长研究并不等同于擅长战略思考，人们常常错误地将两者混为一谈。

💡 战略思维的挑战：战略思维的难点在于缺乏及时反馈。在复杂的长期因素分析中，需要基于过去的趋势和类比进行推断，难以判断其正确性。而研究领域通常有更多的反馈机制，这使得擅长研究的人可能在战略思考方面表现不佳。

📚 良好战略思维的关键因素：文章强调，清晰的思考能力、领域知识和投入的时间是良好战略思维的关键因素。这包括对复杂问题的清晰理解，对研究领域的足够了解，以及花时间思考相关问题。

🌍 战略思维需要多元的专业知识：除了研究领域的知识，战略思维还需要对未来AI的能力和心理、经济和政治环境、解决方案的实施、以及各种风险因素有深入的了解，而这些往往是研究人员所缺乏的。

Published on March 22, 2025 10:13 AM GMT

TL;DR Having a good research track record is some evidence of good big-picture takes, but it's weak evidence. Strategic thinking is hard, and requires different skills. But people often conflate these skills, leading to excessive deference to researchers in the field, without evidence that that person is good at strategic thinking specifically. I certainly try to have good strategic takes, but it's hard, and you shouldn't assume I succeed!

Introduction

I often find myself giving talks or Q&As about mechanistic interpretability research. But inevitably, I'll get questions about the big picture: "What's the theory of change for interpretability?", "Is this really going to help with alignment?", "Does any of this matter if we can’t ensure all labs take alignment seriously?". And I think people take my answers to these way too seriously.

These are great questions, and I'm happy to try answering them. But I've noticed a bit of a pathology: people seem to assume that because I'm (hopefully!) good at the research, I'm automatically well-qualified to answer these broader strategic questions. I think this is a mistake, a form of undue deference that is both incorrect and unhelpful. I certainly try to have good strategic takes, and I think this makes me better at my job, but this is far from sufficient. Being good at research and being good at high level strategic thinking are just fairly different skillsets!

But isn’t someone being good at research strong evidence they’re also good at strategic thinking? I personally think it’s moderate evidence, but far from sufficient. One key factor is that a very hard part of strategic thinking is the lack of feedback. Your reasoning about confusing long-term factors need to extrapolate from past trends and make analogies from things you do understand better, and it can be quite hard to tell if what you're saying is complete bullshit or not. In an empirical science like mechanistic interpretability, however, you can get a lot more feedback. I think there's a certain kind of researcher who thrives in environments where they can get lots of feedback, but has much worse performance in domains without, where they e.g. form bad takes about the strategic picture and just never correct them because there's never enough evidence to convince them otherwise. It's just a much harder and rarer skill set to be good at something in the absence of good feedback.

Having good strategic takes is hard, especially in a field as complex and uncertain as AGI Safety. It requires clear thinking about deeply conceptual issues, in a space where there are many confident yet contradictory takes, and a lot of superficially compelling yet simplistic models. So what does it take?

Factors of Good Strategic Takes

As discussed above, ability to think clearly about thorny issues is crucial, and is a rare skill that is only somewhat used in empirical research. Lots of research projects I do feel more like plucking the low hanging fruit. I do think someone doing ground-breaking research is better evidence here, like Chris Olah’s original circuits work, especially if done multiple times (once could just be luck!). Though even then, it's evidence of the ability to correctly pursue ambitious research goals, but not necessarily to identify which ones will actually matter come AGI.

Domain knowledge of the research area is important. However, the key thing is not necessarily deep technical knowledge, but rather enough competence to tell when you're saying something deeply confused. Or at the very least, enough ready access to experts that you can calibrate yourself. You also need some sense of what the technique is likely to eventually be capable of and what limitations it will face.

But you don't necessarily need deep knowledge of all the recent papers so you can combine all the latest tricks. Being good at writing inference code efficiently or iterating quickly in a Colab notebook—these skills are crucial to research but just aren't that relevant to strategic thinking, except insofar as they potentially build intuitions.

Time spent thinking about the issue definitely helps, and correlates with research experience. Having my day job be hanging out with other people who think about the AGI safety problem is super useful. Though note that people's opinions are often substantially reflections of the people they speak to most, rather than what’s actually true.

It’s also useful to just know what people in the field believe, so I can present an aggregate view - this is something where deferring to experienced researchers makes sense.

I think there's also diverse domain expertise that's needed for good strategic takes that isn't needed for good research takes, and most researchers (including me) haven't been selected for having, e.g.:

A good understanding of what the capabilities and psychology of future AI will look likeEconomic and political situations likely to surround AI development - e.g. will there be a Manhattan project for AGI?What kind of solutions are likely to be implemented by labs and governments – e.g. how much willingness will there be to pay an alignment tax?The economic situation determining which labs are likely to get there firstWhether it's sensible to reason about AGI in terms of who gets there first, or as a staggered multi-polar thing where there's no singular "this person has reached AGI and it's all over" momentThe comparative likelihood for x-risk to come from loss of control, misuse, accidents, structural risks, all of the above, something we’re totally missing, etc.And many, many more

Conclusion

Having good strategic takes is important, and I think that researchers, especially those in research leadership positions, should spend a fair amount of time trying to cultivate them, and I’m trying to do this myself. But regardless of the amount of effort, there is a certain amount of skill required to be good at this, and people vary a lot in this skill.

Going forwards, if you hear someone's take about the strategic picture, please ask yourself, "What evidence do I have that this person is actually good at the skill of strategic takes?" And don't just equivocate this with them having written some impressive papers!

Practically, I recommend just trying to learn about lots of people's views, aim for deep and nuanced understanding of them (to the point that you can argue them coherently to someone else), and trying to reach some kind of overall aggregated perspective. Trying to form your own views can also be valuable, though I think also somewhat overrated.

Thanks to Jemima Jones for poking me to take agency and write a blog post for the first time in forever.

Discuss