Published on June 5, 2025 8:13 AM GMT Applications for The Future of Life Foundation's Fellowship on AI for Human Reasoning are closing soon (June 9th!) From their website: Apply by June 9th | $25k–$50k stipend | 12 weeks, from July 14 - October 3
They've listed "Tools for wise decision making" as a possible area to work on.
Expand for more details.
Join us in working out how to build a future which robustly empowers humans and improves decision-making.
FLF’s incubator fellowship on AI for human reasoning will help talented researchers and builders start working on AI tools for coordination and epistemics. Participants will scope out and work on pilot projects in this area, with discussion and guidance from experts working in related fields. FLF will provide fellows with a $25k–$50k stipend, the opportunity to work in a shared office in the SF Bay Area or remotely, and other support.
In some cases we would be excited to provide support beyond the end of the fellowship period, or help you in launching a new organization.
This is a list of potentially useful projects. Some of these projects may be much higher impact than others and I haven’t thought deeply about the value of all of the projects, so use your own judgement. It’s just a list for listing possibly useful projects in order to spur more work in this space.
Field-building:
At this stage, I consider field-building work around Wise AI as especially high priority. My sense is that there’s starting to be some real energy around this area, however, it mostly isn’t being directed towards things that will increase our chance of avoiding catastrophic risks. Fields tend to be more flexible in their initial stages, but after a while, they tend to settle into an orthodoxy, so it's important to intervene when it's still possible to make a difference:
- Prioritisation research: Wisdom can mean a lot of different things; there are multiple types of wisdom. It seems quite important to try to identify priorities whilst the field is still fluid and easier to influence via this kind of work. One key way AI safety analysis will differ from that of people outside the field is that AI safety analysis is much more likely to take into account the externalities of enabling a capability, as opposed to naively ignoring this. Whilst this work is high priority, you may want to refrain unless you think it’s a particularly good fit for your skillset, given the difficulty of this work. It’s very easy to miss a crucial consideration that breaks your analysis and maybe even reverses the sign.Summarising important papers or other resources (example): Producing a good summary can take a lot of work, but this doesn’t seem to be the hardest skill to learn. There are two primary reasons why you might want to do this: a) increasing the impact of the original research b) building up the field by raising awareness and by making it easier to get oriented.Produce and verify some AI-generated reports related to artificial wisdom: AI-generated reports are often very good, the main downside is that you don’t know if they are reliableLiterature review of previous work on artificial wisdom: Whilst artificial reports can be very helpful, Deep Research still can’t match human judgement in terms of what is important.Summary of how different cultures and disciplines conceive of wisdom: Whilst less direct than reviewing work on artificial wisdom, this would still be extremely useful.Assist with the communication of these ideas: There is some quite valuable work to be done here. PM me if you’re interested in this, though I suspect quality of communication is more important than quantity.
Theoretical questions:
- To what extent is wisdom just the absence of unwisdom[1]? Even if being wise goes beyond the mere avoidance of being unwise, avoiding unwisdom might still be the lowest hanging fruitIn what ways is current AI wise? In what ways is it unwise? This would help identify gaps that could be filled.Insofar as current AIs are wise, what is it about their training that makes them wise?How might AI wisdom differ from human wisdom? How can we bridge this gap?How might neural networks represent wisdom internally? Is there likely to be a “wisdom direction”, and, if so, how would we find it? Maybe we can identify the various components of wisdom?What are the impacts of choosing a particular broad approach to learning wisdom? For example, imitation learning vs RL. What balance is optimal?In what circumstances might a wise advisor lead to a reckless or malicious actor becoming responsible or prosocial? This determines the extent to which wise AI would be safe to proliferate.How can we evaluate an AI system for wisdom[2]? Is this even possible? How accurately would we need to be able to measure wisdom in order to train an AI system to be wise? How does this differ from evaluating wisdom in humans?Is the idea of a “wisdom explosion” a coherent concept? Or perhaps it is conceivable, but feedback loops are too weak in this world to make it work.Is a super-intelligence system automatically a super-wise system? If so, for what definitions of intelligence and wisdomWhat biases or mistakes can cause someone to be smart rather than wise?[3]What level of wisdom do we need in order to navigate these challenges?What would the world look like if it were devoting serious attention to this?[4]What are the strengths and weaknesses of LLMs when it comes to philosophy?
Concrete work:
- Improving model specs: Read some model specs and make suggestions about how to improve the wisdom of the models. This is likely to be incremental, but can still help.User interface design: What is the best user interface for obtaining wise advice when the answer really matters? It seems unlikely that it would be just a simple chatbot interface.Journaling your attempts to make use of AI advice: The hope would be that the journaling would identify issues or considerations that would be easy to miss in Theory Land. For this reason, I see subjective research as massively underrated. It may not be as rigorous as empirical research, but it can provide a large number of bits per experiment and this is especially valuable in the early stages of a field.Attempting to train wise AI advisors and journaling your experience: As per the previous point, at least at this stage, I’d much rather have an informal journal someone kept whilst attempting to train wise AI advisors than a few extra bits of empirical info.Identify and experiment with various toy scenarios to figure out which ones would be most valuable for developing or testing wisdom: This work could be directly applied, but it would help clarify the strategy for improving wisdom as well.Social media advisor bot: Create a bot to advise you on your social media posts. Whether they’re posts that you’re likely to regret in the future, according to your own values, either because you’re being rude or because you’re strawmanning. There’s honestly a chance that this improves the quality of the social media discourse.Tools for reducing cognitive biases: There’s a decent chance that being wise is primarily about avoiding unwisdomDecision sanity-checker[5]: “If catastrophes often involve someone making egregiously bad decisions (knowingly or otherwise), might there be a role for LLMs to help catch these decisions before they are finalized? What information and infrastructure would be needed? - There will likely be decisions that such a tool could catch, where avoiding making a bad decision could be massively beneficial for the world.Crux identification: AI tool that determines the main reasons that people (or groups disagree)Start a group that aims to provide analyses on questions of strategic importance with the assistance of AI tools[6]: Might be closer to a bite-sized piece of work than trying to directly train AI to advise on decisions.
Less important:
- Improve the Artificial Wisdom Wikipedia article: This could be a good project for learning more about the area, but the impact of this is very uncertain, especially as AI models become increasingly important.Running a traditional empirical experiment on AI wisdom: I expect this will be valuable down the line. Right now, subjective research provides more bits, but once people start dividing into schools of thought with vast differences in terms of the underlying assumptions, we’ll start needing something more rigorous to differentiate between these theories. If you’re especially keen on traditional empirical research, I recommend focusing on other areas of AI safety for now.
- ^
Suggested by Richard Kroon
- ^
Suggested by Richard Kroon
- ^
Suggested by Richard Kroon
- ^
From the AI Impact Competition, see the ecosystems section of the suggested questions for more detail
- ^
- ^
Similar to a suggestion in a Forethought post, but focusing on wisdom rather than epistemics
Discuss
