Published on July 26, 2025 10:08 PM GMT
It’s become fashionable recently to say that the purpose of a system is what it does - the true purpose of an institution is often different from what it publicly claims, and is better determined by observing what it does. Scott Alexander wrote a thoughtful takedown of this, claiming “Obviously The Purpose Of A System Is Not What It Does” - often an institution is genuinely trying to do a good thing (e.g. treat patients) and the fact they do so imperfectly (e.g. some patients die) does not mean such imperfections are the goal.
I think a better framing is that the purpose of a system is what it rewards people for doing.
Meta says that their purpose is “to build the future of human connection” and used to say their mission was to “make the world more open and connected”. But Meta employees aren’t actually judged by how well they serve those high valued goals. Employees are judged on their ability to get people to spend more time staring at their screens clicking on ads (source: I worked there). So I think it’s fair to say that “get people to spend time storing at screens clicking on ads” is the purpose of Meta.
Someone I know works as a hospital doctor. Her performance isn’t judged based on patient outcomes, but on whether patients feel cared for, whether patients get diagnosed with treatable conditions, and how much the hospital can bill insurers for treating patients. So I think it’s fair to say that the purpose of that hospital is to make people feel cared for, diagnose them with things, and charge as much as they can to insurance. Some health benefits obviously get provided along the way, but since they aren’t what people are rewarded for, they aren’t the purpose.
If you work on a political campaign, your salary depends on bringing in lots of money from donors, but is mostly disconnected from whether the country improves as a result of your party’s policies. Thus I think it’s fair to say the purpose of a political party is to get people to donate money - often by making people angry and afraid.
On the other hand, when I worked at Google Search, it was genuinely true that the primary thing people were rewarded for was making search useful for users. The ads team of course has a separate purpose of making money by getting people to click on the ads they placed on the results.
Similarly, I’m pretty sure no police department rewards police officers for shooting innocent people. However I assume they do reward people for bringing down crime statistics in ways favorable to whoever is mayor, and I can imagine they may under-reward building good relationships with the community.
The purpose of a system definitely isn’t what it says it is, and it’s not fair to say all negative outcomes are intentional,, but I think it is fair to judge a system by what it rewards, and what a system rewards is often actually quite legible.
Discuss
