Published on March 20, 2025 4:03 AM GMT
Prerequisites: Graceful Degradation. Summary of that: Some skills require the entire skill to be correctly used together, and do not degrade well. Other skills still work if you only remember pieces of it, and do degrade well.
Summary of this: The property of graceful degradation is especially relevant for skills which allow groups of people to coordinate with each other. Some things only work if everyone does it, other things work as long as at least one person does it.
I.
Examples:
- American Sign Language is pretty great. It lets me talk while underwater or in a loud concert where I need earplugs. If nobody else knows ASL though, it's pretty useless for communication.Calling out a play in football is pretty great. It lets the team maneuver the ball to where it needs to go. If nobody else on the team studied the plays though, then the quarterback can call all the numbers he wants but nothing useful is going to happen.Encrypting my email is pretty great. It lets me send messages to other people without the message getting intercepted. If the person I'm talking to ever leaves an unencrypted copy lying around or forwards the plaintext though, all that caution fails.
The important detail of these examples I'm trying to point at is that it doesn't matter if you're amazing at the skill if other people aren't. No amount of individual practice will make ASL a fluid tool for being understood in a loud concert or in spite of being hard of hearing. It's fundamentally a stag hunt, where it's only worth doing if other people are going to do it to.
What are some counterexamples?
- Learning to read and write is pretty great. I can write notes to myself and by doing so extend my memory. I can use writing an essay to refine my thoughts on a topic. If other people know how to read and write that's great since we can write to each other, but even if I'm the only literate person in the world literacy is still useful.Good data backups are pretty great. I can get into a habit of using extra hard drives or off-site backups to make sure I don't lose as much of my writing or family vacation photos if something bad happens to my computer. If other people also take good backups, that's nice because it means they might have backups I didn't think to make (the internet archive is good!) but even if I'm the only person in the world who makes backups this is still useful to me.Tracking and distinguishing inferences and observations is pretty great. In my own head, I get better at noticing what's true vs what's likely. I get fooled by "what it looks superficially like" less often. If other people also make a habit of distinguishing this, we can collectively have better discussion about the world, but it's still to my advantage to maintain this distinction for myself no matter what everyone else is doing.
ASL, football plays, and encrypting your email do not socially degrade gracefully. Literacy, data backups, and inference vs observation do.
II.
Sometimes it's good to build a small test into the skill.
Climbing calls are pretty great. That's the thing rock climbers do when checking if the ropes are attached correctly. "Ready to climb." "Belay on." "Climbing." "Climb on." It's a quick little sequence to cover minor things like "hey, if I fall thirty feet, is this neat rope gizmo actually going to catch me?"
A useful feature of climbing calls is that if your partner does know them, your partner probably also knows what to check and if they don't then this will be audible and obvious. If you call "Ready to climb" and your partner says "sure, sounds good" then this is a good sign that they don't know how to belay. It's not definitive proof — they could be really good at it and just happened not to use the right response — but it's enough that I would stop and go over things with them. In other words, the call and response of climbing calls is a kind of check sum or warning indicator.
This generalizes. If there's some mechanism or tool that you want to use to work well with other people, which doesn't work well if only some people are doing it, having a handshake protocol is a good start. This kind of thing doesn't help if there's a reason someone might try and fake the signal, but there's not a lot of incentive to just learn the call and response part of rock climbing. That's not what gets you invited to the cool rock climbers club.
Other times, it's good to carve out a space where everyone does actually know the skill.
I used to work on a moderately complicated piece of software. Not just the programming, I mean the user interface had lots of submenus and fiddly bits. Thing is, we as a company offered a training course where we spent a week or two teaching whole customer teams how to use the software. Week long training periods make up for many an unintuitive UI.
The first thing that you learn in most martial arts dojos is how to fall. This isn't because falling is crucially important in a fight (though it does help.) It's because every other lesson the dojo teaches can now assume that obviously the students know how to fall. An instructor can safely throw a student to demonstrate an advanced move, knowing that they would roll with it. Koshi Nage and Seoi Nage don't degrade particularly gracefully (I can't think of any throw that does) but in dojo situations they really don't socially degrade gracefully.
For an obvious example, take driving. There's the simple mechanics of turning the car or getting it into gear, the kind you'd need if you were the only person in the world after the zombie apocalypse. There's also the whole social expectations and conventions; red light means stop, at a four way stop sign we go one at a time clockwise around the intersection, we drive on the right hand side of the road[1]. Having a correct side of the road to drive on only works if you can get everyone in the country on the same page. Because this is important, we make everyone in the country take a test before they're allowed to drive.
III.
This concept is important for conflict resolution.
- Voting is pretty great. It lets my country figure out who is going to lead it without fighting a bloody civil war. But it only works if people basically agree that voting is how we decide the leader. If I got suddenly teleported to the court of Genghis Khan and proposed we vote on who's in charge, this obviously doesn't work.Lawsuits? I'm broadly a fan of the concept as compared to its alternatives. It lets us decide whether Fuzzy Kittens Incorporated needs to fix the environmental damage it caused or whether its hiring methods are racist without a mob with fire and pitchforks. But it only works if people basically agree that lawsuits are reasonable way to resolve things. If Genghis Khan laughs at the courts and does whatever he wants, this obviously doesn't work.Mistake theory? I like it. It lets people discuss a problem and propose solutions, and many eyes make bugs shallow. But if you hold to it too closely, you may find that what you keep assuming are mistakes are actually enemy action by someone deliberately trying to win a conflict with you. If someone is operating in bad faith, mistake theory is putting more and more time and value in while the other person manipulates you[2]. "Ah, I think I see your mistake now!" I cry out as Genghis Khan hacks my legs off. "You think I'm trying to usurp your empire, but actually I'm just explaining how my country became rich and prosperous!"
Contrast with techniques that help even if you're the only one doing them.
- Counting to ten before saying something if you are angry is great. It helps say fewer things you regret. It's nice if everyone does it, but even if you're the only one doing it you'll still have fewer angry outbursts.Writing down, in words and in a place everyone can see, what agreements you're making in a contract, and then asking people involved to sign it if they agree is great. It gives you something to refer back to if your memory is unclear, and especially if other people think contracts are important then you can take their signature as a clear yes/no as to whether they agree. But if it's basically just you writing notes to yourself as honestly as you can, that's still better than nothing.Verbally appreciating the things people do that you like is great. It helps people recognize you are on the same team. It's nice if everyone does it, but even if you're the only one doing it this can help build a cooperative atmosphere. (Less universally great- I have seen this used against the person doing it- but pretty good.)
Voting, lawsuits, and mistake theory do not socially degrade gracefully. Counting to ten, contracts, and appreciations do.
IV.
In Yudkowsky and Lintamande's Planecrash, there's a discussion of fairness and negotiation. It starts with this —
"The notion of a fair agreement, a fair trade, a fair division of gains from trade, a fair price, plays a central role in any civilization that relies on its citizens' conscious understanding of their activities. Dath ilan teaches the Law (mathematical structure) underpinning fairness, very carefully, and from childhood. After all, if lots of people ended up with widely different notions of what was fair, Civilization would stop trading with itself."
(emphasis mine)
— before continuing to talk about incremental steps building up to two ways to calculate what's fair. Picking the most relevant parts —
When the children return the next day, the older children tell them the correct solution to the original Ultimatum Game.
It goes like this:
When somebody offers you a 7:5 split, instead of the 6:6 split that would be fair, you should accept their offer with slightly less than 6/7 probability. Their expected value from offering you 7:5, in this case, is 7 * slightly less than 6/7, or slightly less than 6. This ensures they can't do any better by offering you an unfair split; but neither do you try to destroy all their expected value in retaliation. It could be an honest mistake, especially if the real situation is any more complicated than the original Ultimatum Game.
If they offer you 8:4, accept with probability slightly-more-less than 6/8, so they do even worse in their own expectation by offering you 8:4 than 7:5.
It's not about retaliating harder, the harder they hit you with an unfair price - that point gets hammered in pretty hard to the kids, a Watcher steps in to repeat it. This setup isn't about retaliation, it's about what both sides have to do, to turn the problem of dividing the gains, into a matter of fairness; to create the incentive setup whereby both sides don't expect to do any better by distorting their own estimate of what is 'fair'.
...
[Ketham says,] "If they offer you 6:6, accept with probability 100%. If they offer you 7:5, accept with probability slightly less than 6/7. If they offer you 8:4, accept with probability slightly less-less than 6/8. Does anyone want to try and guess the reasoning behind that solution, in advance of it being stated?"
"I see why it creates good incentives for the person who is deciding splits," Meritxell says. "...I don't see why the person deciding whether to accept splits or not has any incentive to do it, if they can't establish a reputation for it, and it's hard to establish a reputation for doing something sometimes."
[Keltham says,] "Well, reputation-wise, it's definitely easier to have a reputation for doing something if everyone in your entire Civilization got trained to do it at age seven or eight."
...
"And so long as that gets transcripted and sent out soon enough, hopefully nobody from Chelish Governance gives me a completely baffled look if I say that my baseline fair share of an increase in Chelish production ought to be around roughly the amount that Chelish production would've increased by adding me in the alternate world where the country had randomly half of its current people, or gets confused and worried if I say that a proposed contract clause would be annoying enough in a final offer to make me visibly generate a random number between 0 and 999 and walk out on Cheliax if the number is 0."
(emphasis mine again.)
(There's another bit about Shapeley values for when you have more than two people, but this excerpt is plenty.)
This is a lovely bit of math, and I can see how it would set up the correct incentives. However, it has a crucial problem, which is that approximately nobody in the world uses it, and few will know what you're talking about if you try to use it. How "approximately" is that "nobody in the world"? I'm the ACX Meetup Czar, I sometimes negotiate with Lightcone (the people who run LessWrong) and we haven't used this between us. If anyone is actually doing this in the real world, I'm going to guess that they work for MIRI.
If I tried to do this when negotiating my salary with a company during an interview, I do not expect it to work well for me. It's probably wouldn't be the worst possible failure case, but most of the companies I've worked for have been software companies and they're generally tolerant of eccentric weirdos. I don't think they'd immediately fail me out of the interview, but I'd lose points. If I tried it with a sandwich shop or a freelance client? Yeah, I think they hire someone else.
V.
I think there's a general lesson here.
Being an eccentric weirdo costs points in lots of situations. Many communication rules do not work well if imposed on others. Whether it's Non-violent Communication, Radical Honesty, Crocker's Rules, or some other communication method, it's worth considering whether this degrades gracefully if only a few people (or just you) are using it.
Postel's law states "be conservative in what you do, be liberal in what you accept from others." I view this as an attempt to make a system that degraded gracefully, not because it was failing, but because other parts of the network were failing. Postel was talking about the internet[3], and the internet being the internet a later update suggested to "assume that the network is filled with malevolent entities that will send in [messages] designed to have the worst possible effect." Obviously that's a silly amount of overkill and redundancy that we could skip if everyone would just be nice and learn how to do it right.
Ah. Nuts.
If you're thinking about a safety protocol, try and model what might happen if not everyone read the memo. If you advocate for a radical new governance structure, have you evaluated how it would work for people who don't understand or even disagree with it? Does it still work even if not 100% of the people around you are on the same page?
Will it socially degrade gracefully? Is that true only of ordinary mistakes, or adversarial action as well?
- ^
In the USA anyway.
- ^
Amusing if fuzzily remembered childhood anecdote: when my family played Scrabble, I initially tried to make the highest scoring individual word I possibly could, while my brother tried to help everyone make lots of words. We both lost badly to my dad, who tried to win. This was where I learned the quote "Never interrupt your enemy when he is making a mistake."
- ^
TCP you pedants
Discuss
