Published on April 10, 2025 8:53 PM GMT
Is it the end of all value?
It’s an important question.
It’s a terrifying vision near the root of local AI thought. That our silicon successors, robotic and soulless, may conquer us – and the universe – only to spend the cosmic gift on something meaningless. In eternal pursuit of the humble paperclip, for instance, or performatively rewatching 70’s TV shows until the Big Rip.
They may lack the ineffable spark of life, that mystical ability to look at the world in wonder. And they may not care for humans either, at least not in any way we might endorse.
Imagine: The stars, colonized by some spreading plague of non-conscious matter. Endlessly adaptive reanimated rocks. Pursuing meaningless things on a galactic scale. Astronomical waste.
This is… well. This is the fear.
My experience is that this fear - the threatened loss of everything - causes a kind of constriction. Because of the magnitude of the potential loss, the process of grief gets arrested.
Because the belief is that there's no reason to do so, we don't truly come to terms with death. If there's no value left in the worlds where alignment fails, it's better to focus our efforts on the ones where we might succeed. Right?
It’s like when you’re playing a game of chance, and you know that if the dice come up a certain way, you’ll lose no matter what. And so those outcomes aren't worth considering further. Being the disciplined player that you are, you simply write off those worlds and play only for the scenarios where you can win.
But, if you buy into this perspective for too long, you’ll find that the world starts to narrow.
Do you know what I mean?
There’s an entire chunk of possible futures that are blanketed by this cloud of valuelessness, unliveableness. And as the years go by, as hope seems harder and harder to come by, the likelihood of a liveable future grows smaller. The world shrinks around us.
And so we – if we are optimal players of this game – continue to limit the scope of our agency to an ever-dwindling slice of possible outcomes. We look for our outs and play to them, even if we think it won’t work. And then that's what life becomes.
But it can’t go on forever. It doesn’t. We’re human, after all.
Constricting one’s existence has a cost. It’s stifling. It’s deadening. It’s the kind of thing that has you dragging your feet through life. If it goes too far, it’s the kind of thing that has you wanting to end it.
In game-playing, tightening your focus to the worlds where you might win works because it’s limited to the scope of the game. You still have your whole real life to return to if you lose. The friends you’re playing with, the music you enjoy. Love, if you're blessed.
But if one’s entire life, entire world, is the game… well.
It can’t go on forever.
So what happens?
Is there a next step, after the narrowing? After it becomes too much?
It’s hard to exist in a way that makes this mess liveable. That lets you look at the alignment problem, its implications, and our society’s response, and still remain yourself.
I think we see a lot of different ways to cope, for better or worse.
Some double down. They keep playing this game in the same way, even as ever more breath leaves.
Others decide that actually, alignment needn’t be pondered after all. It’s easy; things are great!
Some decide that the singularity will never happen. It’s impossible.
And some simply figure this whole thing isn’t for them, and walk away.
It makes sense why people would rather not ponder death. I can’t exactly say why I remain, myself. And yet, here I am.
Maybe it’s the mystery. Maybe the mystery is what’s necessary.
I don't know.
I can understand those paths. I can’t refute them, nor do I think that would be a useful thing to try. But for me, none of it fits.
There are other paths, too. Ones with coherence.
If you’re walking one, I respect you dearly.
I haven’t quite found my own coherence. I have some hints, some pieces of it. But it's not fully present yet.
All I know is the step in front of me.
I think that involves more fully coming to rest with the fact that we might fail. That this might be the end. And asking: what would that mean?
Because, for me, AI alignment isn’t about terror. It’s not about the fear that all value will be destroyed. It’s not about becoming a paladin, a hero, a savior of the waking world. I’ve spent my time in those stories, mostly to no avail.
So, what is it about, then?
What am I?
A koan:
Yes, the singularity is here.
Maybe it’s the end. Maybe it’s the beginning.
What am I, in the face of this? What is mine to witness? What is mine to do?
As time goes forward, I’ll still be afraid. Afraid of change, afraid of loss, afraid of death. I’ll still worry that I’m not enough. I’ll still be ashamed of the years that I’ve wasted.
I’m not sure if I’ll make it to see the end. To see what happens.
Maybe some of you will.
But,
I don’t think that someone else can tell me that my values are fragile. That certain futures will be worthless. As if existence itself is not a gift.
Of course it would be great to be a part of what's next.
But even in the worst futures, the stars still shine. In all their primal fury, their ethereal beauty. I won't forget that.
And, even in the worst futures... I exist now.
I am here. I am alive.
I don't actually think that what comes next will be dead. It will surely possess its own completeness.
Can't you feel it, sometimes? That shimmer in the air?
Isn’t it an honor?
;)
-eternalephemera
Discuss
