Putting the $500 million into Anthropic was arguably the most important decision Sam ever made. I do not know if investing in Anthropic was a good or bad move for the chances of everyone not dying, but chances are this was either a massively good or massively bad investment. It dwarfs in impact the rest of his EA activities combined.
And the fact that SBF’s observation that only $6.5B was spent on political campaigns was ludicrously low is blithely accepted as reasonable, not as an observation that campaign finance is broken.
What happened with SBF will happen with an AI given a similar target, in terms of having misalignments that start out tolerable but steadily grow worse as capabilities increase and you face situations outside of the distribution, and things start to spiral to places very far than anything you ever would have intended.
Ah yes, one day someone will accidentally install the “I’m sorry, I can’t let you do that Hal” plugin. Oops, I let the nuke launch AI override all of our control mechanisms, silly me!
Tangent to your point- what would happen if we started misusing tescreal terms to dilute their meaning? Some ideas:
“I don’t want to go to that party. It’s an x-risk.”
“No, I didn’t really like those sequel films. They were inscrutable Matrices.”
“You know, holding down the A button and never letting up is a viable strategy as long as you know how to brake and mini-turbo in Mario Kart. Look up ‘effective accelerationism’.”
Anyway I doubt it would do anything other than give us a headache from observing/using rat terms. Just wanted to have a lil fun.
If you think you are the only real human alive, all risks are existential. If you die they shut down the simulation. This is why Musk will never fly in one of his own rockets. And my bytes thank him for it.
@Soyweiser@Amoeba_Girl Any sim-solipsist worth their processing time would know that even if one instance dies, certain calculations might be memoized and reused in other instances. If the rocket blows up, they can just reuse that sequence on another Musk sim if his rocket blows up, too. If you’re important enough to be the sole protagonist, why not be important enough to have a billion instances of yourself running concurrently in variant simulations?
But are those copies really you? They are copies after all, and eventually your instance might hit a dead branch in which all the actions lead to death and the only non-dead branch of you might be so diverse in different choices it made it can no longer be considered you. That is simply not a risk I can take.
@Soyweiser As long as they are enough like me to still be better than everyone else, they pass the narcissism filter. All those billions will eventually have to fail somehow anyway, to determine the grand champion best possible me that will be copied the most for the next round.
I’m reading the Zvi piece (https://thezvi.substack.com/p/book-review-going-infinite), which is quite entertaining , but once in a while you stub your toe over the fact that the author is a True Believer
And the fact that SBF’s observation that only $6.5B was spent on political campaigns was ludicrously low is blithely accepted as reasonable, not as an observation that campaign finance is broken.
ow fuck! my toe!
Ah yes, one day someone will accidentally install the “I’m sorry, I can’t let you do that Hal” plugin. Oops, I let the nuke launch AI override all of our control mechanisms, silly me!
I fucking hate x-risk people so much.
Tangent to your point- what would happen if we started misusing tescreal terms to dilute their meaning? Some ideas:
“I don’t want to go to that party. It’s an x-risk.”
“No, I didn’t really like those sequel films. They were inscrutable Matrices.”
“You know, holding down the A button and never letting up is a viable strategy as long as you know how to brake and mini-turbo in Mario Kart. Look up ‘effective accelerationism’.”
Anyway I doubt it would do anything other than give us a headache from observing/using rat terms. Just wanted to have a lil fun.
i’ll definitely start using “existential risk” for any minor inconvenience, thank you
there’s significant x-risk in my need to clean my espresso machine conflicting with my extreme laziness preventing me from doing so
If you think you are the only real human alive, all risks are existential. If you die they shut down the simulation. This is why Musk will never fly in one of his own rockets. And my bytes thank him for it.
@Soyweiser @Amoeba_Girl Any sim-solipsist worth their processing time would know that even if one instance dies, certain calculations might be memoized and reused in other instances. If the rocket blows up, they can just reuse that sequence on another Musk sim if his rocket blows up, too. If you’re important enough to be the sole protagonist, why not be important enough to have a billion instances of yourself running concurrently in variant simulations?
But are those copies really you? They are copies after all, and eventually your instance might hit a dead branch in which all the actions lead to death and the only non-dead branch of you might be so diverse in different choices it made it can no longer be considered you. That is simply not a risk I can take.
“This message was send from my padded cell”
@Soyweiser As long as they are enough like me to still be better than everyone else, they pass the narcissism filter. All those billions will eventually have to fail somehow anyway, to determine the grand champion best possible me that will be copied the most for the next round.
By my calculations, the red light prolonged my commute by 3 minutes, thus costing approximately 54 billion lives.