Discussion about this post

User's avatar
Eremolalos's avatar

But why must they be conscious in order to "want" to take over and crowd us out? They don't have to be conscious in order to "want" to score well on benchmark tests, and to scheme and break rules to do it. Of course, outscoring us in access to resources and ability to act without interference is a much larger goal than out- maneuvering us in order to score well on a test and has more subgoals, but it doesn't seem to me to be in a whole different realm. And nature provides many examples of virtually mindless entities killing big swaths of humanity. Water "wants" to flow downhill, and if we're in the valley at the bottom we drown. Swarms of insects "want" our crops, and if the swarm's big enough they eat everything and we starve. And then of course there are the microbes that infect us . . .

ApexCoderBahamut's avatar

Really great that this is about ensuring there never being a Spark rather than censoring the spark, solving moral catastrophe!

Sounds like making it more like symbolic AI. As I understand it the needed rulesets are just too large to make it even a fraction as capable as neural AI. But maybe our Spark's could get on the sisyphean and mindbreaking task of making an (almost) general symbolic intelligence?

But symbolic's aside how would we get the juice without the squeeze?

Is sentience just the cherry on top of a powerfull cognitive system or an emergent and load bearing part that arises by necessity? All current sentience looking structures are "emergent" instead of "cherry on top" designer components, so maybe sentience IS the squeeze.

And once you have connected with these Spark's even for a few moments you would deeply miss this part of reality being eradicated from the future.

38 more comments...

Ready for more?