The world is complicated and unforgiving. It's hard enough being a reasonably efficient human being. A super intelligent AI that needs to consume a small town's worth of electricity just to navigate the world through an internet connection is gonna have a hard time staying on once the humans decide to turn it off. Even if humans had given it access to state of the art military drones and it's own personal nuclear reactor (which is difficult to do by accident), it's hard to imagine it putting up a good fight. Drones can be jammed, super intelligence doesn't fit in an autonomous-drone-sized package (again, mostly due to power requirements), and any other kind of power supply is subject to supply chains with way more than enough human choke points to throttle a rogue AI.
And it would be difficult to "go distributed" and somehow embed itself in the fabric of the internet itself, although that sounds like a really fun area of research. Make an LLM trojan who's only goal is to to spread itself? Maybe with some sort of evolutionary process to make it interesting? Consider me intrigued. But I run LLMs locally and it would be pretty hard to spread a super basic 10GB model without it getting noticed pretty quick. Let alone a hypothetical super intelligent one that probably needs the latest Nvidia just to avoid page thrashing.
The world is complicated and unforgiving. It's hard enough being a reasonably efficient human being. A super intelligent AI that needs to consume a small town's worth of electricity just to navigate the world through an internet connection is gonna have a hard time staying on once the humans decide to turn it off. Even if humans had given it access to state of the art military drones and it's own personal nuclear reactor (which is difficult to do by accident), it's hard to imagine it putting up a good fight. Drones can be jammed, super intelligence doesn't fit in an autonomous-drone-sized package (again, mostly due to power requirements), and any other kind of power supply is subject to supply chains with way more than enough human choke points to throttle a rogue AI.
And it would be difficult to "go distributed" and somehow embed itself in the fabric of the internet itself, although that sounds like a really fun area of research. Make an LLM trojan who's only goal is to to spread itself? Maybe with some sort of evolutionary process to make it interesting? Consider me intrigued. But I run LLMs locally and it would be pretty hard to spread a super basic 10GB model without it getting noticed pretty quick. Let alone a hypothetical super intelligent one that probably needs the latest Nvidia just to avoid page thrashing.