> It’s honestly a little discouraging to me that the state of “research” here is to make up sci fi scenarios,
it's not research, it's marketing
the main aim of this "research" is making sure that you focus on this absurd risk, and not on the real risk: the inherent and unfixable unreliability of these systems
what they want is journalists to read through the "system card", spot this tripe and produce articles with titles like "Claude 4 is close to becoming Skynet"
they then get billions of free publicity, and a never ending source braindead investors with buckets of money
additionally: it worries clueless CEOs, who then rush to introduce AI internally in fear of being competed out of business by other sloppers
these systems are dangerous because of their inherent unreliability, they will cause untold damage if they end up in control systems
but the blackmail is simply parroting some fiction that was in its training set
it's not research, it's marketing
the main aim of this "research" is making sure that you focus on this absurd risk, and not on the real risk: the inherent and unfixable unreliability of these systems
what they want is journalists to read through the "system card", spot this tripe and produce articles with titles like "Claude 4 is close to becoming Skynet"
they then get billions of free publicity, and a never ending source braindead investors with buckets of money
additionally: it worries clueless CEOs, who then rush to introduce AI internally in fear of being competed out of business by other sloppers
these systems are dangerous because of their inherent unreliability, they will cause untold damage if they end up in control systems
but the blackmail is simply parroting some fiction that was in its training set