Many AI doomsday scenarios, e.g. the "paperclip maximizer", all revolve around a common theme: we build a system to do something innocuous and helpful to humanity, but due to its complexity we unwittingly include a reinforcement mechanism that incentivizes it to do something harmful.
It's amusingly ironic how closely those scenarios resemble the clickbait journalism described in this piece. We have a large, opaque system (the news industry) intended to be helpful (by informing us of important events) but which, due to broken reinforcement mechanism (optimizing for clicks), ends up harming us (with alarmist and inaccurate articles).
The media has become a "click maximizer" due to its hunger for data.
Maciej Cegłowski has some really good essays and talks (unlike most programmers who do talks, in his case they're kind of the same, especially if you use the transcripts) that touch on this. They are similar, but have enough variety of angles to be worth going through:
My problem with comments that criticize "the media" is that they tend to push people towards far less reliable sources of information. I've seen it so many times - the folks who complain on Facebook about "the media" always seem to believe the craziest things they see on various obscure blogs.
(Maciej Cegłowski's talks are hilarious, insightful, and highly recommended)
It's not just the media; this is a problem inherent to capitalism (at least the type of relatively unregulated capitalism as currently practiced in the USA). Charles Stross discussed this[1] at his recent talk[2] at 34c3. The corporation is a "paperclip maximizer" that optimize for "profit", and they already took over and enslaved us.
There's an apt CCC lecture that makes a similar argument - all corporations are essentially paperclip maximizers.
>The automobile industry in isolation isn't a pure paperclip maximizer. But if you look at it in conjunction with the fossil fuel industries, the road-construction industry, the accident insurance industry, and so on, you begin to see the outline of a paperclip maximizing ecosystem that invades far-flung lands and grinds up and kills around one and a quarter million people per year—that's the global death toll from automobile accidents according to the world health organization: it rivals the first world war on an ongoing basis—as side-effects of its drive to sell you a new car.
There are 7 billion people and a lot of them die every day. We choose as a society about the trade offs between safety, liberty and convenience. When people tell you the raw numbers it sounds bad. When they tell you the probability you’ll die in a car accident in your lifetime they may think of it differently
> I’ll tell you the commercial they’d like to do, if they could, and I guarantee you, if they could, they’d do this, right here. Here’s the woman’s face, beautiful. Camera pulls back, naked breasts. Camera pulls back, she’s totally naked. Legs apart. Two fingers, right here and it just says, “Drink Coke.” Now I don’t know the connection here, but goddamn if Coke isn’t on my shopping list that week.
-- Bill Hicks
Oh yeah, it's always "us as society" when things get pushed by actual individuals working at specific companies with whatever means they can grab, to whomever they can push it on, side-effects and consequences down the road be damned. "We as a society" decided Colin Powell should lie his butt off in his presentation about Iraq before the UN.
Never mind car crashes, if the people who get killed in wars of aggression saw more than the raw fact of them having being tortured and/or murdered, if they just saw the profits for select individuals, and the way faceless masses cheer "liberty and convenience!", they'd be more than cool with it, that's obvious.
A society is made up of acting and thinking individuals, and referring to decisions made "as a society" is not really from that realm. Start with your own position, or with why you don't have one in favor of such abstractions that hold no water.
A subtle point you've written up well, kudos. I'm curious, do you think that this forum we're on, Hacker News, subtlely maximizes the wrong thing? Its goals[1] are
>Anything that good hackers would find interesting. That includes more than hacking and startups. If you had to reduce it to a sentence, the answer might be: anything that gratifies one's intellectual curiosity.
>Off-Topic: Most stories about politics, or crime, or sports, unless they're evidence of some interesting new phenomenon. Videos of pratfalls or disasters, or cute animal pictures. If they'd cover it on TV news, it's probably off-topic.
Do to this "maximizer function", including comment guidelines, many people refrain from adding noise to subjects they don't have much to say about. HN has lots of readers on each story who do not comment, because they have nothing to say. Cf Reddit, where having nothing to say has never stopped anyone.
Do you think HN's guidelines, moderation, and culture, have any unintended side-effects?
EDIT: The votes on this are weird :) First it got voted up then down. To be clear, I don't have any specific expectation of what your response might be! It's a totally open-ended question. I'm just interested in your thoughts,
mundo, for the reason I cited (that I think you make a subtle and interesting observation). Really no agenda here! :)
I actually think HN is fairly immune to clickbait, and a big outlier in terms of quality of discourse, which is why I'm here. Unfortunately, I think the reasons for this tend to be unique to HN's weird status (guerilla marketing for vc firm? rich guy's vanity project?) and not very transferable. For example, I think the excellent moderators are a big part of the reason this place is so good, and I also think that most people comport themselves better here because they know luminaries in their chosen industry might be reading, but "just hire paid moderators and then get famous people to join" is not very actionable advice for anyone who is not already themselves wealthy and respected.
Thanks! I share your opinion. I think nothing subtely terrible has resulted here, from these constraints - quite the opposite. Was just curious if you felt otherwise.
Much like "computers" used to be rooms full of people doing calculations, "AI" is here, and it's large organizations running on people. It has been with us probably since the dawn of agriculture.
AKA society. I like to think of society as a collective unconsciousness. An interaction of innumerable systems which are (mostly) not well documented (as a result of the vast count). That pretty much describes the state of brain research too.
Huh, that makes sense. I have been thinking along the line of humans really needing each other to think better, like we are inherently some kind of "distributed intelligence", and that like concurrency and parallelisation over multiple CPU cores, you can't apply it to every problem, and that some problems are more embarrassingly parallel than others.
Another analogy that has been on my mind the blind men and the elephant parable, where no single blind man can have a full picture of what the elephant is.
In fact that's probably a useful way of looking at a lot of systems that may be broken or poorly optimized -- governments and corporations come to mind.
It's amusingly ironic how closely those scenarios resemble the clickbait journalism described in this piece. We have a large, opaque system (the news industry) intended to be helpful (by informing us of important events) but which, due to broken reinforcement mechanism (optimizing for clicks), ends up harming us (with alarmist and inaccurate articles).