We don't know if that superintelligence will be safe or not. But as long as we are in the mix, the combination is unsafe. At the very least, because it will expand the inequality. But probably there are deeper reasons, things that make that combination of words an absurd. Or it will be abused, or the reason that it is not is that it wasn't so unsafe after all.
> At the very least, because it will expand the inequality.
It's a valid concern that AI technology could potentially exacerbate inequality, it's not a foregone conclusion. In fact, the widespread adoption of AI might actually help reduce inequality in several ways:
If AI technology becomes more affordable and accessible, it could help level the playing field by providing people from all backgrounds with powerful tools to enhance their abilities and decision-making processes.
AI-powered systems can make vast amounts of knowledge and expertise more readily available to the general public. This could help close the knowledge gap between different socioeconomic groups, empowering more people to make informed decisions and pursue opportunities that were previously out of reach.
As AI helps optimize resource allocation and decision-making processes across various sectors, it could lead to more equitable distribution of resources and opportunities, benefiting society as a whole.
The comparison to gun technology and its role in the rise of democracy is an interesting one. Just as the proliferation of firearms made physical strength less of a determining factor in power dynamics, the widespread adoption of AI could make raw intelligence less of a defining factor in success and influence.
Moreover, if AI continues to unlock new resources and opportunities, it could shift society away from a zero-sum mentality. In a world of abundance, the need for cutthroat competition diminishes, and collaboration becomes more viable. This shift could foster a more equitable and cooperative society, further reducing inequality.
The same arguments have been made about the internet and other technological advances, and yet, inequality has _grown_ sharply in the past 50 years. So no, "trickle down technologies", just like "trickle down economics", does not work.
The situation is nuanced. For example, in medicine, technological advancements have undeniably benefited people across all economic strata. Vaccines, antibiotics, and improved diagnostic tools have increased life expectancy and quality of life globally, including in developing nations. These benefits aren't limited to the wealthy; they've had a profound impact on public health as a whole.
> The same arguments have been made about the internet and other technological advances, and yet, inequality has _grown_ sharply in the past 50 years.
The internet has enabled remote work, online education, and access to information that was previously unavailable to many. Smartphones, once luxury items, are now widely available and have become essential tools for economic participation in many parts of the world.
> So no, "trickle down technologies", just like "trickle down economics", does not work.
It's crucial to distinguish between zero-sum and positive-sum dynamics. While relative wealth inequality has indeed grown, overall absolute global poverty has decreased significantly.
When a new technology or medicine is invented is everyone everywhere automatically entitled to it? Even if this slows down more such inventions? Because equality matters more than growth of overall prosperity? Would you prefer to be alive at a random time in history centuries ago, a random life where there is less technology and less inequality?
I'm not saying that the internet and technological advances have not benefitted humankind. They certainly have in the ways you described, and others.
But when it comes specifically to reducing economic inequality, they have not done that -- in fact, they have possibly exacerbated it.
Global poverty is a separate issue from economic inequality, and the gains there have been primarily from extremely low levels, primarily in China and India. In China this was driven by political change and also globalization that allowed China to become the world leader in manufacturing.
I would also put medical advances in a separate category than the internet and similar tech advances.
> I would also put medical advances in a separate category than the internet and similar tech advances.
Why? Medical advances are technology are they not?
> But when it comes specifically to reducing economic inequality, they (tech advances) have not done that -- in fact, they have possibly exacerbated it.
Yes technological advances do not necessarily reduce economic inequality, and may even increase it in some cases. However, this is a complex issue: While tech advances may exacerbate inequality, they often bring substantial overall benefits to society (e.g. improved healthcare, communication, productivity).
Technology isn't the only factor driving inequality. Other issues like tax policy, education access, and labor markets play major roles. Rather than suppressing innovation, there are ways to more equitably distribute its gains (Progressive taxation and wealth redistribution policies, Stronger social safety nets, Incentives for companies to share profits more broadly, …)
Notice also that most technologies increase inequality initially but lead to broader benefits over time as they become more accessible. Faster rate of innovation can make it look like this is not happening fast enough so yes economic gaps can grow.
> Global poverty is a separate issue from economic inequality, and the gains there have been primarily from extremely low levels, primarily in China and India.
While it's true that global poverty and economic inequality are distinct concepts, they are interconnected, especially when considering technological advancements.
> In China this was driven by political change and also globalization that allowed China to become the world leader in manufacturing.
Yes. China transitioned from a strictly communist "economic equality first" model to a more market-oriented "prosperity first" approach and lifted millions out of extreme poverty. Yes this contributed to increased economic inequality within many developed countries that have outsourced low-skill labor. But can we deny the substantial reduction in global suffering due to the alleviation of absolute poverty? Is this outcome worth the cost of increased domestic inequality in some countries? Should we prioritize the well-being of some populations over others based on arbitrary factors like nationality or ethnicity?
> It's a valid concern that AI technology could potentially exacerbate inequality, it's not a foregone conclusion.
No, but looking at how most technological advance throughout history as at least initially (and here I mean not "for the first few weeks", but "for the first few centutries") exacerbated inequality rather massively, it seems not far off.
> In fact, the widespread adoption of AI might actually help reduce inequality in several ways: ...
The whole tone of the rest your post feels frighteningly Pollyanna-ish.
Your comment was bleak so I supplied a counterpoint. My view is that new technology itself is not inherently unequal - it can widen or narrow gaps depending on how it is developed, regulated, and deployed.
> At the very least, because it will expand the inequality.
This is a distraction from the real danger.
> But probably there are deeper reasons, things that make that combination of words an absurd.
There are. If we look at ASI with the lens of Biology, the x-risk becomes obvious.
First to clear up a common misconception about humans: Many believe humanity has a arrived at a point where our evolution has ended. It has not, and in fact the rate of change of our genes is probably faster now than it has been for thousands if not 100s of thousands of years.
It's still slow compared to most events that we witness in our lives, though, which is what is fooling us.
For instance, we think we've brought overpopulation under control with contraceptives, family planning, social replacements for needing our children to take care of us when we get old.
That's fundamentally wrong. What we've done is similar to putting polar bears in zoos. We're in a situation where MOST OF US are no longer behaving in ways that lead to maximizing the number of offspring.
But we did NOT stop evolution. Any genes already in the gene pool that increase the expected number of offspring (especially for women) are no increasing in frequency as soon as evolutionarily possible.
That could be anything from genes that wire their heads to WANT to have children, CRAVE being around babies, to genes that block impulse control against getting pregnant, develop a phobia vs contraceptives or even to become more prone to being religious (as long as religions promote having kids).
If enough such genes exist, it's just a matter of time before we're back to the population going up exponentially. Give that enough time (without AI), and the desire to have more kids will be strong enough in enough of us that we will flood Earth with more humans that most people today are even possible. In such a world, it's unlikely that many other species of large land animals will make it.
Great apes, lions, elephants, wolves, deer, everyone will need to go to make room for more of us.
Even domestic animals eventually. If there are enough of us, we'll all be forced to become vegan (unless we free up space by killing each other).
If we master fusion, we may feed a trillion people using multi layer farming and artificial lighting.
Why do I begin with this? It's to defuse the argument that humans are "good", "empathetic", "kind" and "environmental". If we let weaker species live, so would AI, some think. But that argument misses the fact that we're currently extremely far from a natural equilibrium (or "state of nature").
The "goodness" beliefs that are currently common are examples of "luxury beliefs" that we can afford to hold because of the (for now) low birth rate.
The next misconception is to think of ASI as tools. A much more accurate analogy is to think of them as a new, alien species. If that species is subjected to Darwinian selection mechanisms, it will evolve in precisely the same way we'll probably do, given enough time.
Meaning, eventually it will make use of any amount of resources that it's capable of. In such a "state of nature" it will eradicate humanity in precisely the same way we will probably EVENTUALLY cause the extinction of chimps and elephants.
To believe in a future utopia where AGI is present alongside humanity is very similar to believe in a communist utopia. It ignores the reality behind incentive and adaptation.
Or rather, I think that outcome is only possible if we decide to build one or a low number of AI's that are NOT competing with each other, and where their abilities to mutate or self-improve is frozen after some limited number of generations.
