What's interesting is that:

0. Internet is initially pretty good.

1. Google introduces search algorithm that is pretty good.

2. SEO becomes a thing because of Google, and makes the web bad.

3. AI (including Google's AI) bypasses that.

4. The web is eradicated, and Google/other AI companies are the only place where you can get information...

Seems like the web would have been better off without Google.

You are forgetting:

0.5 User 1,563,018 puts their credit card details in to make the world’s first online transaction!

0.50001 The web is filled with spam and unsearchable for real information

0.9 Some smart nerds figure out an algorithm to find the signal in the noise

1.9 Google throws out “don’t be evil” because search is cost, ad’s are money

4.1 Google and the rest of the AI/s subvert human decision making in ways that marketers could only ever dream

∞.∞ Humans toil in slavery to the whims of our corporate overlords

I wanted a different dystopia.

All you need is good information and AI tools are giving you information without you knowing whether or not it is any good. You may think it is good, but unless you know more about the answer to your query than what you needed to create it you won't be able to tell the difference. If you did then you would not have been asking in the first place. Effectively you are now believing in an oracle.

Which is different from believing the information on an ad and dark pattern infected website how?

Since the AI is trained on the data on the websites below the AI summary, the summary quality is basically lock step with the quality of the websites.

It is different because the AI is 'Google' branded and not 'ad and dark pattern infected website' branded. That proximity to Google's branding conveys trust, but it just whitewashes the content from that ad and dark pattern infected website. Only now you don't know about it.

Okay, but when those sites go out of business, where does the AI get its information from? This is obviously not sustainable.

It's only the advertising-funded sites that go out of business and a lot of those sites were in any case just scraping other sites. What proportion of reliable online information is only available from a web site that is funded by advertising? It's not zero, but it's not a very big number, either, I suspect, so it might be sustainable.

But that also kills the good sites.

Only if those “good sites” were dependent on advertisements for survival.

How many of the "good sites" do you pay for regularly? 99.9% of ALL sites are ad-funded, good or bad. Most people, even people with the disposable income, don't pay for the good sites, esp. because their value is only fractions of a cent from that one google search a month.

My irrational hope is that the "good" sites establish a shared Spotify-esque model, where I pay a basic subscription that then gets distributed roughly by usage to all the websites. There is no chance in hell anyone is willing to have the 20 subscriptions to support all the websites they've gotten utility from (directly OR indirectly) this month.

Some do