I feel that Google search has been slowly dying for many years now.
Where in the past I could find documentation, forum-posts, wikis - concise information - these days I find SEO-optimised marketing fluff, link-farms, clickbait, long-form articles with thousands of words that ultimately say nothing, political tirades that don't make sense for any non-U.S.-citizen, and ads - tons and tons of ads.
It would seem that it was Google ads and Google search's own algorithm - which directly led to this state of affairs.
And that's the main reason why asking some LLM for answers is so much more attractive than using search: the AI cuts through all that crap, and extracts the actual information. If it's not hallucinating, that is.
The web is no longer for humans or for finding information. It's for bots, crawlers, for the Google algorithm and for marketing.
Humans simply can no longer digest it - we need AI to generate something more concise, more to the point, more digestible. Without the marketing, fluff and bait and filler.
They very much killed the golden goose. It's a real failure to think in terms of systems. Google relies on a healthy open web, yet as arguably the biggest single influencer of the web it has completely failed to maintain that let alone improve it.
As commercial enterprises currently are set up, they have no incentive whatsoever to preserve the ecosystem they are thriving on. Worse, they profit more when harming it. As seen here with Google, that is true irrespective of the nature of the affected system.
It's not just the quality of websites that is negatively hit either. The real victims are us, civil society and our future.
The information realm that is the internet should provide not only some basic tidbits, it should inform to the full extent of human knowledge.
Even more than that, it should provide us with the freedom to discuss among ourselves and reflect on ourselves, humans and humanity, about the future of society.
Without such self-reflection, there is only stagnation and retardation. We cannot change what we do not understand and we understand ourselves the least.
> Even more than that, it should provide us with the freedom to discuss among ourselves and reflect on ourselves, humans and humanity, about the future of society.
I usually hear this "freedom of speech" stance until somebody hears something they don't like, then it's out the door and censorship gets a foot in the door.
As long as there is no consensus in society what speech is ok and what's not, we won't get anywhere. Currently, we are mainly leaving this to platforms to decide, and they cater to their customers (which are ad clients), not to what is best for society. Obviously there needs to be some moderation and banned contents. But a line needs to be drawn and anything beyond that needs to be free. No matter if some individual gets hurt by it or not.
I don't get why this is supposed to be Google's fault. I mean it doesn't matter of course: this is happening, and the only real question is if it will continue, maybe if something can be done. Who's fault it is ... Google is doing a lot to preserve the web. It's not enough, perhaps. Probably. But it's quite unclear to me what they could do to reverse this.
Plus we've all seen what this has lead to in meat-space. Information is one sided. BBC is supposed to be the paragon of neutrality, for example, and while it is pretty neutral on the situation in many places, it's not exactly neutral on UK and Australia political issues for example. And it suffers from the same problem as pretty much every other meat-space information source outside of libraries: the only information you can get from it is really quite superfluous.
Their recent "is AI evil?" series, for example, is neither neutral nor does it give enough information for people, even if someone wanted to go really in-depth on the issue, and have a really well-considered opinion, the only place where you could collect enough information to do that would be Youtube imho. With some blogs and maybe a scientific paper or two. Oh wait 3 out of those three work with a lot of help from Google (just try finding scientific papers without Google scholar)
I agree the situation is bad. But frankly, while Google perhaps has some blame in this, I'm very sure the situation would be 10x worse without Google.
It is mainly Google's fault since Google is the main actor in this space.
You put up a false dichotomy, as if Google disappearing would mean a void to wander the world eternal.
The internet is very much like public infrastructure. It is crucial to explicitly determine what functions there are indispensable for society to thrive.
Ironically, finding available information is just such an indispensable utility. Imagine roads only being present or usable when conducive to advertisement.
I suspect that thanks to android phones collecting all of our offline data, and chrome collecting everyone's browsing history, Google no longer needs our web searches to collect the most intimate details of our lives which is why they aren't interested in spending time and money on their search product. In fact, the more useless google search is the more google can make companies pay for ads placed at the top of listings because if those companies don't pay up, their customers may struggle to find their site at all through all the spam.
Honestly, this blog post made me think Kagi has no clue what they’re doing. Nobody in 2022 would claim that Google uses raw PageRank and doesn’t do “user personalised queries”.
Google makes billions from advertisers - the companies who pay for ads, the companies who create those SEO-optimized marketing-pages. Those are the target audience, that Google wants to make happy.
Why would they care about us users? We don't pay for search.
One would think that given the purported depth and breadth of our economic system, it would be possible for that system to address all audiences instead of just catering to the most common denominator.
But we've been on a race to the bottom in most sectors for a long time now.
The other day I searched for "canned cabbage soup". The mobile website locked into a mode I've never seen before.
It would only provide blogspamish recipes for canning your own cabbage soup linked through the tiled suggestion interface. Not a single text-style link was returned and none of the results were pre-canned soups. I scanned though page after page without getting anything else.
Oddly, when I searched "canned cabbage soup history" it switched modes and started providing normal results, including the product and Wikipedia.
What strikes me about this is that I was actively searching for a product I wanted to buy. If there's any business case for ruining Google with weird specialty searches it would be selling products.
Why is there a "provide blogspam" mode?
Also, I'm now asked if I'm a robot at least five times a day.
Because blogspam sites all implement Google Adwords for monetisation, meaning Google can now show you 30 ads instead of just 3 at the top of the search result.
Remember, Googles customers are the advertisers and the site-owners implementing Google Adwords, not you.
Last night I searched for a phrase that was the title of a page. Zero results. I changed my search and what I was looking for was the first result. Google has really gone down hill severely.
> we need AI to generate something more concise, more to the point, more digestible.
This is what I don't want from a search engine. What I want from a search engine is a list of links to places that have relevant information. I don't want the engine to summarize or distill what those places are saying. Too much knowledge and understanding is lost by not doing that myself.
> Humans simply can no longer digest it - we need AI to generate something more concise, more to the point, more digestible. Without the marketing, fluff and bait and filler.
I think one of the fears some have about AI/machine learning/etc is that they realize their entire advertising & marketing world could become irrelevant.
Users could get short, accurate answers without having advertisements in the way.
What exactly does gatekeeping mean in the context of LLMs? One has several options: you could directly use a search engine, employ an LLM equipped with a search engine plugin for queries outside the training set, or personally fine-tune a LoRA with your specific data in a matter of hours. We have easy, moderate, and complex solutions at our disposal.
Rather than gate keeping, LLMs actually compress the search index into a manageable size, enabling easier sharing. While it's impossible to download the entirety of Google, you can conveniently download a LLaMA, promoting both privacy and autonomy. This marks a departure from the days of having every click and search keyword tracked.
Perhaps the advent of LLMs and Generative Image models by 2030 could allow us to cut the cord and navigate a virtual internet, have a room of our own to dream and be free, essentially functioning as an augmented imagination where we set all the rules.
Local LLMs solve the old problem of accessing information without leaking out what you are looking at, something that bothered me every since BitTorrent and copyright wars.
The entire internet has become a market for ads traded among corporations, and that the ads themselves are useless to 70% of the audience is meaningless because the market has its own momentum now.
The side effect is that the below-median brainpower of the 30% of the audience who doesn't know any better than to watch ads and click on them is now the driving metric for content.
In a world where programming revenue comes from ads, and only idiots see ads, the programming will cater evermore to the tastes of the most-idiotic viewer.
If I do scroll through the search results, select a few promising ones, wait for the page to load (slowed down by multiple fat ads), click away the cookie warnings, reject the GDPR consent, disagree with the newsletter popup,switch to reader mode to remove the visual clutter, scroll down the page skimming the content for the right paragraphs... then I will eventually get the same information, that ChatGPT could just have provided me instantly without all the hustle.
The funny thing is the site which should have been removed like the of Stackoverflow spam clones, or sites like Canva and Pinterest that make thousands of similar looking pages with slightly different heading are still allowed and rank on Google.
Also hate the top 10 pages whenever you search the best something, like best domain name registrar. I don't want to read a spam blog post with affiliate links, I wish google would show me the actual domain registrars instead (like chatgpt does when i ask it). Google has been gamed so badly and they have been doing nothing about it just because the spam blog posts contains their ads.
There are some tricks I learned on HN to use uBlock origin to filter these spam sites but Google really needs to fix this. There is only so much an adblocker can do to fix search. And right now all the useful content is getting blocked while the spam content is not only allowed but ranking on top of everything.
The moment I saw the term "SEO" it was like a stopwatch ticked on until search was dead. It used to be frowned upon to do little tricks, like keywords in a tiny, transparent font picked up by crawlers, but not seen by users.
When gaming search engines became a profession, the end of search appeared on the horizon. Guess we're headed back to web rings and link indexes (which will be consolidated, heavily monetized, and abandoned). If we're lucky, we'll be back to dialup BBSes by the 2040s.
Yes. We can point to the exact moment in time when Google turned to the dark side. It was on August 9, 2006, when Eric Schmidt, CEO of Google, addressed the Search Engine Strategies conference.[1] This was the moment that SEO, now officially endorsed by Google's CEO, became respectable. Until then, SEO was considered to be a branch of the spam industry. There were conferences such as the Web Search Spam Squashing Summit in 2005 on how to kill it.
There was/is whitehat SEO: properly creating links to relevant content within a site, using keywords that help bots find relevant content, traversing your own site to make sure there aren't dead zones with content that will never be found.
Google should have leaned into just this small segment of tooling and done much more of a ban hammer on the bad actors. They didn't and a bunch of people have left. I use DuckDuckGo mostly and sometimes Google but never by default. I don't even like DDG that much but it's good enough.
SEO sends the message that you have a content writer, and, thus, that you are a well established business (The content that is required to be produced is more nefarious than having hamster marketing guys spin in wheels, but the nature of the work is irrelevant to Google).
I feel like the turning point for me was when Google removed the ability to always exclude certain sites from searches. I had a number of sites configured Ty always be excluded because the results were always useless. Ever since, the list of useless sites in my search results has been slowly creeping up.
It really seems high-quality search is fundamentally in opposition to serving ads, alas. (At least once every page in existence probably serves ads via the search operator's network.)
That's exactly what Brin and Page said in the paper where they presented Google...
"[W]e expect that advertising
funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers"
and
"[W]e believe the issue of advertising causes enough mixed incentives that it is crucial to have a competitive search engine that is transparent and in the academic realm."
Both from The Anatomy of a Large-Scale Hypertextual Web Search Engine by Sergey Brin and Lawrence Page (1998)
... but more seriously: Yeah, their reasoning (now) isn't exactly going to be unbiased and/or unblemished by their own experience. They probably have the most extreme survivorship bias ever. (Not their fault, they just do because of whatever factors got them into the position they're in.)
Totally. Advertising is fundamentally about distracting someone so you can put money in your pocket without regard to the impact on that someone.
I have a lot of complaints about, say, McDonald's, but they make their money through giving people something of value to them. Advertising legitimizes the making of money in a way unrelated to value delivery. (The same is true about a lot of finance.)
When you combine that with up-and-to-the-right numerical goals and standard executive incentives, over time you pretty much guarantee what Doctorow calls "enshittification". Delivering value becomes at best a side effect of the system.
That's unrelated to the point GP is making. The point is that they are not incentivized to put poison into burgers to get money from the poison industry.
This is a good point. The problem is that it is very hard to make a living serving high-quality results which is likely why ad-funded search still dominates. The vast majority of the world will likely never want to pay for search on its own.
There are, of course, a few relatively successful paid general purpose search engines but these serve a niche demographic if you consider the world-wide scale of google et al. Possibly specialized search (we build one) will be able to thrive in the future, but these engines also serve niche markets in the end.
Thus, the real competition to ad-based search is not high quality search and that is likely why search results don't get better.
I think that there are current SEO practices that make sense that a search engine uses as a ranking signal, like accessibility (mobile friendliness, screen reader compatibility, +), time to load (performance, image optimization, no js bloat), security (https), no stuff appearing on top the content (modals), use of h tags and breadcrumbs to order content, srucured data for bots and more.
Of course people game the system, apply shady practices, sell courses with tips. It is a nuance topic, the name now is bastardized by marketing companies doing shady things, but on his core is just practices to create good websites and provide a good user experience
You might have stumbled onto a practical and useful application of LLMs. Yahoo (or whoever) could crawl the web, categorise each page, and provide a genuine "Index".
Like in the back of a book, or an old subject based card catalogue ... Dewy-decimalise the web :)
I've always felt that an index by subject would be more useful than string-match based searching. Of course, the index might rank links within each sub-sub--sub-sub...category with something like the original page-rank.
Now if Yahoo (or whoever) could avoid the enshitification trap ... imagine what a fabulous resource that could be.
Until ChatGPT came along, I figured it was inevitable that human curated search came back into ascendancy as the crawler model has become such a failure.
Now we can use ChatGPT to filter through Google's infested mess, but this double edged Sword of Damocles will be able to create infinite attempts to bury genuine content with ad spam.
I might pay like max. 3 EUR a year for this to get a search engine that gives the good results without ads, SEO spam and bogus clone sites.
That amount of money is probably more than Google now makes from my online presence because I adblock, block 3rd party cookies, tend to click "block" to everything including the idiotic "legitimate concern" and never ever click on ads.
Can't remember exactly where I saw it, but last number I saw said that Google makes ~$12 a year per user. Which begs the question... why have they not atleast tried a "Google Premium".
Fuck it, I'd pay $15 a year to have a Google search that puts as much effort into finding me the shit I actually want as Google does today in serving me BS ads I never pay attention to.
I always wanted to see just a "price transparency" aspect.
Tell me exactly how much the advertiser paid for his placement, and that's a hugely important signal here.
If I'm searching for weird hobby parts, even though it's a high purchase intent query, they're probably paying pennies per click.
But if you start searching, say, financial stuff and the ad placement figures start showing multiple dollars per click, it's a warning "these people are willing to spend THIS MUCH MONEY to present a message to you, this probably means there's something sketchy involved."
I know, for example, anything pertaining to insurance and financial products is highly likely to turn into a farm of cross-selling and personal-information harvesting, because the cost per acquisition is so high and the tendency for everyone to sell the information to everyone else is so great.
I'd argue most people do not even realize they click on ads. My mother! The ad call-outs got so inconspicuous that they're almost indistinguishable from ordinary search results. And if you don't realize that and just click on the top results, you're amongst the top profitable users.
This is an interesting point. User managed or curated indices offer unique advantages, especially when 'depth of coverage' is more important than 'breadth of coverage'. I believe that we are witnessing people shift away from demanding 'search breadth' as we speak, so someone might possibly decide to do this.
Everything else is effectively the influencer scene. Which is increasingly deplorable as well.
Anything with wide enough reach becomes cost-effective for gaming.
So one would have to return to a highly fragmented world to make gaming the system cost prohibitive.
And that would get us to a pre-Internet world. But then again, it’s not entirely unthinkable that we’re headed towards increasing Internet fragmentation if various governments get their way.
> If we're lucky, we'll be back to dialup BBSes by the 2040s.
Unfortunately, there's no going back - the closest we can get is BBS-via-SSH. The entire landline phone infrastructure is crumbling around us (or in many cases, completely gone). Voice calls are packet-switched now, rather than circuit-switched as in the past. The upshot is, fancy modulation techniques that made full-duplex 33.6k possible over voice-grade connections aren't going to work, and even good old Bell 103 (300 baud) may end up being problematic.
I'm not sure I can even get new landline phone service, and if so, it's going to be expensive - and the wire plant is an unmaintained mess. When I got my folks off their landline and onto VoIP some years back, their old landline had so much hum it was nearly unusable. Once the inside wiring was disconnected from the landline and connected to an ATA, the hum was gone. It wasn't our wiring.
I am trying something different that might work for you with aisearch.vip
The challenge will be staying true to not showing ads, respecting user privacy, and not requiring a subscription. So far, the only thing that works is free daily quota + pre-paid
More than 1.5 cents per search is a lot, they're pricing model really discourages me from making them my default search engine. A $10 unlimited search would be more justifiable. $15 would be stretching it.
If I'm not on a functionally unlimited unlimited plan (rate limiting is fine if it's reasonable) then I have to think about it every time I search. Should I really be searching here or should I be using the search box on stackoverflow or github or Wikipedia? Just introduces some constan cognitive frisson to the experience.
There is Ultimate which is not capped. That seems to be what you want?
Personally, my record was below 900 searches and usually below 800, so the 1000 searches (technically 1500 because early adopter) are absolutely fine for me.
They have block/lower/raise/pin for domains, and additionally, they support regex rewriting of domain names, all my twitter results on Kagi redirect to nitter (though I don’t know if that still works, I was gone last week)
It is better now, before the revamp, it had incredibly outdated information. Even now, there is very little reason for anyone not to go to the same page on MDN instead, unless they are a literally-just-starting-out beginner (as MDN has more information). That, coupled with them ranking high on organic results, makes people want to block them.
I always have a feeling they could get surprisingly far by having one person doing random searches, then tagging all the obvious spam to be deindexed. But Google is very adamant about never hiring a person to do a job well, if AI can be trained to do the same job badly
Don't they have whole teams of people looking at sites through the lens of the leaked 'quality rater guidelines' - ?
I thought it was like a few thousand people in the philippines downranking sites for thousands of reasons - without being transparent about possible biases and such.
Maybe I'm thinking of somewhere else.
And wasn't there a story about a bunch people who were doing similar as 1099 contractors in the US striking because they were not getting a living wage or something?
Looking at the quality of current google searches I highly doubt it.
I mean I could single handedly remove thousands of domains in a month and improve google search result, I have a hard time believing that they have 1000s of people doing it and the quality is this shitty.
I'm aware, but that's not the point. I went to that company's website around the time that case was in the news, and 100% agreed with google downranking it.
The point is, downranking decisions by google have a non-trivial chance of being litigated in court, and at that point, I doubt that the decision will actually be made on the merits of the particular site. A sympathetic website owner with a sympathetic regulator is frequently going to win against google, regardless of how shady the site actually is.
first ten results include:
theworkathomewife dot com › google-ads-quality-raterHow to Get Paid to Be a Google Ads Quality Rater
Sep 18, 2022 · As a quality rater, you ensure ads on Google – and other search engines – are both accurate and visually appealing. You will be given sample search terms and potential
Forbes -
After Months Of Protest, Google Search Quality Raters Finally Get A Raise
with a hand wavy - these raters don't actually effect the results..
I read dozens of pages of the raters guidelines pdf years ago - and I can see how they are being sneaky with things the 'down rank things that lack trust signals' - which in of itself sounds okay - especially with some situations..
but that exact thing can be used to push down entertainment sites, and can help adwords make more money by forcing lots of sites to pay to show up in the top 10 or miss 90% of the internet traffic.
So I agree that they are pushing shitty results in many search verticals - but yeah it does seem they are using thousands of people to create them this way on purpose.
It's no longer a better search engine - it once was - today it's popular because it's a default on many devices, and the network effects make it so people build their sites and update their listings like it's the only yellow pages in town.
Their high-brow censoring leaves much entertainment and other things better found using other portals to search and find imho.
Yea why cant i just remove these spammy Stack Overflow clones. Its wasting me precious time whenever I need some answers. Seems like such an easy feature but yea Google already lost its way a long time ago…
so Google should have a list of titles that if found cause them to ban the result, which frankly might affect people with little personal sites who aren't very good at making sites but just want to put up some stuff they really know a lot about, but anyway is there any other items in the list?
I'm pretty sure SEO sites will be able to figure out don't title your page "title (recommended)".
When all this search engine 'things' came about websites were indexed organically.
Then websites started to optimize for what Google actually parsed. Meta tags, no things behind JavaScript, semantic markup etc. And that worked really well. Stuff was easy to find.
Those were the best days. There was some luring, but at least if you were looking for something technical you found it (provided it existed and was indexed).
This is no longer the case. Give you an example:
Any search term for Windows+Error+KB<number> is a nuclear wasteland of companies having copy-paste pages only to sell you their services at the end. Lately the same has been popping up on YouTube.
Any search for an issue with formatting a harddrive lists 20 pages which sell formatting tools.
A search for checking whether a certain animal is dangerous yields wildly inaccurate results just to sell me pest control. Spoiler: it was a Jerusalem Bug. You don't get a fever at all. The thing just has huge mouthparts, and that can physically hurt.
When you put "(recommended)" in your title because you know it's then displayed verbatim in Google is nasty.
All driven by these websites which 'recommend' things to you.
You can use the AdGuard Annoyances list in the uBlock Origin settings to block certain types of content. This includes blocking cookie scripts that pop up every time you opt out of non-essential cookies. It also has an unbreak filter list that, when used in combination with EasyList, can remove anti-adblock warnings .
> The funny thing is the site which should have been removed like the of Stackoverflow spam clones, or sites like Canva and Pinterest that make thousands of similar looking pages with slightly different heading are still allowed and rank on Google.
More investment in Blogger with better linking between blogs (the way you can @someone on Twitter) and discovery (#tag searches allowing you to see posts from multiple blogs, curated blogrolls), more realtime/live logging functionality, easy microblogging functionality, etc. would probably have led to a lot more content existing on Google owned spaces, as opposed to being siloed behind Facebook and Twitter paywalls.
> The funny thing is the site which should have been removed like the of Stackoverflow spam clones
Absolutely. I have a browser extension installed specifically to block domains that are this type of spam from my Google search results, and I’m adding to it almost weekly.
Come on Google, you’re seriously ranking that as front page worthy?
> There are some tricks I learned on HN to use uBlock origin to filter these spam sites but Google really needs to fix this.
Oh I did use ublacklist which mixes blocking in both of the google search and the google image search. But curious what is the filter you have in your uBlock origin?
There are various sites that seem to crawl Stack Overflow and present the content verbatim in their own UI. They're annoying because they are sometimes out of date, they generate duplicate results and they're practically uninhabited.
Google stated goal was the "organize the world's information". Nevertheless, they didn't come up with Wikipedia, the highest quality curated human-readable information repository. They also didn't come up with ChatGPT, arguably the first LLM good enough that can perform non trivial tasks of data recall and organization.
Google had early success with search. Then decided to cement their lead by just throwing money at "all the smartest people"[1] so that their competitors would have difficulty hiring. This put Google in the situation where it's better to spend its resources fighting to keep the advantage that they have (by controlling the Internet infrastructure; android, chrome, email, etc) than innovating.
Google is dead man walking, unless something very substantial changes.
[1] "all the smartest people", read, "academically successful but rule-obeying, unimaginative, and risk-averse people".
> They also didn't come up with ChatGPT, arguably the first LLM good enough that can perform non trivial tasks of data recall and organization.
Not only did Google make the main breakthroughs that all LLMs still rely on today (mainly attention architecture, and large scale training of DNNs), but they had a chatbot similar to ChatGPT (Meena [1]) way before ChatGPT was a thing.
Granted: they didn't release it to the public, so there is a failure of innovation there, just not on the technical/machine learning side.
Xerox Parc also invented the laser printer, the transistor was invented in Bell Labs yet Intel (et al) take advantage of it. Google seems more like history now regarding search. Also, it doesn't seem good that the company founders are not there anymore. The last auccessful companies were run by the founders until a more mature point.
Also, regarding search, there are simply worthy competitors now. This is a bigger deal than I think people credit for. There truly was a time, maybe even ~ 2 years ago where even though it was broken Google was still the best. Kagi, while costing money to use, it fantastic.
YouTube is just bollocks now too, it contains a stunning amount of junk content and clickbait.
If someone tells me they learned something from YouTube, I actively question their authority on the matter at all. I didn't feel this way 2 years ago. YT seemed to be a generally credible source of information. Words cannot describe how polluted my suggestions are. I actually find it disturbing how much fake, incorrect and bias information is being suggest to me all the time. If I took it all on face value I'd be an idiot.
I've noticed recently that even people I thought were pretty credible have upped the clickbait severely in recent times. I guess this is what it takes to get seen but it's at the point where I just don't open YouTube unless I already know what I'm looking for so I can avoid it.
I never, ever log in on YT, and I'll sometimes clear my cookies as well. I find that if I stick with channels I like and carefully consider the first two or three recommendations, I don't get too much junk. But yes, the clickbaitification of even good channels is getting pretty grating.
Innovating is "easy", getting it in production is though.
If the Google models are so genius and perfect, and nobody can use or see them, it's like if they don't exist.
That should be very strong quotation marks. I would say that it is the other way around. We should be glad that not everything is immediately turned into a product.
Bingo. Getting it into production is a non-concern for non-entities with no reputation, but Google has a lot more to lose. The big guys watch and wait, and use their sizeable wallets when the time comes. I have no doubts that Apple is going to eat Meta's lunch soon, for example — a lunch Meta didn't even realize was there, for all their innovation and go-to-market.
Google had LLMs before OpenAI. It also produced the core techniques for making it happen.
My take is that it was taken by its own weight. The government fines & incorporation that started to fear those fines slowed down the progress. How difficult is to launch a change in that environment?
Same for the baseline -- people react to changes. You consider the baseline to be the Golden standard. How difficult is to launch a radical change considering that?
IMO, the only way out is to shrink the product. Split, shard, whatever. Be small enough compared to the rest so you are not the default target (Coca Cola, Procter & Gamble, etc. do it right).
> IMO, the only way out is to shrink the product. Split, shard, whatever
Microsoft and OpenAI nailed this in my opinion. The funding deal between the two was genius. OpenAI gets billions of dollars from Microsoft that it then spends on Azure.
Microsoft got priority access to ChatGPT, equity in OpenAI, high-scale testing of Azure, plus some margin on OpenAI's usage for training. At the same time, it _didn't_ get the legal liability of running ChatGPT. Maybe it could eventually be hit for allowing training models with copyright data on its servers, but that feels unlikely.
It got all of the access to cutting edge productized LLMs that it could have wanted, including financial upside, without the bad parts.
Are… are you blaming the government for Google’s enshittification? Because that’s the only factor related to its size you cite as causally connected to its decline.
Two parts. The government & organic size issues. Both contributing xx%.
Any high churn change is difficult to get out at that scale. Now add all corp-fear because you are under lense of all governments. You can't keep the startup mentality with that kind of environment.
> Wikipedia, the highest quality curated human-readable information repository
I have some doubts about the quality of Wikipedia content. Everything that can be politicised is politicised , everything that can be played in order for someone to advance his/her plans or point of view is played.
Apart from bad faith, some article lack relevant info or contain errors due the volunteer nature of it.
Wikipedia suffers from massive bias and subjectivity issues, and also from accuracy problems. Yet, all the attempts to do better fizzled out so far, and most non-niche knowledge and information repositories have the same or worse issues.
I think some subjects are just not fit to be learned by reading a biased encyclopedia article, such as political subjects. These subjects are better learned by instead reading forum debates, where several biased sides duke it out. A wiki article will just have the bias from one side.
If we look back at history, many of the earliest written works were published in this way, as a philosophical debate between different fictional characters.
However it seems increasingly difficult to find good online debate with several sides of an issue. People tend to silo themselves into their own bubbles, and denounce any platform who even dares to let an opponent speak his mind. I remember just a decade ago where you'd find all opinions on the spectrum on the same message boards. Now, people prefer to shelter themselves instead.
I know what you mean but if we consider that Wikipedia is the only such existing platform of that scope, it still technically is the one with the highest quality :)
Being of the "highest quality", in this case, is comparative to the landscape we have available. With that said I believe it may be the highest quality open information repository.
I think it’s possible for Wikipedia to have certain flaws and for Wikipedia to be the highest quality curated human-readable information repository available.
Android being open source means that any competitor can take it and run with it. Tbh, the main issue with that is the staffing required in terms of software engineers, but we’re Google to make the wrong move, I imagine Samsung would waste little time.
> Android being open source means that any competitor can take it and run with it.
Oh yeah we are really seeing all those successful forks taking off. /s
On a more serious note, the key parts are not open source, that's why all those alternative ROMs are having a hard time and nobody can really run Android without official builds outside a niche of enthusiasts who are fine with sacrificing certain functionality.
A few days ago, some people were discussing Schumpeter's idea that (very) large companies drive innovation. I'm curious what they would make of Google Search.
The core of Schumpeter's position here is that innovation requires making large long-term capital investments in risky projects, which small firms can't afford.
Things are somewhat different today then when Schumpeter was writing. On the one hand, governments and venture capital firms are now major sources of R&D funding. On the other hand, the growth of the financial sector has to some degree crowded out higher-risk investments within large firms.
The financialization of like, literally everything, makes most of that “classic” work of political economics people like to cite wobble badly. Hell, so does advertising if you look at the classic Austrian Econ stuff. It’s invisible in Mises’ Human Action (because it barely existed at the time! no fault of his I guess), and it causes massive tensions if you try to confront his framework with it.
This is conjecture, but I don't think it's a long shot: The rich are going to pay them to manipulate responses, exactly like happened with google. Whether it's ads or political influence, with the current state of wealth those who deep down only want to make money (and OpenAI has shown their colors, in my opinion) need to look upwards for those cash injections, not towards their consumers.
Right now they are producing value for consumers, in order to attract investors. Investors will want a return: the product will be the consumers.
And the vast majority of their ad money, comes from ads that show on searches. So if they lose search, they lose the ads too. So their money is in search.
You ask ChatGPT to write a poem and it replies with a poem about butter puff pastry biscuits made with layers of light, crisp puff pastry and expertly baked with nutty Gouda and mild Edam cheese which you can order with a click but in the style of Geoffrey Chaucer.
I find that talk page fascinating. I have only rarely looked into how the sausage is made over at Wikipedia, but I'm intrigued every time I do.
Not that anyone asked, but I think he is absolutely guilty of spreading vaccine propaganda - however, I think one editor has a very good point about how this should be approached. Here is what they said:
>I too would avoid the pejorative word "propaganda" on WP:TONE grounds. If they must be included, terms like "propaganda" and "conspiracy theories" should not be stated in wikivoice. Instead, it should be stated dispassionately what the sources say. See Deepak Chopra for some examples of more appropriate wording: "His discussions of quantum healing have been characterised as technobabble"; "The ideas Chopra promotes have regularly been criticized by medical and scientific professionals as pseudoscience." (emphasis mine). HappyWanderer15 (talk) 12:46, 17 June 2023 (UTC)
Doing planetary science, I used to be in the habit of searching for image IDs on google. Usually particular orbital images of the moon which had been mentioned in a paper. There are of course specific archives to search for this stuff, but google used to give surprisingly good results. Now I get literally zero results when I search an image ID. What changed? Did they remove their index of the Planetary Data System? Did they remove the indexes of planetary science papers?
I don’t think they removed the results from index, but I believe they started searching for vectors instead of text matches. Your ids are nowhere in vector space.
Is this why back in the day if you typed in 2 works it would show results with those 2 works and how the majority of time just the first?
When I'm looking for a technical solution I combine words. But Google gives me the middle finger, returns results for the first one which is like 'Windows Service' only to show me a generic website which has a download link for every KB number out there.
Shouldn't it be possible to work around this by wrapping the query in double quotes? Google promises to do an exact match search when wrapping in quotes.
So I'd imagine the OP's query should work when the image id is wrapped in double quotes. Otherwise, it'd mean that it's not in the index anymore.
It's like trying to write a Gmail filter with the "=" and then a word wrapped in quotes. You would THINK that would be a strict X = Y... but it's not.
At some point a few years ago the Gmail team made the "=" operator a "fuzzy equals", so for example it matches on parts of a word, even if you explicitly say you want to filter a word with a space at the end because it's "close enough".
Drives me fucking mental - I have an email filter that instantly archives every email from Jira except those that have my user name in the body of the notification (meaning someone has tagged me).
In Outlook I simple filter for "my_username " (with a space at the end), and it's able to tell the difference between that and my email address. In Gmail, "my_username " somehow matches both "my_username" and "my_username@email.com"... meaning EVERY FUCKING EMAIL I GET MATCHES THE CRITERIA.
I studied literature, and used to be able to find the most obscure tip-of-my-tongue references by describing fragments I remembered of a text or paper.
Try to use yandex instead, it is order of magnitude ahead of google when it comes to image search. Edit: sorry I was convinced you are searching by image, not image id.
It sounds like your suggestion came about by mis-reading the parent comment, but I will offer my 2 cents: yandex.com's english results for hard to find, technical terms are infinitely better than google's.
as someone who remembers How Google Used to Work, Yandex feels like Old Google. It indexes the hell out of long tails, will find exact, specific terms and phrases, and 100% respects generic search operators such as exclude "-".
do I look for news articles on it? hell no. do I find the unfindable (on google, and increasingly, on duckduckgo)? yes, I do.
My suspect is BERT and semantic search. Before it most word embedding models were monolingual and non contextual, so they were not useful for search. So Google had to use inverse indexes and an algorithm fine tuned for years.
With BERT they can have a model to embed all the search queries for 99.99% languages and works sort of fine for 99% of the queries, so they are fine with that as well.
It would have been sort of easy detecting that a query works better with the previous index than the new one, but it seems that they decided against that. So we lost a lot of usefulness, like phone number search.
I've heard that Google will process queries that don't seem to get results for the searcher and they improve if you come back to it. I never test this because I've usually given up (wasn't important) or moved on to other ways to find the information.
Not sure about Google but DDG changes the results when I come back to them. So, for example, if you search for something, click on one of the links to open it and then go back to the search results, there will be a couple of new ones added.
Not sure if some are removed also but it definitely changes.
Google has multiple search clusters, and indecies are not fully in sync, since they are updated asynchronously. So often the difference is the cluster your query is landing on.
Google's death is deeper than just the corpus, their ranking algorithm is also deeply degrading.
A week ago, a search for < James Palmer manchester foreign policy > wouldn't return https://foreignpolicy.com/2017/05/23/i-love-manchester-but-p... on the first page while < James Palmer manchester site:foreignpolicy.com > would [1] [2]. This appears to be fixed as of today but this kind of kindergarten level search intent not being fulfilled would have been unthinkable for Google even just a few years ago.
As you can see from the highlighted results, most are not actually exact matches. They're not even fuzzy matches. They're mostly synonyms. The search query also contains a lot of words that would be considered stopwords (i.e. ignored) in a simple keyword search and don't provide much semantic value in a natural language query (i.e. if it is parsed as a question), so those are probably deprioritized.
My assumption would be that the search results are garbage because the query is extremely ambiguous in the search engine's eyes, especially if you omit or modify any of the words (which it is allowed to do because it is not a search for an exact phrase).
So many people and groups want to profit with as little effort as possible that the commons (freely available data, open APIs, FLOSS software, etc.) is being overrun with little to no regard for the long term effects.
Data can be copied endlessly, but it has to be generated the first time and updated or else it becomes stale and its usefulness decays. If no one is willing or able to generate or update it, there is nothing good and the signal-to-noise ratio falls off sharply. Everything is noise and it sucks.
Yes, but in the history of humanity, economic growth of above 0.1% a year is also relatively recent. One of the things that C18 radicals insisted upon was a decent system of IP.
Id argue we're likely relating economic growth (which is measured by contended metrics) to the boom in quality of life we've seen. The problem with this is that the improvements in materials chemistry that have allowed for all these magical things was on the cusp of happening with or without IP. The drive for developing an IP system was more out of necessity in order to capitalize on the coming boom than it was to help that boom occur.
Case in point: look how far LLM's have come since llama hit the public sphere vs. how far organisations like google managed to get, with more time and more resources at their disposal.
Many years ago, Google was very useful without Reddit. There is a reason why they are such a behemoth: they actually used to create very usable tools and services. People wanted to use them, because their products were good. Search, Gmail, Maps, Translate (etc)... all of them are (were) gems.
Don't know if they let search rot or they broke it intentionally. But it got broken regardless of Reddit. Reddit has a lot of info I guess, but it is far from being the only website on the internet. However seemingly google stopped indexing more than the top 150-200 websites on the internet (and even those results are often lacking the searched words).
This happened because a few corporations were able to capture most of the internet due to a mix of laziness on the part of users and also the death of independent blogs and fan sites.
I suspect that the independent internet died for pretty much the same reason why many people won't post anything interesting or authentic on social media anymore: Being active online can lead indirectly to real world harm to yourself or your family. Evil people use the internet to harass other people and to attack their livelihood or put the safety of their family at risk. Many employers now discriminate against employees or even just people applying for jobs who openly express opinions on the internet. And even posting pseudonymously isn't completely safe because it is usually possible to deanonymize posters if an attacker is obsessed enough.
The sad reality is that many people do inauthentic things for the sake of career advancement (think of all the college students who do extracurriculars or projects for the sole purpose of "resume padding") and refuse to do things they actually believe in for the sake of their career. When creating quality online content becomes a risk to career advancement, people are going to stop doing it. In the long term, that means a siloed internet full of corporate propaganda, SEO optimized blogspam and clearly inauthentic posts.
In short, I believe Google search is getting worse because the internet is getting worse. And the internet is getting worse because our society permits employers to have a chilling effect on the online speech of their employees and those who might be their employees someday. And because the independent internet has shrunk dramatically, the handful of corporations that control almost the entire internet can now get away with siloing it off so that's what they're doing.
> I suspect that the independent internet died for pretty much the same reason why many people won't post anything interesting or authentic on social media anymore: Being active online can lead indirectly to real world harm to yourself or your family. Evil people use the internet to harass other people and to attack their livelihood or put the safety of their family at risk. Many employers now discriminate against employees or even just people applying for jobs who openly express opinions on the internet. And even posting pseudonymously isn't completely safe because it is usually possible to deanonymize posters if an attacker is obsessed enough.
Discussions moved from independent forums to Facebook groups and Reddit. Discord later. Facebook and Reddit are not more anonymous than independent forums.
Reddit do have a sense of being more anonymous since you can edit and soft delete your content. This is very rare for forums and most of the things I wrote 20 years ago still show up on Google. All my comments on Reddit are "gone".
The decision to limit the API might therefore actually be a good for the average user if it stops archives such as ceddit and unreddit to show your posts after you've edit or removed them. Most people probably don't know about them though and believe their content is gone just because they click delete.
Facebook and Discord groups have been private since as long as I can remember. Sure, someone might leak if you invite the wrong people but compare it with Twitter where you can find all posts from user X including keyword Y. Or even do it through Google cache. It takes minutes to find something to use against anyone if you really want to. Based on the last election in my country, it can be something as simple as listening to the wrong artist when you're 14.
Possibly, but when I think of "old internet" I think of people posting under pseudonymous handles for basically this reason.
> our society permits employers to have a chilling effect on the online speech of their employees
More generally than that. Dylan Mulvaney made one beer advert while being trans and got death threats. She would be counted as a "contractor" rather than an employee.
I disagree. The large sites which are charged with facilitating this breakdown have been around (and commanding a large portion of internet traffic) since before Google began degrading in-earnest. So did "effective" SEO, in fact. Google used to care about, and was still effective at, returning useful links well into the period that other entities were trying to undermine it. The first time that I noticed a reduction in Google Search efficacy was when they introduced Instant Search (or whatever they called the feature that presented results as users typed, rather than waiting for users to submit a full query). Even when they got rid of it, they never backtracked on whatever back-end changes enabled it (and degraded results in the service of presenting them faster).
My suspicion is that this was the beginning of the public presentation of corporate infighting, where what the public saw from Google began to be less about good products and more about the products that particular execs, managers, and developers needed to "win" in order to bolster their careers (or what they perceive as such). This is why Google has developed a reputation for killing useful products, often replacing them with flashier and more frustrating alternatives (if at all).
I don't like putting this on external factors. I think Google is doing this to themselves. It's a more private and stretched-out version of what's happening very publicly with Twitter right now.
One of the applications I used to use on a daily was Google Reader. Then one day, they just shut it down for no apparent reason, and that was that. They don't seem to be focused on customer satisfaction. Hard to understand.
Google Reader, like many other Google products, was developmentally mature (read: relatively cheap to maintain) and drove users to profit centers. An example that more closely fits your criteria is Google Play Music, which was superior to its replacement, Youtube Music, and which I paid monthly for. I cancelled my subscription when they sunset it.
I tend to equate the two. Especially in the case of Google, where you had to register, so they were in effect letting you use the product in exchange for personal data.
that makes a lotta sense even to my ignorant mind. so everyone got login'd prompted since companies make the most money data gatherin or what have you, plus reducing bots, stuff like that, it all makes individual sense and then amounts to search engines not working good anymore. Plus real blogs or websites with real human writing get buried or never shown because of bot or that SEO thing I tried to understand?
I'm just trying to wrap my head around it but I think I'm making sense of it now and it's kinda like that race to the bottom I see sometimes mentioned here? Do I got it right?
I'm all for reducing bots and spam, but registration doesn't seem to have solved that problem for reddit.
On top of this, siloing adds the extra bonus that it breaks things like The Internet Archive. There are a lot of communities on Slack and Discord that are just prime for vaporising without a trace. A TON of knowledge will just disappear once someone decides to turn it off, for whatever reason.
It’s easy to be mad at Facebook, newspapers, Pinterest, Discord, etc. for locking the internet away behind a login or paywall. But it’s also the result of Google capturing all the value from the free internet.
The end result is companies trying to capture more of that value for themselves and not letting Google eat their lunch, which required moving content and communities behind closed doors.
Google didn't invent the idea of a web crawler or search engine. Because of how the web was designed, I think it was always an inevitability that the internet would start open but trend closed.
You could say that google started it. They ate everyone's lunch by including what people were looking for in a box at the top of the search results. Of course sites will respond in kind.
Google search is broken because the internet has evolved in the AdWords era. Every person with a heartbeat has realized the monetary potential of making money from page views and SEO, so there’s a race to the bottom to capture the top spot for every search term in Google. Even if Google comes at it from an objectively user centric perspective this seems like an insurmountable problem to solve. But adding insult to the injury, post Sundar Google is just IBM/oracle at this point. So it’s just circling the drain at double the acceleration.
The second they paid people to game their search engine (ie adsense), the entire thing was on the road to uselessness. There always would have been that incentive, but Google turbocharged it, with $33B of incentive in 2022.
> Don't know if they let search rot or they broke it intentionally.
From the point of view of a shortsighted company this might be a false dichotomy because if ad revenue from search grows, search is fine: neither rotting nor broken.
You are assuming that Google treats users as customers rather than as an exploitable natural resource.
> What if Wikipedia started charging or restricting API access?
Wikipedia has a downloadable data dump that would cost almost nothing to serve to Google, and it has an organisational mandate to make that data available. If they decide to charge for access to that I'm sure Google can afford it. Let's not throw around completely unrealistic hypotheticals.
Great, this shows that they are unlikely to restrict as they already have a working business model for it with seemingly reasonable pricing (free for non-commercial, free monthly dumps, etc).
> What if Wikipedia started charging or restricting API access?
Then my Puzzle Game Redactle (https://redactle.net) would be threatened much like what happened to GeoGusser being charged for Google Maps.
Related to the article: I've also had trouble ranking Redactle on Google because there are a few poorly implemented ad-ridden versions which are part of link farming groups. Many of them pretending to be my 'Redactle Unlimited' brand. Google loves a bunch of spam sites linking to each other more than thousands of links from authenticated users on Reddit, Facebook etc apparently.
> If they decide to charge for access to that I'm sure Google can afford it
Sure, Google can afford it.
How about asking for 10% of revenue when Wikipedia data is used to show search results? Would Google agree or starts another rival to wikipedia and kill it after 4 years?
Systems are so complex that, it is very difficult to predict how they would behave given parts of system gets impacted by other constraints (Reddit vs OpenAI data scraping, Reddit's urge to make money, Reddit moderators protesting against rules, 2 day blackout extending to infinite, Google releasing LLM papers, which are impacting its own business through ChatGPT and so on)
Well that would just be obviously against their mandate. They're a not-for-profit dedicated to providing an open encyclopedia, requiring billions from likely their main distribution channel would be nonsensical.
Despite the recent actions of Twitter and Reddit, last ditch attempts to maintain any relevance in the public consciousness, most organisations act relatively rationally at a high level, and Wikimedia has the added bonus of not needing to make a profit or being publicly traded.
The true meta problem here is the age old question of how to fund services, including search, on the web. Google, Twitter, Reddit, Gfycat are all immediate variants of the same problem, which does not have a technological solution. The naive idea that all these things can be free, especially when promoted by those in an ecosystem as propped up as it is by ad revenue, is ridiculous.
Someone has to pay for these things to be developed and operated and typically when they do they get to call the shots, which leads to the modern UX disaster.
At least now the VC situation means we have a lot less product-dumping-by-any-other-name intended to destroy the market for legitimate participants.
I agree, and people's unwillingness to pay for the things they value causes the thing to no longer be 'for' them. It's for those who are willing to pay for it (investors).
Totally agree. But I believe we are at the beginning of the willingness to pay for internet services cycle. Many people can't or don't want to yet, but I think in the next 5 to 10 years it will get a lot more main stream adoption.
On the other hand, when I was growing up, HBO, Cinemax, and The Movie Channel were all part of my local cable system's basic lineup. Feature-length, commercial-free movies were part of the basic cable fee. I guess that was the hook to convince people to sign up for cable, since this was not a fringe reception area. It was only later that the ad-supported channels began cropping up and ultimately taking over, cable fees be damned.
Google problem now is not Reddit nor Twitter. Google converted in just a brand with really awful results and full of AdWords campaigns where it is not clear what marketers are paying for. It seems like they need to return to the original Brin and Page paper [1]. Google continue to do amazing things (e.g. AlphaZero, Project Zero) may be in the same way as Xerox Parc did amazing things in the past but nothing really new cames from the organization. BTW Google Workspace (e.g. Sheets, Docs, etc) is a good competitor in the office space. Good business execution, the search in Google Drive is awful.
Unlike the author, I'm not very worried about Wikipedia. Yes, they ask for donations, but that's not a sign that they're dying; donations are their main revenue source. That might be unusual among the web's top sites, but it's not in the grander scheme of things, i.e. there are lots of charities and nonprofits.
I didn't know that LLM training sets used Wikipedia. I would have thought that the CC-BY-SA license (on all text) would keep them away. It's not like Chatbot-4000 can cite all the Wikipedia articles used to train it, nor is Chatbot-4000 going to license all its output under a CC-BY-SA license (as required by the license terms).
Wikipedia has assets north of 100MM last I read about it. They don’t need donations to run the core wiki media products for a real long time. Their appeal for donations is absolutely distasteful and misleading. All the money they get they use for projects a lot of folks might not necessarily agree with. Such disingenuous behavior from supposedly the “best of us” is why public loose trust in all of modern progressive movement.
Is it just me? 100m doesn’t seem like “a lot” of money to cover legal, rent, infrastructure, engineers, administration, for a “really long time” to me. Having a reasonable buffer seems like a responsible thing to do.
To run just Wikipedia it seems quite sufficient. It’s KTLO on multidecade old technology with no real upgrade or new feature added in as much time and no exponential scaling needs. They should be set for a decade if not More even if their engineering stack is shit (which it’s not). They publish their spending. They are not spending on their website.
Honest question: What projects are potentially disagreeable?
Are you suggesting that: "free knowledge, open source and open data, transparency, privacy, and collaboration" are ideas that "a lot of folks might not necessarily agree with?"
(I fully admit that I may be missing something from the few minutes of research)
I am interested in this because I believe that the users of social platforms are far better served by non-profits like Wikipedia instead of the enshitification that is fueled by the profit motive.
When most people see the Wikipedia donation banners, they think they're barely surviving, and that their donation goes towards running the site.
That's very far from reality. The money goes to Wikimedia Foundation, which is swimming in cash and reserves only a fraction of the money to run actual Wikipedia. That organization now has hundreds of highly paid administrators. Not a cent goes to any Wikipedia contributor nor does this staff directly contribute to Wikipedia itself at all.
Out of this Wikimedia Foundation, they created some spin-off organization that answers to nobody and gives away grants. To what a conservative commenter might call "woke" projects without any tangible return on investment.
But the point isn't whether you agree with that label or not, the point is that the organization is incredibly misleading in what it asks donations for, how their financial state is, how their organization really works, etc.
It's quite similar to Mozilla. You want to support Firefox development but instead the money goes to a foundation which spends it on lots of well-intended but generally useless projects.
Google's Search is dying because of Thousands of self-inflicted cuts.
A couple of years ago there were about 3 or clearly marked ads on the first page, followed by generally useful results.
Right now, I don't have the patience to scroll past the ads which are almost indistinguishable from useful links.
The cut that hurt the deepest for me is when negated search terms stopped working. You used to be able to get subsets of data, finding things without a homonym, but now the algorithm just gives you the most common results and the most paid ads.
They retired "links:" (or something named similar) to find websites that linked to a given one. Was very useful, and I've been fearing ever since that they will do away with "site:" next.
There was also a time you could paste a URL to a .GPX file into Google Maps and have it loaded. Quick and easy, well before OSM and Leaflet.
I am starting to wonder more if Google search died or if the web did. You read a lot of comments about how in the past you could find a lot more information, but the reality is that no one is posting that information outside of the walled gardens. It's no wonder Google search sucks, if they don't have direct access to these walled gardens the information is locked behind the door...
They need to stop trying to so heavily curate what users are allowed to see in their searches. Since 2020, Google's attempts to operate as the internet's ideological censor have had a noticeable negative impact on the quality of their search results.
This is broader than DMCA takedown notices. For example, if you tried to look up the origins of covid, for a long time Google would try to shepherd you away from the lab leak hypothesis, since apparently that's dangerous wrongthink.
This post doesnt seem frontpage worthy, but I'll join in on the google bashing whenever I have a chance.
The other day I tried to google search 4chan, imgur and reddit came up before 4chan. heck 4chan wasnt even on the first page...
On a similar note, they are trying their best to hide wikipedia, now I have to specifically mention wiki most of the time.
I have the best website for a specific thing, I do well in quite a few SEO searches. However, the generic term, (which I'm still the best in), I am nowhere to be found. Blogspam and lower quality advice are all over the first 2 pages. Heck, there is actually terrible advice in the first 2 pages, like dis/misinformation.
The other day I was searching something I knew existed, I knew the page, and google would not give me the page. Ended up typing in the website.
Whatever is going on at google is in the major red flag territory. We need to perma fork android, we need to get off chromimum, find an alternative to Pixel. We are near the end times of google, they are going to be AOL taking advantage of old people and those who refuse to change their ways.
I've written many times that the consistent censoring results makes it ripe for competition.
I firmly believe that many of the algorithm changes for choosing results are based upon ideology, and a desire to push commercial things down below the first page as a money grab forcing people to pay the ppc auction extortion.
Which means worse results for most searches beyond 'local shop open / phone number' - google, the new yellow pages.
New smaller, niche engines; less censored, more federated is what I'm looking forward to.
> I have the best website for a specific thing, I do well in quite a few SEO searches. However, the generic term, (which I'm still the best in), I am nowhere to be found.
It's both, really. I seem to recall somewhere around 2006 Google changed to providing personalized search results based on your history. Since then the quality of search has declined horribly and my assumption is that it's a side-effect of the combination of hyper-SEO and a narrow focus on individual interest rather than categories.
I'm starting to believe that some sort of Yahoo model is the best solution because you want a lot of source material to search through, but it needs to be context-aware so that searches are meaningful. I am vastly more effective in my job when search results return reference information rather than blog articles about somebody's weekend hobby project that only explores the most basic parts of a problem I'm working on. And this extends to other categories in my searching as well.
I'm also personally wondering if Google has realized (even implicitly) that because of all of this, it no longer has to do deep searching anymore and can save tons of money by regurgitating cached copies of "related" garbage because people will give up before any expensive operations take place. It makes me wonder if Google's internal resource accounting has anything to do with it but that's just pure speculation on my part.
It's quite common for some queries to return a few pages when they might've previously yielded dozens. I'm not sure when it happened but couldn't have been more than 1-2 years ago. There's also less balance (ultimately results) for topics some might deem political, I may be on the left but I'm not blind.
Here's something to try: think of a really obscure TV show you watched when you were young, even better a specific quote. In my experience Google was pretty good at finding sources on where it came from. But now it's incredibly rare.
I would really love to read a deep dive on why Google Search quality has declined so much. There's a lot of anecdotes but I haven't read much that sounds particularly authoritative.
On why, or on whether. I personally find Google Search to be pretty great, and recently switched back from DuckDuckGo which I had been using for ~4 years, because I found I was having to use !g in half my searches. It's also noticeable faster. When I'm looking for an answer, I find Google much more likely to give me the answer. So I'm not convinced that search quality is declining. I'd love to see a deep dive on whether it actually is declining.
Disclaimer: I work at Google, but not on anything to do with search, and used DDG for my first year here out of habit. I have no inside info on search.
What's interesting is that I find Google's search quality today to be on par with the quality that existed before Google showed up and changed everything. Whether it is better than all the other garbage isn't really important to me. I want the good Google back.
It's the Circle of Tech. Earlier search engines such as AltaVista and Excite began to return what they thought you were searching for rather than what you were searching for, and Google was the fast, uncluttered alternative. Now the circle has turned, and today's Google looks an awful lot like yesterday's AltaVista.
This is more the fault of DDG / Bing being awful, than it is Google doing a good job. I’ve found Kagi and Brave at worst are on par with Google without being bloated with ads. And oftentimes they’re simply better.
I think it's unsurprising that a paid search engine (Kagi), or a likely unprofitable and optionally paid search engine (Brave), are better in some ways.
I don't find Google search to be bloated with ads. If I search for hotly competitive keywords then I do see ads, but the same is true where there is ad inventory on other sites, and when there isn't ad inventory that's not the true UX and it's only a matter of time until the ads arrive.
However, the majority of my searches are fact-finding, and I get facts, with no ads, or I don't even see the ads because I've got the answer before I get to them.
This is why I'd like to see a deep dive on this stuff, because the HN opinion is just not one that resonates with me at all at the moment.
In the TPUv4 paper it sounded like Google search moved over to some really crude AI embedding vector stuff. If you search "book of master system" it gives only results for "book of genesis" because it mixes up the biblical book of Genesis with the Sega Master System.
Searching any programming terms, like camel-case variable names from a system library without exact quotes, just gives random results about Katy Perry and stuff now.
Google search results are driven by clicks. There are much more non-programmers than programmers, so they pop non-programming topics to top of the list. Average user don't care about programming languages. Google is a general search engine, not a specialized one. There was attempt to spy on users and split them into different buckets, but it's dangerous to population, because a government can just ask Google for the list of «wrong-thinkers».
I get results for the Bible book when I search for "book of megadrive". Genesis and Megadrive are different names for the same SEGA console. Master System is a different console altogether.
What would you say is the primary issue? Personally I see vectorization and similar fuzzy techniques as the things that have most affected search results in the last <10 years, both positively and negatively. On one hand it allows me to search really dumb queries and still get sort of relevant results, on the other it makes well thought query tuning for more specific results seem futile. I don't know what goes on internally, but it's as if Google just discards as noise most of the words in the search query. "Verbatim" mode seems to tune that down somewhat.
Yeah it was book of megadrive that gave the biblical genesis results, not master system, I misremembered. It was going around on Twitter a few weeks back.
What is the motivation of anyone to make a website at this point? I don't see it.
I've not seen very much discussion about the issues these LLMs are going to cause, this article is sort of talking about it - twitter and wikipedia could restrict their content, but - what about the millions of webmasters who ultimately created what the web is today?
What is the motivation of anyone to really create content any more, apart from for actual social discussions on social media? LLMs will steal the content, and no one will read it.
I really don't know what this will bring in the future. Will online content fizzle out in 2-3-5 years and fresh content will just disappear? Will LLM only be good up to 2025 data?
This is a massive problem that changes the web as we know it, yet I don't see any discussion of this. Mark Andreeson bought it up for a sentence on lex Friedman, but that was it. He didn't know the answer.
> What is the motivation of anyone to make a website at this point? I don't see it.
Making websites to sell stuff and services for example, ie a business. You'll find some of the best information on websites that are a business, because it's in their direct monetary interest to get you that information. As opposed to ad-supported websites.
Google has sucked as a company from the very beginning. Not necessarily the technology, but the founders and the culture.
How many of you remember back in 2005 or so, when the first lawsuit for delisting was brought against Google by a small business owner? The Google attorney showed up with a complete history of this person's searches and proceeded to demean him in court.
Probably not many of you, because Google buried the story. Today it can't be found (I can't anyway). It was written about in a highly critical book of Google a few years later.
Think Reddit's API story is bad? Google did the same thing back in 2010-2012, slowly tightening the clamps on anyone monitoring them to put them out of business. My company was saved by a call from a famous billionaire to Matt Cuts, after which our little problem promptly disappeared.
I believe it's "Search & Destroy: Why You Can't Trust Google" by Scott Cleland but I haven't read the book in years (and I'm not about to buy another one).
The reason I remember this is because I wrote three books about Google, and I was sorely tempted to include the story but my publisher decided it would wiser to not include it.
completely tangential, but this thread triggered a wave of nostalgia reminiscing how good the few years around 2010 were (or rather seemed to be through rose-tinted glasses) in tech. Among all else it was an era when it really felt that google did just great on almost all fronts, search included. Web in general was lot less app- and mobile-centric because those things were still pretty new things comparatively.
Has there been any actual analysis showing that Google Search is getting worse nowadays? All I'm seeing are anecdotal reports and feelings, and your link shows that this sentiment has been around for more than a decade.
In 2010, I worked for a company that was active in SEO spam. Google began putting a stop to it with Panda in 2011. Search results were pretty bad back then, too.
For the years I worked at Google I always wondered when they would update the “organize the worlds information” mission. The reason was simple: information eventually will reorganize itself around everyone trying to make their own money. Not money for Google. It felt like a mission that would only decrease in urgency not bc they are solving it but the world (internet) would get better at it over time
in the 90's one search engine after another went this way. Each new engine worked for a while then advertisers started to game it and it gradually became useless.
It actually impressive that google has managed to avoid this flaw for so long.
The only thing I am not sure of is have advertisers finally found ways to game the system or is google doing it to themselves having forgotten why we all started using it in the first place.
I reckon that chat GPT will end up like the Douglas Adams HHGTG dystopian elevators at some point too :)
The author is overlooking that Google can cut a deal with Twitter and Reddit and everybody else. That's for mutual benefit. It keeps driving traffic to them which is great PR and it keeps Google relevant.
I use DuckDuckGo now almost exclusively for search. It's the default in all my browsers. A few years ago Google still did better for some searches but now I don't feel that to be the case.
I am not sure a different search engine does any better in the face of reddit and twitter outages, if those places held the content I want to find.
In the old days, we ran our own personal websites and we want our content to be discovered, in contrast with today where someone else host our content and wants to have a lot more say on what is discoverable. I think we need to have less reliance on centralized hosts if we want more control over searchability of content.
It’s a skinned version of Bing and Microsoft has forbade them from introducing any chat-based functionality. Hence their little demo going away quickly and quietly.
I also use it, but if Bing starts to make real gains and Microsoft thinks more DDG users will go to Bing than Google if they kill DDG, they will kill DDG.
They’ve been playing a very risky game for a long time.
>> Large models are trained on public data scraped via API. Content-heavy sites are most likely to be disrupted (why post on StackOverflow?) by models trained on their own data. Naturally, they want to restrict access and either (1) sell the data or (2) train their own models. This restriction prevents (or complicates) Google’ automatic scraping of the data for Search (and probably for training models, too).
This is going to be the interesting part to me - one HUGE usage scenario for AI would be doing automated searches and distilling results from the web - if sites start preventing that then AI innovation could be stifled - OR we'll see a scenario where the average person is very limited with that their AI subscription can access. We could have a situation where e.g. Fandango prevents AI from searching its site to help people plan their movie outing, but instead has their own model that they'll charge for access. We could have models talking to each other, deals made between the owners of the models for access, and the average citizen uses a search engine optimized for monetization, that may have data that's months out of date but they can pay extra for the model that provides current data.
Before, there was a very strong if not existential incentive to get indexed by scrapers, in particular search engines. Now the situation is flipped where there's a strong incentive to put up ever higher walls, in particular for LLM scrapers.
The difference is obvious: LLMs don't give back. They just rob your life's work and monetize it locally. Why would a platform or its users open up to that?
In the very early days of google, you could search for a simple problem, and the result would be on the search page. No point in clicking. I guess a few people clicked the other answers, so they rose to the top.
So now it is very rare to get the answer on the search page, because it makes no money for google.
While that's very true, I actually like searxng for that. And thanks to Googles slow death and the (it feels like that at least) Google's slow death, we'll see more of that and improvement to existing projects in the future, imho.
I think the article is mostly incorrect. The deep web has been way larger than the open Internet for a very long time. I see other problems with Google search like it does not work as well as it did some years ago.
>Large models are trained on public data scraped via API. Content-heavy sites are most likely to be disrupted (why post on StackOverflow?) by models trained on their own data. Naturally, they want to restrict access and either (1) sell the data or (2) train their own models. This restriction prevents (or complicates) Google’ automatic scraping of the data for Search (and probably for training models, too).
I've seen the other side of the coin, far too frequently. Many websites allow Googlebot to index for free, but once a person clicks on Google serach results, they ask for money.
Since Google collects a lot of personal data from each user, search results for the same query are different and personalized. This seems like a perfect solution - beside the tracking.
For example, if someone searches for ‘python’, Google can use the context to show results about either a snake or a programming language. But what happens if someone wants to learn to program in Python as first programming language and has previously searched for a frog and visited the zoo last week?
In my opinion, the information that Google collects is useless and doesn’t enhance the search results.
Y’all know there are other search engines, right? Yandex is becoming increasingly more useful for politically incorrect content and DMCA violating content. Bing has always been the best for porn. DuckDuckGo is great for many things as well.
SEO becomes a lot more difficult to pull off when there isn’t one search algorithm used by 99% of users.
If large websites like Reddit, Twitter, Wikipedia ban Google Search, maybe they die, too along with Google search.
Which means we return to late '90s and early 2000s situation when there were no behemoths hoarding data and knowledge and there were many specialized websites. I am happy with checking tech news on HN, checking stuff about cars on a car website, read about photography on a photography website and so on, instead of searching countless Facebook groups and subreddits.
From an DTC advertiser’s perspective, Google’s made so many hostile decisions of late: selling remnant space through Performance Max, ditching UA for GA4 without porting data, releasing Bard… Just failure after failure while the end consumer search experience just goes down the tubes.
Lots of smart people making very bad products that get in the way of other smart people making good products. It’s shocking to look at the wreckage of their decades of poor decision-making.
Reports of Google's death seem greatly exaggerated. The still have 95% share on mobile and 84% on desktop, the latter because Microsoft can make Windows try to force Bing upon you. Google probably have like 99% of the profits in search.
The idea of sites making their data inaccessible is not new. Experts Exchange is still out there siloing the answers but Stack Overflow has all the market share though being open. Google will be fine.
I switched to Bing recently. It looks exactly like Google with a few extra features and is slightly better at providing relevant results. What Google does is no longer good or unique. As for non-search, Google has killed so many products I will never invest in something new they make.
Seems like a lot of confirmation bias on this thread, but I've found google search to be fine for most of the search queries I've used. The real threat seems to be chatGPT which has been a great timesaver for information that needs to be customized
What google is no doubt trying to do right now as their highest priority is train their AI bot to sell products to you as part of its responses. I guarantee they are putting 100 times as much effort into that as they are into AI safety.
Because what Google considers a good search result is not what users consdier a good search result.
Why is every search result for even the simplest yes/no question a 1500 word article with 2-3 picture and an embedded video? Because that's what Google thinks is a good search result!
Google penalises your search ranking if users quickly navigate away from your page - this means if you have a page that quickly an efficiently answers a searchers question you get penalised whilst the site that buries the answer two thirds of the way through pure blog spam gets rewarded.
What is the incentive for "fixing" anything? Removing 1 ad improves the search experience but decreases revenue. On the other hand, adding 1 ad increases revenue. At this stage that's all the innovation I see google doing.
Google services are paid for by advertisers. On the fair market, there is competition that forces prices to decrease. Google is forced to sell more ads at the same price, or competitors will displace it from the market.
Because their revenue don't depend on search quality, they get paid for the top spots regardless of what's on the bottom and they are at a near monopoly.
Do they make more money if I click on a few SEO spam sites full of Adsense, and page through a few pages of Google results, before finding the answer or possibly giving up - or if I find and click on the top result?
Already everyone and their brother rate limits everything except googlebot (or everything except googlebot and bing)-- it's already a big part of what creates a search lock-in for google.
Over 50% of searches don't end up to being a click to a site which means less and less traffic is gained from Google who still monetizes the data they have scraped and the answer they give to users who search.
For content creators this has meant they need to find more and most importantly more agressive ways to monetize their content since less and less traffic means you need to show more ads to keep the money flowing. If google lowered the amount of answers it actually gives and would lead people to click on the sites more there would be less ads and users would be happier as well and there would be less paywalls as well.
Where in the past I could find documentation, forum-posts, wikis - concise information - these days I find SEO-optimised marketing fluff, link-farms, clickbait, long-form articles with thousands of words that ultimately say nothing, political tirades that don't make sense for any non-U.S.-citizen, and ads - tons and tons of ads.
It would seem that it was Google ads and Google search's own algorithm - which directly led to this state of affairs.
And that's the main reason why asking some LLM for answers is so much more attractive than using search: the AI cuts through all that crap, and extracts the actual information. If it's not hallucinating, that is.
The web is no longer for humans or for finding information. It's for bots, crawlers, for the Google algorithm and for marketing.
Humans simply can no longer digest it - we need AI to generate something more concise, more to the point, more digestible. Without the marketing, fluff and bait and filler.