Some really good thoughts here. I'll summarize the ones that hit me:
- "Why are people searching Reddit specifically? The short answer is that Google search results are clearly dying. The long answer is that most of the web has become too inauthentic to trust."
This is it for me exactly. I search for the following kinds of things on Reddit exactly because results on other sites aren't trustworthy: Reviews are secretly paid ads. The "best" recipe for pancakes is only what's trending on instagram right now. The latest conditions on mountain bike and hiking trails are being shared inside communities like Reddit but not on the web. The same for trending programmer tools.
- "It is obvious that serving ads creates misaligned incentives for search engines..."
What I'm shocked by is that Google somehow maintained a balance on this for so long. Well, at least a good enough balance that people still use it primarily.
- "Google increasingly does not give you the results for what you typed in. It tries to be “smart” and figure out what you “really meant" ..."
This is the most annoying behavior because I really mean what I write.
- "There’s a fun conspiracy theory that popped up recently called the Dead Internet Theory..."
I hadn't heard of this. Now that's some sci-fi level of conspiracy but in today's world it seems totally plausible.
There's a reason why it seems shocking that Google has been able to balance the ads well enough that people still use it. They haven't! Google has orchestrated a monopoly over search engine distribution that allows them to get away with search results that are dominated by ads and spam, without losing most consumers.
Let's be blunt here - almost no consumer consciously chooses to use Google search anymore. Google has a distribution monopoly through Android, its deal with Apple on iOS and MacOS, and on desktop through Chrome.
I'm working on a search engine startup. It is in all practical senses impossible for an iPhone or Mac user to change their search engine to a new search engine on Safari or at the iOS level. And despite being technically possible on desktop with Chrome, it is for all practical purposes beyond what any typical consumer can easily do.
Their monopoly over distribution - not search result quality - is what keeps consumers searching Google and clicking ads.
> It is in all practical senses impossible for an iPhone or Mac user to change their search engine to a new search engine on Safari or at the iOS level.
On my IOS device, under Settings -> Safari -> Search Engine, I have a drop down with options, including Bing and DuckDuckgo, but defaulted to google.
On Macos, with Safari running, Safari -> Preferences… -> Search, Search Engine I have a drop down, defaulted to google, with Bing and DuckDuckgo amongst other choices.
Agreed on google”s effort to get their search engine as the default. However I just don’t understand how changing search engine is impossible given what I’m seeing on my devices? Nor does it seem over the top onerous to my eyes.
Yes, but you can't add a new search engine at all! So if a search engine isn't one of the tiny number of options in that dropdown, you can't change to it. That applies on both iOS and MacOS. And that option is used for the entire system-wide search, not just Safari.
So here's a challenge, try adding a search engine not on that list. You can see the search engine I'm working on in my profile if you're interested (I don't want to hijack this thread with self-promotion). I challenge you to change to a new competitive option like it. You simply can't. That is a clear monopoly over distribution.
On desktop in Chrome, as noted it is not something any typical consumer can do easily. But even if they could, Google does not allow you to set the New Tab to another search engine, even by setting the homepage to one. So every new tab opened on Chrome takes you back to Google search, even if a consumer figures out how to change their homepage. As for changing the nav bar search, no ordinary consumer is going to be able to work out how to change a search URL pattern. That is clearly intended to prevent consumers changing.
So I stand by my point, especially on an iPhone, you simply cannot change your search engine to a new search engine like us. It is impossible.
I fully understand your point and defaults are very strong.
That being said, I try new search engines from time to time and always get back to google, because non of the others have worked for me (in a professional context). I probably do 200 searches per day and google is most likely to give me relevant info on my first query (maybe 80-90% success rate). All others I have tried have been around 40-50% win an avg. of 2-3 search queries to find my result. That is a huge daily time sync on 200 searches per day.
I will also test your search engine.
And before having tested it, I have some unsolicited advice ;) At least these are things that would make me switch:
1) you are strong in my vertical. 50% of my daily search queries are professional. Probably 10-20% are programming related. If you were better 20% better than google at delivering results for that subset, I would probably use you.
2) If you had very strong support for my locale. Based in Germany, 50% of my private searches are in German. Most search engines, apart from google, suck in German. My assumption is their market share is so small that they don‘t put effort into any language specific search syntax understanding. German or large language groups like Spanish, Hindi, French come to mind.
3) If you can‘t become a default search engine on safari, maybe you can role your own browser (chromium fork or something) where you are the default. You could package it as: MySearchEngine App. It is actually a fully functional browser, but users really use it because they want to use your search engine. That might give easier access than having to manually navigate to your website in safari.
The ultimate test of any search engine is always the results. While the project I'm working on is definitely still an alpha, I'd love to chat with you when you try it out. I don't want to take the conversation here off-topic. To your general points though, there are definitely opportunities to provide better results within specialist areas of knowledge, and for local markets.
I think most of the search startups are doing their own mobile app. On iOS, the system search and browser remains Google/Safari (and the App is essentially just a wrapper on Safari for browsing). But at least it is something. I think you'll see more people doing desktop apps, although the dominance of Google through Chromium forks for this isn't a coincidence. It feels like the bad old days of re-packaging Internet Explorer with a custom homepage all over again.
Multi-lingual search results in general; Google (and FB) just for some reason cannot comprehend that someone might have reasons to search in multiple languages or even regions. For instance, I at one point was an editor/fact-checker for an academic publisher and one project involved checking a lot of information from official government sources around the world, and Google did not know what to do with that.
Hell, I'd love to have the option to make the search results be completely language agnostic.
Don't you just set the region and language at the bottom?
I also search in multiple languages but I also know that's an exception. The majority of the people in the world likely would only use a single language and it's easy to believe that search results are better for those users with languages separated.
Yes, hence why I'd want an option for language agnostic results. It would be a terrible default.
There are cases when I'd even like REGION agnostic results, or least ones not bound by national boundaries. For example, I have an interest in my state history (MI), and there's plenty of relevant articles/commentary from the Canadians.
I don't think it's too niche, I think they're just American.
A Canadian/Indian/EU-based Google probably would've ended up with multi-lingual support, whereas a China/UK based one probably wouldn't have.
By the time Google got big enough for international considerations, there were already a bunch of baked in assumptions about their users, like them being monolingual.
Metager.org is my go to google search replacement. I imagine (although my German is non native) that the German search results are quite good. . . it being a German search engine. To be honest it the only search engine to get me off google - DDG is OK, as is searx, but I kept going back to the dirty G until Metager.
Also a fan of Brave browser here. The more people trying new things the better! I know some people don't like the crypto-angle with it, but I think it is a positive to test new economic models to support online content, personally.
How much of the German challenge do you think is due to German being an inflected language? So a search engine that can't figure out the inflected variants of a word are going to potentially miss out on a large number of the relevant search results.
In addition to inflections, another factor may be the tendency to use compound words, so that a relevant word in context might not have a space at each end.
In English, Google is pretty good at figuring out when to add spaces in your queries (if you have a typo with your spaces). Similar technology should help it with German?
There's nothing magic about putting a space between words or not. Similar to whether you hyphenate or not.
Maybe it isn't a big deal; my thought was that if you're crawling a website and trying to associate its contents with search terms, parsing an uncommon compound word into the correct parts is going to come up more often in German than in English, especially if you're smaller than Google and German isn't your main focus. Are search engines good enough now to know that a string like electricovenmonitor is about ovens but not covens or Venmo?
At least in part this seems to be due to the profiles that Google has built. Whenever I try to use Google in anonymous mode, the results become noticeably worse.
Which, of course, means that it is yet another barrier to entry for any would-be competitors.
This could also be a perverse incentive reflected algorithmically (whether on purpose or not). Google has better data and makes more money when users are identified. So they have a vested interest in making users think that they will get worse results in Incognito mode.
Personalization is a double-edged sword. @pg once wrote about Google results becoming "what's true for you" rather than what's objectively true. And filter bubbles and subtle "personalization censorship" are also dangers. I think it's possible to have high quality results and privacy/anonymity, and it's not a binary choice. It's a challenge worth figuring out.
Totally agree that personalization is a double edged sword. Problem is getting people to actually want to change their personalized feeds/results. Most people have no clue what they’re missing and this issue has huge “Medium is the message” like implications.
I disagree about google being the best. For a start, their image search is massively behind bing. They do 0 work to remove pinterest spam. And DDG is way ahead of google on normal adhoc search.
Thanks, just to let you know, yes, it does have an opensearch description. But in practical terms that doesn't help much even if a startup search engine adds it.
Unfortunately, while OpenSearch is great where it's supported, outside of Firefox, the only real support is for in-site-search on other platforms (where you type a site name and then a search string), and not for changing your browser search engine. And it doesn't work at all on iOS even for site search.
So unfortunately it doesn't fix the problem of how a consumer can easily change their search engine to something new on Chrome or Safari or an iPhone.
I don't want to sidetrack the discussion, but if you want to confirm the opensearch description, you can open our site in Firefox, then click the "..." in the browser address bar and then click "Install Andi Search". Or reach out and very happy to talk you through it.
Depending on the platform, you might need to right-click on the Firefox browser address bar and choose the "Add search engine" option from the dropdown (where a website has it). Do you see an option to add a new search engine when you right-click the browser address bar at all?
[Edit: just saw the edit that you found it - thanks again for looking around for it too!]
I concur. And would add that on Safari and iOS, it suits Google and Microsoft too keep others out; noting all options are Google, Bing or Bing sydnicates). And it suits Apple nicely; $15 bn from Google, pure margin. How much do they get from Microsft/syndicates? Meanwhile all search listed options in Chrome are Google or Microsoft (Bing and Bing syndicates). And, to complete all Edge options are Bing, Bing syndicates or Google. Disclosure: also alternative search engine CEO.
Presumably you count DuckDuckGo as a "Bing syndicate". I think that doesn't do it justice - many of the things I like about DDG are specific to DDG, and I could not care less whether the underlying crawl was run by Bing or not.
Yes, but I'm biased. Your perspective is as valid and I respect it. DDG does have some great features and we stand with them in fighting for user privacy. What benefits us all is a variety of different search engines and search services. It is the search engines that subtly determine our informational pathways, so variety is healthy. For one independent review of search engines and syndicates, including DDG see:
https://seirdy.one/2021/03/10/search-engines-with-own-indexe...
100% agree. The more people working on this and building different features and trying different approaches the better. The more new search engines, the more everyone innovates, the better for the whole world. It is deeply unhealthy to have a single monopoly with its mono-search have 90%+ of the market. There are billions of search users with many different needs, and there is room for different approaches.
I see your point. I would have sworn that DuckDuckGo was added as a search entry when I installed that app on my ios device, however my memory is hazy from that long ago, so perhaps that search engine was added at a different point, like when it became big enough for Apple to notice them.
I'm a huge fan of DuckDuckGo. My understanding is that it took a significant amount of effort and public lobbying for them to get added to that list, and it was back in 2014 that was announced.
I suspect the big problem is that even if your search engine succeeds, once you get enough traction you'll just sell up to Google and everything slots back to normal 'do some evil' mode.
I don't want to get too off-topic, but personally I can promise you we will never sell to Google. This problem is very personal. As well as being a programmer, I used to work as a journalist, and I've seen friends in media have their lives and businesses destroyed by Google's ad-tech. And I've seen the Internet turn from a wonderful thing to become a cesspool of content farms, clickbait and seo spam. I can't speak for other search startups, but we will never run ads or use ad-tech or sell user data or invade people's privacy. We're just two people. But we're two people on a mission.
We pay nothing so are not included. On the other hand we are happy to provide a one click search from our search engine or eight other search engines/services, currently from our web app [0]. One click for Google (if that's your thing), one click for Bing or Gigablast search engine results, one click for Brave and some of the many Bing and few Google syndicates - DuckDuckGo, Ecosia, Startpage.
Even if DuckDuckGo and Bing pay absolutely nothing, it's good leverage to make Google pay even more, and its already public knowledge Google pays a lot to remain the default search engine on iOS, to the tune of billions of dollars a year:
The OpenSearch standard is great and definitely an improvement where it's supported, but unfortunately as far as I'm aware in most browsers that's limited to site-search (in website search after typing a url). It doesn't work at all on iOS.
On Firefox it makes it easier to talk a non-technical user through how to change their default search engine, and at least they aren't entering search pattern strings into a settings field as with Chrome.
As far as I'm aware and the OpenSearch docs, support is limited to in-site searches on most platforms outside of Firefox (not changing default search option for example using Chrome on iOS). Would be really interested in the steps you followed if you're happy to share them.
I then went to my site on Chrome iOS and it showed up under Settings > Search Engine > Recently visited. I then selected it and now anything I put in my url bar gets sent to my server. It's pretty sick.
That's interesting, I don't get that option on Chrome on iPhone (Search Engine > Recently visited), except for the officially supported five (Google, Bing, Yahoo, DDG, Ecosia).
I'm wondering do you see that as an option visiting our site (we have an opensearch description for Andi also) or other search engines like You with one?
Thanks for having a look. The opensearch file is https://andisearch.com/andi.xml, and it's correctly referenced from the "<link rel="search"..." opensearch tag. If you have a look in Firefox you'll see it prompts in the nav bar. So I don't think that would be the issue. Do you see anything for other engines like You with the opensearch head item?
I'm thinking maybe Chrome on iOS has its own flavor of implementation and possibly it needs the filename to be "opensearch.xml" rather than looking at the <link> tag. I'm renaming the file to see if that makes a difference. If you had an example of another site where it worked, that would be awesome. At least one more platform where OpenSearch works for setting the default site search would be nice.
You have a section in Andi Search which lets you try and set it as the default (in Firefox at least) but it doesn't work. Is that what you mean, or am I doing something wrong/weird?
BTW I really like what you're doing, and I'll definitely like to set it as default in Firefox somehow if I could just to try it out. My first tests were surprisingly good, I didn't expect it to deliver results which are accurate and which appear to at least match DDG. Nice one. [slightly unusual page format on desktop though :)]
Thanks! I don't want to distract from the main topic too much but would love to chat with you if you're open to it. There's a thread below with @neffity and I made some changes (trying to get Chrome on iOS to use OpenSearch) and broke that for a little. Update is just deploying now. I sincerely appreciate the feedback and you having a look too! :)
Firefox supports an open search standard which is a big improvement, so that's probably where you saw this. It provides an easier experience from the nav bar to add a new search engine to the browser. In practice, while it's a huge improvement, I've found talking with users that it's mostly helpful if someone technical is talking them through the change.
Hello Jed, that sucks. I understand that custom search engine exclusion is a financial decision by companies that can get away with almost whatever they want to do.
For iPhone, iPad, and macOS, you could use Swift and SwiftUI to write a single search app for your service. Flutter is pretty good also, then you could cover Windows and Linux also.
That's simply not true. Even as a technical user who knows what you're doing, you get a hard-coded choice of exactly five - Google, Bing, Yahoo, DDG or Ecosia. You can't add new ones at all. If you wanted to add a new search engine (like the one I'm working on, or any other), there is no way to do it.
But let's try it. Go to Safari > Preferences > Search and choose the "Search engine" dropdown, and you'll see you have a dropdown with only five options.
Google reportedly pays $15B per year for that top spot.
A work around is to add a safari extension like "Keyword Search" and associate a key word to your search. Its nothing like setting a default search engine for a normal user, but atleast there is a way for the slightly more technical user.
Though, I couldn't add a keyword to your site in firefox with its "Add a keyword" feature. Might be due to the javascript calling some url behind the screen.
Adding a shortcut to a website to the home screen is a nice convenience. Unfortunately, it doesn't add it as a search engine to your iPhone search, or to Safari. Even with a PWA (progressive web app), the App is just a wrapper on Safari. For a startup like us, it is probably the best option currently available, and we've tried to make it a good experience. But it does suffer the same drawbacks as a regular App as a browser replacement. If you swipe down from the home screen and type something into your iPhone's search bar, you're still using Google and Safari.
I'd be interested in the steps you took. Third-party Apps on iOS can't change the Safari or system-wide options for search engines as far as I'm aware. Installing a third-party App just gives you a wrapper around Safari for browsing while you use that App only. If you swipe down on your home screen to use the system search, or open Safari itself, nothing has changed. I can ask you to install our App or another search provider's App, but it doesn't change your iPhone's search engine or add it to Safari.
Yes, I found it a few weeks ago. It changes how safari works and improves it quite a lot.
I can block elements, and cookies in a site, define custom JavaScript or css, change the default search engine with one in a list o define a new one, and a whole lot of other things. All inside default safari.
That's interesting. I didn't think an extension could change the search engines available under Settings > Safari > Search Engine on iOS from the defaults (Google, Bing, Yahoo, DDG, Ecosia), or change the system-wide search used. Do you see different options now under Settings for search engines and did it change your system-wide search too (i.e. swipe down on home screen and then use the built-in Search)?
Swiping down doesn’t work, it’s in-browser only in the address bar. As for in safari, I don’t get options, but if the extension is enabled it redirects the search to Kagi.
> especially on an iPhone, you simply cannot change your search engine to a new search engine like us. It is impossible.
See my post above about iCabMobile, where there are TWENTY-FIVE search engines to choose from.
There are also other browsers than Safari and iCabMobile on iOS, many of which give alternatives to their search engine choices.
Naturally if you think only users who choose the default browser are interesting as your market, I wonder if those users would take a chance on your alternative search engine?
I replied to your duplicate post separately, but it is worth noting that even if you install an alternative browsing App and use it, the iOS system search (swipe down from the top to access the search bar) is still using Google and Safari. And even if you were to use another browser like iCabMobile, it also simply does not let you add in a new search engine not already in its own options.
> And even if you were to use another browser like iCabMobile, it also simply does not let you add in a new search engine not already in its own options
I duplicated my post because you so pervasively in this thread duplicated your inaccuracies.
You are wrong about this, iCabMobile allows the user to very easily add search engines:
Settings > Tools > Search Engines > Add
Naturally I expect to be plentifully downvoted for this accurate information, just like pointing-out that iCabMobile has 25 search engines to was also extremely unpopular.
I may not be following you, but are you saying that when you change search engine in your App, it also changes the default search for iOS and Safari system-wide?
Or you are just talking about within your iCabMobile App itself, like anyone can do in their own app?
As far as I know, all the alternative browser and search Apps (including ours for example) are wrappers on top of Safari on iOS. And they are unable to change the iPhone's system-wide search or browser. That's the search engine accessed, for example, when a user swipes down on their iPhone home screen, and enters a query into the "Search" field that appears. Normally, that's Google and Safari at the iOS system-wide level.
Or are you saying that you are able to change these including adding alternative search engines under iOS Settings > Safari > Search Engine?
I'm curious how you're doing this if so, and I'm sure lots of other people would be too. My understanding is that App Store Apps or Extensions can not normally change the iPhone search engines, but I'd love to be wrong about that.
> Google does not allow you to set the New Tab to another search engine
Firefox has removed the ability to set a default page for new tabs and requires users to install an extension to restore the functionality, which in fact provides degraded functionality. As originally implemented the new tab would load the new page instantly. With the extension, a new tab is created, focus is given to the URL bar and after a brief but noticeable pause, the chosen page loads.
Apple is operating a search engine right under our noses, hiding behind Siri. Applebot is regularly crawling the web. They’re probably much further along toward a privacy-focused search product than any of us would know.
"Web results" are. "Siri Suggested Websites" and "Siri Knowledge" are not — those are fed by Applebot. "Applebot is the web crawler for Apple. Products like Siri and Spotlight Suggestions use Applebot." — Apple's support document about Applebot.
I've wondered why no one has bought wolfram alpha and used it to power knowledge bases. Certainly it seems cheaper for Apple to have down that as a start point than to start from scratch. Maybe it's too technical for most users?
xSearch for Safari makes the search experience on the Apple ecosystem much better. https://apps.apple.com/us/app/xsearch-for-safari/id157990206...
Makes switching search engines easy and you can use "bangs" like in DuckDuckGo to use other search engines within safari.
"Let's be blunt here - almost no consumer consciously chooses to use Google search anymore."
This bluntness does not go far enough. People do not change defaults, no matter how "easy" it may be to do so.
A default is a pre-made choice by someone other than the consumer. There is no set-up process where the consumer makes a choice. The choice has already been made. Consumers do not make this choice. Even if they could, in practice they don't. That fact may seem insignificant but it is worth billions of dollars.
If I am not mistaken, the current CEO of Google spent most of his time working on "default search engine" (or "default web browser") deals before taking the CEO job. In probably the most important one, Google pays Apple a hefty sum to be the default search engine. It was estimated at $10 billion in 2020 and $15 billion in 2021.[1]
Defaults are effectively permanent settings. It does not matter how easy it is to change a default setting if practically no one ever does it. $15 billion is too much to pay for something that may or may not change. It does not change. It is money in the bank.
> If I am not mistaken, the current CEO of Google spent most of his time working on "default search engine" (or "default web browser") deals before taking the CEO job.
I mean you could spend two seconds to search and realize you were in fact mistaken before bothering to write "If I am not mistaken..."
Sundar Pichai was responsible for the Chrome browser and Android operating system. [1]
While the comment might be a little oversimplified, I think it's reasonable to say that those deals would have fallen under his ambit. And there's no question that Chrome and Android are the two central planks of Google's search distribution monopoly with consumers, along with the Apple deal.
> While the comment might be a little oversimplified
let's see:
> spent most of his time working on "default search engine" (or "default web browser") deals before taking the CEO job
even if we take your convenient assumption as correct ("it's reasonable to say that those deals would have fallen under his ambit"), it's still wouldn't be a true statement. But feel free to point out which part is true if we relax any more assumptions.
Based on the public record, I don't think the original commenter's statement was unreasonable or mistaken. And they have provided extensive additional documentation on a separate comment supporting the assertion.
Based on Pichai being the senior executive at Google responsible for Chrome browser and search defaults, as a matter of public record he held corporate responsibility for getting google search as the default search engine on as many devices as possible.
You stated that the original commenter was mistaken with no supporting evidence and a high level of acrimony in your phrasing. I'd be interested to see supporting evidence for why you think the commenter was mistaken, in the context of the other resources they provided.
Sure, if you're comfortable with your job being described as spending most of your time working on affiliate link deals, we can agree that's a good description of someone working on the Google toolbar plugin and then Chrome OS, Android, etc.
If anyone else was curious, Sundar Pichai had not worked on Search prior to becoming CEO, it seems:
> Pichai joined Google in 2004,[8] where he led the product management and innovation efforts for a suite of Google's client software products, including Google Chrome and Chrome OS, as well as being largely responsible for Google Drive. In addition, he went on to oversee the development of other applications such as Gmail and Google Maps. In 2010, Pichai also announced the open-sourcing of the new video codec VP8 by Google and introduced the new video format, WebM. The Chromebook was released in 2012. In 2013, Pichai added Android to the list of Google products that he oversaw.
Consumers don’t want to see the results of their search or find the answer to their question. They want the assurance of being told the answer by an authority. Google has tried to become that authority. It’s true, just Google it.
More about Pichai's role in making Google the default search on more computers and directing search traffic to Google, by any means necessary.
"Pichai started at Google leading product management for the Google toolbar, a critically strategic product that enabled default search queries on different web browsers to go through Google and allow them to track browsing behavior to power the AdWords targeting engine. At the time, Internet Explorer was the "installed by default" incumbent for many users, while Firefox was the alternative browser of choice."
"Pichai identified a weakness in Google's strategy, and Chrome began as a defensive play against the established browsers to protect and grow Google's search business (which still generates much of the company's revenue)."
"Sundar Pichai is the one who introduced the toolbar, which led to an increase in user searches. It was later merged with Chrome, which became the most used web browser in the world."
""Most people here didn't want us to do a browser, so it was a little bit stealthy. Once we had it up and running, I remember showing it to Larry and Sergey - and even then there was a lot of scepticism." But Pichai got his way: Chrome was released in 2008 and now accounts for nearly 60% of the market, according to NetMarketShare, while Internet Explorer languishes on less than 16%."
"Ten years ago, the Indian-born Pichai, 42, was a product manager at Google, and his domain consisted of the search bar in the upper right corner of Web browsers. He then persuaded his bosses to wade into the browser wars with Chrome, which in time became the most popular browser on the Internet and led to the Chrome operating system that runs on a line of cheap laptops called Chromebooks."
"Android runs on 1.2 billion devices around the world. It drives users to the company's hugely profitable search engine and the ads on its maps service. Google search and maps are available on phones made by Apple and Microsoft, too, but Google pays those companies referral fees. The more people use Android, the more Google can keep that revenue to itself."
"Google distributes the latest versions free under the agreement that device makers will highlight profitable Google services-especially search and maps-while their own brands and services take a back seat."
"Pichai joined the small team working on Google's search toolbar. It gave users of Internet Explorer and Firefox, the dominant browsers at the time, easy access to Google search. He proposed that Google build its own browser and won the support of the company's co-founders, though he faced an objection from then-CEO Eric Schmidt, who thought that joining the browser wars would be an expensive distraction."
"Rubin had introduced it, but Pichai created an interdisciplinary team with the search group, which had voice search technology and algorithms that could discern what information might be most important to users. "Sundar helped me to formalize a relationship," says Johanna Wright, the product manager who runs Google Now. "Because search and Android sit in two different buildings, we ended up doing a people swap.""
Thank you for putting that together. I knew there was some resistance to doing a browser internally, but it was a brilliant insight that long term Google would need control of distribution for its search results as it became more and more a mass consumer service, especially after becoming ads-focused and moving toward more mainstream users, and with the world moving to mobile after the iPhone launched.
Just tried https://andisearch.com/ and I like it. Felt like a fresh look on results instead of the same old SEO ones. For example, searched for a few Java queries and found very informative website/results that weren't dominated by Bealdung. Searched for "soccer scores", "chelsea FC", "prince andrew", "WP export" and found things that would never have been on Google's first page, but were excellent returns. Nice work.
Thank you. While I don't want to distract from the topic here, I do really appreciate your feedback and you trying it out, and would love to talk more if you'd like to reach out! Speaking generally, the world needs more people working on search, and I think there is a lot of room for completely new approaches.
I'm pleasantly surprised by the high quality of your top results. That shows the search algorithms (whatever your choice was) are good enough and Google can be eventually disrupted. As a fellow software engineer I'm curious about your stack and size of your operations. Are you blogging about this somewhere? You definitely should!
> That shows the search algorithms ... are good enough and Google can be eventually disrupted
I doubt this. I'm guessing that 90% of Google Search's development effort is concentrated on defeating SEO, and as soon as a new competitor succeeds well enough to attract SEO attention, their results will suddenly turn to custard.
The competitor then has to combat the extremely sophisticated SEO practises developed against Google over the years. This is likely to be an even bigger barrier to disruption than building market share against Google.
You're assuming here that Google's primary interest is consumers, but their paying customers are the advertisers. The reality is that the worse the organic results are, the more likely a user is to click on the ads instead, and that's how Google makes money.
At the least that's a perverse incentive, and at worst it's a corrupting influence.
Whether it is conscious or not, Google does better financially when there is more SEO spam in results.
That's why the better Google does financially, the worse the search results are getting.
I doubt this very much myself too! I little wishful thinking to be honest.
However you're arguing as if new competitors had to monetize the same way Google does and that's not the only game available. Imagine a competitor that doesn't monetize clicks but works on donations (like wikipedia does). If results are higher quality than Google's the users will follow. Or maybe a search engine that promotes a particular product like a CRM tool for example.
Disruption typically doesn't come in obvious ways or else someone else would have done it (including Google and its gazillion dollars).
Thank you. I appreciate that very much. We plan to share on HN how we've built it and how it works, and get as much feedback as we can. That really needs its own thread, and I'm conscious of not getting off-topic or promotional here because this is such an interesting thread already, and it wasn't my intention to distract from that because this is one of the big problems in the world right now. Plus, to be honest, we have a few things that are broken and that we're working to fix! So to answer your question, yes! We hear the feedback asking us to share more, and we're going to get a new release up and share with HN soon. And sincerely, thanks for the encouragement to do that. We think it will be interesting to the community here.
Hey thank you. Seriously I appreciate the feedback and encouragement to share Andi with HN. I know there have been a few comments and questions, even though I'm trying to not get off-topic because this is such an interesting thread. You're absolutely right that this needs it's own thread where we can answer questions properly.
So we're definitely planning to share what we're building with the community here soon. Andi's very much an alpha and a few things are broken, but we're working on a new release that fixes them, and then we're going to share it and learn as much as we can from the feedback and wisdom here.
use the "go" command (e.g. go reddit)
close the newly opened tab (e.g. reddit)
click "Learn more about me"
click the top left icon (Andi)
=> goes directly to reddit.com
Thank you! And ugh! I thought we fixed that bug. On it :)
One of the secrets with Google is search is that the top searches aren't searches at all. Number one search on google? "facebook" - it's people navigating. So the idea behind that is to let people navigate directly when they want to go somewhere specific, like "go youtube cute cats".
Without wanting to go off topic here, may I ask what sort of error did you get? All looks fine from here, although it is an alpha. If you were happy to reach out I can try to help figure it out with you.
Secure Connection Failed: An error occurred during a connection to andisearch.com. PR_CONNECT_RESET_ERROR. Also Firefox is flagging it as "Not Secure" in the address bar.
Interestingly, once I turned off my VPN, it works again, Firefox doesn't display the message either. Note that other web pages work fine on the VPN.
update: It work's with MozillaVPN, but not with my University's VPN. Perhaps UC Davis has flagged it from within their firewall.
Oh awesome, thank you! Yes just say "/bug" any time and tick the little box to attach details (we don't see what searches are so that let's us know) or just hop on our Discord from Contact Us. We really appreciate you trying it out. It is an alpha and we have lots of work to do, especially performance. I've been trying not to go off topic from the conversation here but wanted to answer your direct question, but we can chat more directly.
I love the HN view option...mostly because I find that form so easy to scan.
I've conducted a number of professional searches about neuroscience, obscure R packages, topics on D&D and Pathfinder, and it has done exceptionally well. The one thing my obscure demographic would love is a replacement for Google Scholar, which I use all the time.
I love the HN view too - it is compact, efficient and information dense. I think it works well for search results. And I love HN so it seemed like a neat homage :)
Thank you for trying it out and the great feedback too. There hasn't been too much discussion here about Google Scholar, but better research and academic results is something I hear a lot of people talk about as a growing problem.
Very cool effort but just the first thing I tried which is a mega common search for everyone. "Denver weather."
Took a while to load, and only gave me short text of current conditions.
When I did "denver weather forecast" it gave a bit more and showed results on the right from Wolfram and others but they don't tell me anything. Wolfram is current conditions and wind tomorrow. Next one just meta description for weather underground.
It's going to snow tomorrow. That's what people are searching for.
Google gets this stuff and makes that info super visually accessible.
I really don't get the hate on their search quality. If it has gone down I haven't experienced this outside of mega spam things like recipes.
But I still experience the black magic of typing in some vague thing you are trying to remember and somehow they know what it is.
I've been trying not to be off-topic or self-promotional on this thread because it has so much other great discussion, and it needs a separate thread, but I wanted to answer your question quickly. Thanks sincerely for the feedback and for trying it out. We'd really love to chat more with you about this if you're open to it. The project is still an alpha and we have lots of work to do. We outline some of the good and bad things on our About page, including that it's weak at local searches (like weather or businesses or localizing to region) and there is too much spam in product reviews and ecommerce especially.
My own feeling is that the spam and ad problem (and therefore actual result quality) on Google is at its worst for categories like finance, health and travel. Commercial promotion has really taken over there. For many people, ads are essentially spam when they take over the entire screen of results (try, say, "best home loans in the US").
There is huge variation though. Some searches are still pure. Thanks again and please feel welcome to reach out to chat more about this!
There's a reason why it seems shocking that Google has been able to balance the ads well enough that people still use it. They haven't! Google has orchestrated a monopoly over search engine distribution that allows them to get away with search results that are dominated by ads and spam, without losing most consumers.
I disagree. Two to three years ago I could get more what I wanted in a complex search once I tuned it properly. So Google had a twenty year run of good and useful searches. Google also worked to strong arm their monopoly, yes. But I claim they still served some quality after that. It's not that unusual for a monopoly built on quality to maintain their quality for a period of time after it achieves that monopoly status - institutional standards die but they can die over time.
> almost no consumer consciously chooses to use Google search anymore...Their monopoly over distribution - not search result quality - is what keeps consumers searching Google
I don't disagree with this as a fact, but I think there are a lot of things that work this way that aren't actually monopolies in the competition-preventing sense. If I wanted to launch a new breakfast cereal, getting my product into grocery stores would be one of the major challenges of starting that business. Competition for shelf space is a core concern of a lot of consumables. This definitely creates a lot of stickiness and barriers, and that comes with its share of downsides, but there are also good reasons that distribution systems work the way they do. Transaction costs are important.
I don't think competition for shelf space is the right analogy here. Perhaps for Apps within the App Store you could argue that. But when there are only two mobile operating systems with meaningful market share, and when they make it impossible to change to a new search engine at all, and the results all come from only two sources (Google or Bing) that's a straight monopoly over distribution.
It's a similar situation with the App Stores also. They are monopolies. We've gone from a world of personal computing where software was a free market with open choices, to a closed and proprietary world where there is only one available source of software.
> We've gone from a world of personal computing where software was a free market with open choices, to a closed and proprietary world where there is only one available source of software.
That’s true but at the same time I think most people are pretty happy with it. HN readers aren’t typical in this regard.
I’ve been writing software as my job since the mid-80’s and it’s only been in the past 4 or 5 years where I realized that I’m finally pretty happy with the tech I use day-to-day.
If I had any complaint it would be that app stores have made software too inexpensive. When I look at something like Procreate which I think cost something like $10, I’m blown away. This can’t be sustainable.
You have a point but shelf space is physically limited. Online real estate is not so limited. In my country there is reasonably healthy competition between supermarkets. Supermarkets do have self-branded products but they don't cross-sell competitors self-branded products.
Here we have Apple with Google and Bing on their shelves. Microsoft have Bing and Google on their shelves. And Google have Goggle or Bing. Is that healthy or an oligopoly?
There's a reason web designers call specific pages "valuable real estate".
For example Google's search page, the one with the input, is probably the most valuable web real estate in the world, closely followed by the first page of results once you've typed your query and hit Enter.
I'm willing to bet $100 that the second page of results probably gets less than 1000th the hits the first one gets. Heck, make that 1 millionth of the hits the first results page gets.
That's silly. Everything is limited by the scarcity of human attention spans, not just websites.
Shelf space refers to the market with which someone competes, not whether people are thinking of a candy bar or finding a bathroom or a facebook post. Your argument commits survivor bias because it's ignoring the millions of other websites that exist and are being used. Being popular does not mean something is a monopoly, nor does it mean there's limited shelf space.
Following your example, if Google spammed Pixel ads on it's home page, the page would become less popular. One of the reasons it became so popular was it's strict adherence to focusing on utility.
Even if true, what does that have to do with shelf space? Search results do not constitute the internet, and there are many more search engines than just Google.
And then there's the problem of the difference between your cereal and the big ones aren't going to be big because cereal is a finished art. The same with search. My outcomes using bing or google are almost the same. The reality is a lot of good conversation is locked within social media discussions and reddit is the only one that allows it all to be public by default, hence google + reddit. We're moving to walled gardens and most of them will simply keep google out. Google is probably as good as it can be, but it just doesn't have access to discussions in places like private facebook groups, discord, etc.
Not to mention reddit is terrible outside of tech concerns. It leans conservative, young, male, and white. As a woman, contributing there is an invitation for harassment. Even when I don't contribute I do things like research cars to buy only to see forums dominated by "car guys" who mock safety standards, focus only on performance and the "get laid" aspect of cars, and are dismissive towards groups like "soccer moms." Well, I'm a soccer mom and felt wholly unwelcome in those communities and the advice there is actually terrible advice for parents wanting to buy cars.
Then there's a whole religion on gaming be it consoles vs consoles or vs pcs, or publishers or franchises and just endless tribalism. Politics is an absolute nightmare as you can imagine. The savvy reader will say "well you have to know how to sort the comments a special way and never visit the sub you want but the 'true' version of the sub you want, etc" which is a million times more hostile to non-technical people than scrolling past some google ads and finding an article about what they want from a reputable newpaper or consumer reports.
Reddit is just too wild west to be a google replacement. Worse, the good content is almost impossible to find. Google weighs popular discussions more than recent ones so it keeps giving me discussions from many years ago, even if I try to put the year in the search box. It has no idea how to crawl reddit and make it digestible for us and the reddit leadership want nothing to do with google it seems, for capitalistic reasons of keeping out a potential competitor and having their search be really good "any day now."
Posting to reddit is its own kind of nightmare, full of rules per sub, each different and with an algo that will decide if your question gets any visibility, often only getting mean spirited comments in return, if not harassment. For as far as facebook has fallen, I can still visit my town or neighborhood group and ask people what cars they like and get something of a normal discussion. At reddit, I'll be asked for nudes or just mocked/gatekept.
If anything if google is an old man at web 1.0 then reddit is an old man at web 2.0. I suspect google with outlast reddit as reddit looks primed to be overtaken, a bit like how myspace, slashdot and digg looked like unstoppable juggernauts during their time. Its "manboy" culture and its super hostile default subs and everyday misogyny, transphobia, and racism scare normal people away. Forums have always been the seedy underbelly of the internet but reddit is seemingly proud of being seedy, with spez coming out to say that he welcomes covid disinformation when reddit was recently called out about how its become the home of covid conspiracy theories.
If reddit was publicly traded and you asked me between google and reddit, I'd say buy google 100%. I think we'll be reading a lot more "what happened to reddit" articles in the near future, not "what happened to google" articles. Reddit becomes more toxic over time and that's just advertiser poison. Remember, it happily was the home of "jailbait" subs showing sexualized photos of children and "creepshots" showing non-consensual photos of random women until CNN called them out. If there was no call out, then spez would happily be selling those subs as points of pride and growth. "Reddit is the new google" narratives are very echo-chambery, shortsighted, and highly questionable. I think it forever remains this cesspool that chases away advertisers while Google continues to adapt to a new online world and continues to be vastly profitable. Meanwhile, Reddit has yet to make a profit.
Every time I hear this "Reddit is conservative-biased" I wonder if people are living in the same reality as I am. I just checked /r/all, the first political post is related to the Canadian trucker protest of which support is broadly split down the political divide. Top voted posts there slide as I expected, generally on the left-aligned component against the protests. The same, at least in my experience, applies to all political topics on reddit.
Hell, Bernie, and then Biden were both top of reddit during the elections. In what way is that indicative of a "conservative leaning" on reddit. I'll give you young and male, but conservative? You've got to be pulling my leg here.
/r/all is just a spam of new items, not a view of its culture.
It was the largest Donald Trump fan site in the world (/r/thedonald before it was shut down for being so big its brigading tactics were damaging the site) and now /r/conservative takes that role. Before it was the biggest Ron Paul fan site. Its deeply pro-guns in nearly every comment section and any mention of Hillary or AOC is an invitation for angry comments and downvotes. Random subs are full of transphobic content and misogyny is near everywhere. /r/conspiracy is a right-wing paradise catering to right wing views.
You cannot talk about money or finance without being yelled at about how bad fiat and the fed are, which are right-wing talking points. You can't have a covid discussion without an army of covid skeptics yelling at everyone. You can't discuss police brutality without dishonest "but both sides" types full of racist dog whistles.
Its absolutely right leaning and token "liberal" positions like legal pot or better healthcare doesn't really change that. I know GOP voters with those positions but they always vote GOP for culture-war related reasons. Not to mention, being able to do drugs in your home is really neither liberal or conservative, its just in the USA only the liberal party is making any effort to make that happen.
Covid skepticism, which spez defends, is extremely coded right-wing, so even leadership acknowledges who reddit users really are.
The few liberal and feminist spaces that exist on reddit have draconian mod rules because of the constant brigading and some of them just give up and lock discussions because mods are exhausted and tired of fighting it. Just running a liberal or feminist or queer sub is a lot of work because reddit conservatives are constantly bridaging. This is not the sign of a liberal community. I mod a few subs and its just a nightmare out there. Just keeping conservatives from trolling and fighting with everyone is a big job. If you see "liberalism" its because the mods are keeping the everyday redditors away and trying to keep up the values and themes of that specific sub and its relatively tiny audience compared to the conservative majority.
Cherry picking AI spammed Bernie posts doesn't change the culture in the comments or what visitors receive when they post. I'm in a lot of Democratic, liberal, feminist, queer, socialist, etc spaces and I can guarantee you reddit is absolutely nothing like those spaces.
Also you are taking my comment out of context, not only is reddit conservative, but as a hypothetic competitor to google its extremely conservative. I can google for things without being hit with Ron/Rand Paul narratives, Trump worship, pro-gun narratives, covid skepticism, transphobia, or racist dog whistles. So just the idea that Reddit is as welcoming as a Google search is very, very questionable. Google will transparently present you ads. Reddit will drown you in propaganda, culture war politics, and harassment. As the meme says, we are not the same.
/r/all sorted by hot is items roughly sorted by popularity. Only if you sort by new, would it be a flood of contents by time.
We can take a quick survey of /r/all sorted by "hot" as of this moment. The very first post I see with AoC is literally this:
"""AOC tells Joe Biden: Cancel more student loans to have "any chance" in 2022"""
The top posts? All in support. Where is the dominant conservative force that warps conversation around it.
HermanCainAward, literally a subreddit celebrating the deaths of the unvaccinated tops all regularly. How the hell is that "covid skeptic right dominant"?
Is your idea of "reddit is conservative" that reddit hosts conservative communities at all? Even if they're smaller than the left-aligned communities there?
EDIT:
Some other quick statistics:
Size of /r/conservative: 923k
Size of /r/lgbt: 864k
Size of /r/hermancainaward: 500k
The "lol antivaxxers died" community is literally half the size of the largest conservative subreddit. The LGBT subreddit is the same size as OP's bugbear /r/conservative. The very much Dem supporting /r/politics is an order of magnitude larger than either of those, at 8 MILLION subscribers.
Yeah, I was waiting for this comment. The OP you're responding to is way way WAY overexagerating the opinions of the body politic of reddit.
Reddit is so large that having a potent conservative force is an inevitability - and much as we don't want to admit, for all the nastyness that goes on there, it's still a space that forces conservatives to be relatively speaking on "good behavior" (imagine if gab/voat/parler successfully displaced reddit), and more recently also forces their ideology to be heavily moderated (as in, to become more moderate), and watered down.
Also LOL at criticism of the car community for being bad at recommending SUVs. Reddits car enthusiast community is made up of people whose favorite car by and large is the Miata. Is the Miata a good car for a soccer mom? No. Is the Miata the car of dudebro conservatives? Uh, HELL NO!
I hate to say it, but sorry Karen, Reddit is for your son, not you.
My suspicion is that any broadly successful community aggregator will have enough people holding belligerent, incompatible views, that one could never feel truly safe participating there.
Your loathing (for lack of a better term) of the discourse on Reddit has me wondering if the differences I perceive between our political and cultural tolerance stems from being on opposite sides of a few gnostic / agnostic boundaries, or if the difference between our demographic pigeon holes has spared me orders of magnitude of relative grief, allowing me the additional advantage of getting less overwhelmed by ambient asshattery.
I will readily admit that the latter could easily lead to the former; I guess I wonder if the 2 —> 1 causal order holds overwhelmingly in practice, and how to satisfactorily determine that in a way that manages to be honest, systematic, and compassionate.
>My suspicion is that any broadly successful community aggregator will have enough people holding belligerent, incompatible views, that one could never feel truly safe participating there.
I'm of the opinion that this is fine. Rather, the better approach here might be instead gatekeeping and exclusivity. Now, before you tear my head off for that, let me elaborate a bit.
Much of the issues of existing communities, seem to me to stem from an inability to deal with scale. A single heretic isn't a problem, but when the heretics outnumber the believers, they can then proceed to dominate the community. One potential solution I've wondered about is entry-restrictions and finer grained restrictions on permissions for a community. For instance, a community might be public view but member-only for posts/comments, with invite-only membership. Or perhaps memberships have to be approved by N randomly selected members.
The catch of course, is that what I propose is also yet another contributor to the death of the open internet, much like discord is doing.
Reddit as a whole is absolutely a community and the default subs and the comments in them reflect that. Its also one that brigades a lot into other subs, so there's a "real reddit" you see and its absolutely right-leaning.
I've changed my default search engine to Bing for a while. Before I did that, I did compare the results with Google search and found that the clickbait websites that keep pissing me off are shown only by Google search. Those content farm sites have been on Google search as the top results for years to the extent I think it literally cancels any advantage Google search provides.
“…almost no consumer consciously chooses to use Google search anymore”? 6-8 quality leads in the last 14 days (ave. sale at $3,200) on less than $220 spent on ads begs to differ with you. We’ve only started advertising the last two weeks. We’ve had calls and form submissions _all_ from Google and we only launched our site roughly 45 days ago. I’m not a Google fanboy and I think Search does need an overhaul but people are mostly definitely using Google Search. Another client of mine gets 8-12 new customers per month all from Google searches and she doesn’t spend a dime on Google.
In many ways I think that supports my comment. People use Google search because it's the default and a monopoly, and it has a total monopoly on search ads as a result. But that is not a choice that consumers consciously make to go and use it. It's pre-installed as the only easily accessible default on their phones and computers, and no one ever thinks about what search engine they use or has chosen to use it. If you buy an Apple or Android phone, your search engine is Google, and you just assume it is the only search engine. It's great that it gets good results for your ad spend for you. That's why Google continues to set new records for revenue. Advertisers like you are their true customers. And the people searching are the product being sold.
They were not saying "nobody uses Google search" but rather that people were not consciously choosing to use it over other services, but using it because it is the default on virtually every device and browser, despite the fact that the majority of the results were ads. The fact that people are clicking on your ads doesn't exactly disprove that hypothesis.
Is there a reason why you've chosen the chat style interface vs the standard search box at the top and results at the bottom layout?
This is not a comment on the search results itself - always appreciate the efforts to break out of the standard google results and surface other sources, but I found the interface confusing and the previews were also taking up a lot of space. A compact view would be better - or giving the option to turn the previews on / off.
Thanks for trying it out. I don't want to take to discussion too off topic, but if I can try to answer in general terms, I think that it is not just that the search results on Google have been getting worse, the user experience has been too. There has been very little real innovation in how search works in the last 20 years, so it is good to try new approaches.
With different views like compact vs visual, our feeling is that it's good to give people choices. If you get chance on desktop with Andi, try a search and then under Search Results, click "Change View", and try some of the other views. List view gives straight compact text results, and there is a Hacker view that presents results in the same information dense view as HN. That's my favorite. There is even a view Goggles that has a similar format to Google circa 2000 :)
I'd love to chat more with you about this, and it is a great topic for when we share what we're building in its own thread here. Just based on feedback, we have two fairly passionate groups of early users on this topic - some love the conversational interface and others just want it to look like Google. So the approach we're taking is to give people choice.
"almost no consumer consciously chooses to use Google search anymore"
May I ask how you arrived at this observation? This is the first time I am hearing this. I know of NO ONE who uses any other search engine. The term "Googled" is not yet a proxy for other search sites.
People use Google because it's what comes installed on their phones and computers when they first turn them on, and they never actively choose it. So while everyone uses it, few consumers make an active choice to use it. From talking to users a lot, many just assume it's the only option - as you say, Google has become synonymous with search in the same way Xerox became a word for photocopying.
Consumers use Google because it's the default and the only visible option when they turn their phones on. Unfortunately, that's what a monopoly on distribution looks like. People no longer make a choice and don't even realize they have one.
I would think that there'd be an online opportunity for a search engine that only searches humanly curated sites. Those sites would be ones that have quality information rather than spam. Some obvious examples - wikipedia, reddit, hackernews, public domain books, etc.
It's easy to game an algorithm, but hard to game a human - humans know garbage when they see it.
As an aside, whenever I get a prescription, included with it is a dense two page sheet of detailed information about the drug. I see nothing like that online with a search. Why is this sort of thing not online?
Maybe Yahoo's time has come again! Maybe Google's decline started when they no longer had competition from Yahoo?
The interesting thing would be coming up with a sustainable business model for it. One way might be the users pay for it, either per-search or per-month. This way the incentives to provide good search results align with the interests of the people doing the searching, not the people being searched.
The people who want to be searched would have an incentive to make a quality site that the search service would believe would please their customers.
I can think of people willing to pay for quality searches - professionals looking for things they need, like programmers, lawyers, researchers, etc.
>The interesting thing would be coming up with a sustainable business model for it.
Even though I loathe ads, I wouldn't mind one simple, clearly designated ad spawned from keywords in the search. No tracking and no cross site linking. No result promoting, etc...
And yes, I believe that the current iteration of the web requires human moderation to be usefully searched.
At least France and Belgium have public websites with the information sheets of all authorized drugs. I think at least the French one generally comes up in the first results on Google (when searching from a French connection).
pretty much what we're building at breezethat.com -- currently launching about one topic / week, and opening door soon so others can curate / moderate a topic
Humans can also game systems to promote their garbage if they care to. Spammers can hire a click farm to privilege garbage results. Spammers and scammers seem to see enough returns to invest in ways to game the internet’s openness. There would need to be some kind of trust system to make the curation trustworthy.
I’m not sure what you’re talking about wrt prescriptions. I just googled “diazepam” and got all the information on my pharmacy insert, and more. And often, more clearly presented.
Just tried andisearch and am extremely impressed. It has so far handled all queries I have thrown at it better than brave search and DDG. Will continue to experiment, best of luck and awesome work!
Thank you! I really do sincerely appreciate that. If you'd like to, please reach out to us there because we'd love to chat more. It wasn't my intention to take the discussion off-topic here because this thread is so incredibly interesting, but at the same time it's super encouraging to see the unprompted feedback and questions, and we'll share what we're doing properly on HN soon so we can address questions properly.
Firefox on Linux Mint was pointing at something else for a while (DDG I think? Bing? I don't recall).
I gave up after a few weeks and had to switch it back to Google. Google's not perfect - it's never been perfect, it was just better than the alternatives - but it's still less bad than others.
Distribution will come if the product is better, but it is a hard problem. I try every new search engine I can and they are always worse/slower than google.
I have tried using DuckDuckGo as my default search engine, but Firefox changed it back to Google with every update, so eventually I just gave up on that endeavor.
That's really strange, I've got it set as the default in Firefox on 3 different computers (2 mac 1 windows) and it's stayed the default over several updates. I think something might be wrong with your computer?
With the current Mozilla leadership it never even occurred to me that it could be a bug. I just assumed that it was something they do on purpose to get more money from Google.
>Let's be blunt here - almost no consumer consciously chooses to use Google search anymore
Do you have anything substantive to support this? I highly doubt it is true given the fact that the verb "to google" literally means "to search the internet".
I think you missed the point here -- people synonymize googling with searching and therefore aren't choosing to google -- they're choosing to search but ending up using google despite having made no conscious effort to do so (it's just there).
Google is the default search on the vast majority of phones and desktop browsers by default.
People don't change their search engine from something else to Google, because it is already the default search engine on the devices they buy and the web browsers they use.
So people do not make a conscious choice to use Google. The vast majority make no choice at all. Google is synonymous with search because it is already the search engine on their phones and computers. They are simply never asked which search engine they want to use.
Most consumers have no idea that you can even change your search engine. After talking with hundreds of users, they find it's either impossible to change (iPhone/MacOS) or too hard (Chrome).
If you're Duck Duck Go or Bing, at least you're in a very limited dropdown list if someone does want to try something else. If you're a new search engine startup, you're not an option at all.
Your argument supports the original poster. It is no longer a conscious choice, "Do I search for this via Google? Maybe I should use Bing? What about DuckDuckGo?", it is, "Oh, lemme Google that".
the other day, on HN i mentioned i was trying to find some relevant meme on DDG, and someone said "try googling 'foo bar baz'" and i thought it was funny.
I don't use google search if i can avoid it. I'll try 3 others first, and generally just give up. Google doesn't deserve any money.
> It is in all practical senses impossible for an iPhone or Mac user to change their search engine to a new search engine on Safari or at the iOS level.
There are five (very simply accessible) different choices for Safari on iOS.
But if you switch to iCabMobile on iOS there are TWENTY-FIVE search engines to choose from.
I think it's reasonable to point out that is not something most consumers are going to be able to do. The only meaningful search engine choice is that available within Safari. And you did install another App, they still aren't used from the system search on iOS, or from Safari itself.
I think you might as well be asking regular consumers to root their device so they can use whatever Apps from outside the App Store, or whatever search engine they want.
Also, even for a technical user, there is simply no way on an iPhone to change to a new search engine not already on a tiny list, and from talking with hundreds of consumers, I have not talked with a single non-technical person who could work out themselves how to change their Safari search engine to even one of the 5 limited choices, let alone a new option.
Unfortunately, installing an App doesn't let you change the system-wide search on iOS (or Safari browser), so rooting the device would be the only real way. My intended point is that if you're a consumer trying to change your search engine to a new option on an iPhone, there is no way to do it.
> This is the most annoying behavior because I really mean what I write.
Tons of people don't, though. They type whatever unprocessed half-second thought they have into Google and expect Google to lead them to the water, even if they're tugging and trying to go in the completely wrong direction. Google has optimized for working 'most of the time' for 'the most people', and that means striving for fixing the complete word soup of search results people type in.
A single mediocre experience optimized to work ‘most of the time’ for ‘most people’ is quite contrary to the narrative that has made Google such tremendous amounts of money (“let us surveil you so that you can have a more personalized experience”) though, isn’t it?
Given all of the data collected about Google users, ought not one of the applications of that data be some way to give users specifically what they are searching for if their past behavior suggests that they mean what they type? Couldn’t the “search only for <exact query>“ option be a very good data point on making that determination automatically, or enabling a user setting for “give me exact results based on what I actually typed by default”?
It seems possible to me that this behavior has more to do with the value of ads for “big” keywords than with (poorly) inferring user intent.
I have a sense that this is the dirty little secret of the spyware advertising industry, personalization just isn't that great. Yeah, putting you into a male or female bucket, parent or child, homeowner or renter, that's worth a little bit. But, to find out your name and address and search history and how long your last bowel movement took, just to deliver an ad that's theoretically hyper-optimized to make you buy something... I just don't believe it.
I don't believe that it's worth anything near what they are charging for it, except perhaps in the case of politics, which has always been an extremely efficient use of money. And even then, it's not worth a tiny fraction of the real cost it has to society.
Netflix has achieved the dream of movie studios going back more than a century now. They have the talent, the money, and more than two decades of data. Netflix knows what you watch, when you stop watching, how often you watch, which movie covers work best.
And yet, it's hard to look at Netflix as anything more than a total failure of the promises of data analytics and personalization. Netflix should be putting out nothing but hits. A dozen Breaking Bads or Game of Thrones.
Yet they are not. In fact, they do not even have a single show that is to the level of Mad Men, Breaking Bad, or The Wire. HBO and AMC are running laps around Netflix. Meanwhile, Netflix is making live action Cowboy Bebop and cancelling it before people even know it existed. I'm really curious what the data said about funding that particular project. On one hand, you have the cult following of the anime that will absolutely tear a live action version to shreds. On the other hand, you have to convince the uninitiated into viewing a remake of 23 year old anime.
Then there is the personalization. The fact that there is a meme about spending more time browsing the Netflix catalog than watching content tells you everything you need to know about how little people trust Netflix recommendations. Their new "top 10" feature is just depressing most of the time. It looks like a list of ten random DVDs in the bargain bin near the Walmart checkout line. Oh, and, their top 10 feature is currently the biggest recommendation feature on their site. And it's not even personalized! If that's not a complete admission of defeat I don't know what is.
Netflix has produced a lot of fairly solid content. Not at the level of the all-time great prestige TV shows like the ones you mentioned, but enough to keep a lot of subscribers happy for long periods of time at an accessible price. House of Cards (at least until the Kevin Spacey scandal blew up), Stranger Things, Orange is the New Black, BoJack Horseman, Disenchantment, etc are a few that come to mind that I watched and enjoyed.
I think Netflix has two big issues. The first is the way they drop seasons all at once prevents the natural cycle of pre-episode hype, post-episode interviews, speculation and leaks, fan anticipation, fan arguments (ship wars, etc), etc. Fan culture can't develop around this content as easily because there's never any breathing space for fans to collectively sit with the story so far. The shows that become a cultural force like Breaking Bad or Game of Thrones need us to keep coming back to the conversation every week. They have us talking about what happened last week and will happen this week with our colleagues at work and around the dinner table. How can these Netflix IPs enter stable orbit in the cultural zeitgeist when they're once-a-year events? It devalues the work, positioning it more like a movie that you watch and then forget rather than a story you become invested in over a long period of time.
Full season drops also allow people to binge a whole season of content for a single month's subscription and then immediately churn (ask me how I know). I assume they have some data-driven reasons for doing this, but it makes absolutely no sense to me.
Besides that, I think they should put more wood behind fewer arrows. They've developed a reputation for aggressively cancelling smaller shows with passionate followings which makes a lot of people not even want to bother until something has become an established mainstream success. I think of them now as the Google of content-creation, putting out a lot of solid (but not amazing) products and then cancelling them once people start to grow attached.
Wrote something similar before:
"The writing has been on the wall for some time:
1. Grading system changed from 1-5 to 1-2 (thumbs up/down). They thought that the users where full of crap when rating. I do believe some bosses just looked "bad" when buying in the next Adam Sandler movie. This started a cozy culture where no one in Netflix was wrong. Recommendation engine becomes comically bad, even with the best and the brightest.
2. They started to buy everything under the sun. South park made an episode about it even. All the comedians got their own stand up specials. It was now way easier to get a top score (thumbs up). Bosses where happy.
3. As they no longer focuses on quality which they no longer can measure (measuring time watched and churn is not that useful!), they start to strive for quantity. Which is expensive, very expensive.
I guess that in the next decade Netflix will become the next Comcast and cost 35 USD per month, and it all started in an innocent change to the grading system."
That's not how the entertainment business works. If I had to guess, Netflix's data-driven approach to content production is like card-counting in blackjack. It only gives them a slight edge on the house. A net positive outcome over hundreds or thousands of hands but offering no guarantee over the outcome of any single hand.
> HBO and AMC are running laps around Netflix
Netflix did win the most Emmys in 2021.[1] And they only started producing original content in 2013 or so. That's pretty good.
> Then there is the personalization. The fact that there is a meme about spending more time browsing the Netflix catalog than watching content tells you everything you need to know about how little people trust Netflix recommendations. Their new "top 10" feature is just depressing most of the time. It looks like a list of ten random DVDs in the bargain bin near the Walmart checkout line.
They may also be optimizing for revenues as opposed to recommendation quality (homegrown content being cheaper than licensed)
Part of this might just be account switching issues though. If I’m watching for myself I can usually find what I want quickly. The problem comes when I’m trying to browse with my wife to find something we can both live with. At best it gives the my preferences and the Union of me + wife’s preferences (and vice versa on her account.) But what we’d really want is a separate recommendation feed that shows the intercept of me + wife’s preferences.
But most people are lazy and won’t account switch for different contexts like that anyway, so there’s just no way they can keep the profile data as clean as it needs to be for a television.
Netflix is not recommending what they think you'll like, they are recommending what they want you to watch. Once you're a subscriber, they want to keep you there as cheaply as possible.
This is exactly what their data analytics has told them to do.
Have a few popular, quality tv shows with star-studded casts as loss-leaders to bring in new viewers. Otherwise the model is to produce and recommend the shows that get them the most eyeballs per dollar; the bare-minimum to keep their subscribers there:
- Stand up specials are dirt-cheap, quite popular, and provide never-ending variety.
- Ditto with 'reality' shows, bake-offs, make-offs, expose documentaries, etc.
- Old sitcoms and b-movies that have a proven re-watch-rate.
Throw in a handful of first seasons to keep the FOMO up, and you've got a captive audience on the cheap. Maybe one or two will catch on and become the next loss-leaders.
They may not have the quality shows that are 'running laps around' HBO and AMC, but by any of the metrics Netflix cares about they are simply running laps around HBO, AMC and everyone else.
>And yet, it's hard to look at Netflix as anything more than a total failure of the promises of data analytics and personalization. Netflix should be putting out nothing but hits. A dozen Breaking Bads or Game of Thrones.
I don't follow this argument. Knowing what people like has very little to do with the quality of original creative content; surely you don't expect Goodreads to put out Shakespearean novels, or Spotify to be producing original hits on par with the Rolling Stones? Should ESPN have better pro sports scouting and coaching talent than the professional leagues?
Knowing what people like, however, _does_ have to do quite a bit with selling those people a product - which Netflix just reported 15% YoY growth to $7.7B yearly revenue, they're clearly very successful to this end. I think you actually have it backwards - if anything, Netflix represents a total fulfillment of the promises of data analytics and personalization. Despite mediocre original content, this is a $200B company with 200M subscribers growing revenues by double digits two decades after IPO.
If Netflix paid $450k+ salaries to screenwriters instead of engineers, you'd very likely get better movies on a worse streaming platform. And when Netflix has shelled out for Hollywood talent, like Mindhunter which has David Fincher and Charlize Theron, the results are quite good.
Regardless, to take the fact that Netflix pays for premium engineering and analytics talent, but does not pay for premium filmmaking talent, and then spin that fact into Netflix being "a total failure of the promises of data analytics and personalization" is a questionable criticism.
Strong comment. I agree completely with your negative assessment of the “value” of consumer habits to optimize Netflix recommendation. In my own case I feel trapped in a very shallow local minimum. Yes I watched a revenge flick or two but now I am type-cast for life.
Netflix really doesn't need to produce something like the wire or mad men or breaking bad. There's no reason to make a show that appeals strongly to 70% of the market when they could make 70 shows that 1% of the market is fanatical about. They don't have only one channel that competes for content and they don't seem budget limited.
Ironically, it seems like AmazonPrime is far better at that.
As for then top ten being bottom barrel stuff, I think you overestimate how popular mad men was vs. something like king of queens.
I will say, Netflix seems to fail in many cases, and I don't understand how they think content discovery is supposed to work.
Right, people radically overestimate how much a profile is worth. Someone who owns a house in a rich area is somewhat easy to identify, and you target them... along with everyone else who is also trying to reach that rich slice, so you pay more.
The very high quality pieces of information can be things like "wants to buy a life insurance policy this week" or "just had a baby" or "just bought a plane ticket to XYZ," or "is in the frequent flyer program and spends more than $20,000 per year on travel."
However the majority of information about people, the overwhelming majority of whom have no significant disposable income, is worthless and not worth tracking for the most part. You reach those people through traditional mass marketing means.
It's a great point you have made. I work technically as a data scientist, but my domain is scientific data. I have quite a few GitHub packages and get recruiter calls for data science jobs almost every week, with pretty generous salary offers.
And from what it seems to me, there is a giant bubble. The vast majority of companies doing "data science" jobs are things that a smart undergrad can do with a month or two of training. And this is because I believe C-suites have completely gulped down the data is oil mantra. There are entirely charlatan companies with unicorn, even decacorn valuations now being built on this mantra - for example, CRED in India.
Yet, as you said, and as I believe too, most of the data is worthless.
Right, but unlike oil, most data is worthless most of the time.
For example, I'm about to list my house on the market. The real estate agents who reached out to me last year and the year before that to try to induce me to sell the house had no chance of succeeding. Now is the time if they knew the secret that I'm about the list the house that they should all be competing for my business. The only relevant piece of information is that I'm about to sell the house. That is a valuable lead that many in my region would bid on. My background information is frankly not that relevant to the value of that lead, and isn't something that is readily surfaced by the kind of deep profiling that is supposed to be going on. It can be signaled by me signing up to some kind of list that sells my lead to a zillion people at once, but is never going to be surfaced accurately to the people who can earn the most profits from it by my Youtube habits or whatever.
Zillow sells the logged in user data in the market I'm buying in to the real estate agents listing the houses, but that again is not something being modeled by some kind of big data operation, but is merely the same kind of "little data" provided on things like dating websites or LinkedIn when people browse your profile. There's no modeling going on there that requires sophistication.
> However the majority of information about people, the overwhelming majority of whom have no significant disposable income, is worthless...
I've had a supposition for a while now that the targeted advertising industry should be closing the consumer cashflow loop by advertising effective self- and employment improvement, with the objective of increasing the disposable income people have so they can then make more brokering traditional sales
>I have a sense that this is the dirty little secret of the spyware advertising industry, personalization just isn't that great.
Personalized adverts and recommendations can be incredibly, horrendously dumb.
Here's what I see when I hit amazon's homepage at the moment :
A "buy once again" column that features blackout curtains I bought 3 months ago (no, curtains don't need to be replaced every months, amazon.), USB cables I bought multiples of in the same time frame, a wireless charger (I already bought two before).
An entire line dedicated to showing me backpacks (I bought one less than a year ago)
An entire line dedicated to headphones (I recently bought wireless IEMs)
An entire line dedicated to watches (same)
I don't get it. Supposedly the best and brightest work at firms like amazon and google to brainwash us to buy stuff, but classic, random, non-targeted advertisement is more likely to make me discover products I'd buy than targeted advertisement because the latter only shows me things after I don't need to buy them anymore!
Here's what I would expect actually intelligent targeted advertising to do :
After buying a smartphone, recommend accessories (cases, screen protectors, USB-C dongles, chargers, whatever)
Here's what targeted advertisement actually does :
show me smartphones ads everywhere I go after I already selected and BOUGHT a smartphone. No, I don't need to buy another smartphone weeks after a recent replacement, amazon!
The same sort of phenomenon can happen after google locks on searches I did to buy something. I can't wait to see the internet advertisement industry crash and burn, it's overvalued nonsense.
A "buy once again" column that features blackout curtains I bought 3 months ago no, curtains don't need to be replaced every months, amazon.),
Disagree there. About 75% of the things I buy on Amazon are repeating purchases that I nevertheless don't want to be automatically scheduled. It used to be a real pain in the neck to reorder something manually, so I'm glad they made that easier.
But yes, in general, Amazon is full of low-hanging fruit that's been neglected on the tree for a decade or more. Buying clothes from Amazon still manages to be a worse experience than going to the mall, for instance, which is really saying something.
I think the fact that most recommendation algorithms have seemingly converged on what seems like a really poor and naive implementation - fixation on very recent activity - shows that the sort of deep personalization touted is mostly BS.
Both YouTube and Amazon heavily personalize by recommending primarily the 3-4 things that I've interacted with in the very recent past.
This is not true. For example every time Summoning Salt uploads a video, which happens every few months, it will show up on my recommend feed because YouTube knows I'm willing to watch their ~1 hour documentaries even though I'm not subscribed to them.
Youtube seems to be a rare exception here in that people actually feel like its algorithm is useful. However, even then, their algorithm mostly seems to devolve to "what creators have you usually watched videos from" and (usually directly after you watch such a video) "what videos did other people who watched that video watch?" Basically the same principle as PageRank, just with a lot less spam to deal with.
This could (probably isn't) be a very quick implementation with a heuristic like 'if percentage of viewed videos from channel x (essentially per channel viewed) > threshold ==> show new video from channel x on homepage next time user appears.
Make it fancy and use a multi armed bandit and call it machine learning/AI/data science.
The most common pattern I see relating to personalized advertising as someone being advertised to is that I will often see an ad for something I just bought (or some competitor to it) repeated relentlessly for a couple of days after buying it and this is after not seeing any related ads during the days prior where I was actually doing some research into the product space.
Maybe I'm an outlier but they seem to miss the window of relevance on me often enough that I notice it as a commonly repeated pattern.
I know it seems moronic, but I think it might actually make sense from the advertisers point of view. Some percentage of people who buy a thing are going to return it and buy something similar in the next week. That percentage is almost certainly large compared to the percentage of the overall population who's going to buy that thing in the next week, and it seems plausible to me it's even large compared to the number of people who have been browsing for the thing but haven't bought yet. (Think of it as the ratio of people just browsing vs ready to buy.)
Even so, wouldn't it be much smarter if they kept track of what the expected life expectancy of the thing you bought is, and then years later start feeding you ads for a replacement? Or is it too hard to track people over such a long period of time?
I don't think any advertisers would be able to offer "People who bought a washing machine 3 years ago" as a category that can be targeted without a riot
Same experience. What's even more mystifying is that often it is for items that no human would be likely to be buying many copies of in a short span of time (high ticket items, or items where you probably don't need more than one).
Just because I bought something doesn't mean I kept it. And those 0.1%, or whatever, returning items are very likely to buy another one of a different brand.
Exactly. If I just bought some power tool for a home improvement project, I am the least likely person in the country to want to buy that exact same power tool the next day.
Not if you hate it and want to return it. In fact there’s a calculation to be made - what percentage of people return or dislike their drill? Because that subset of the population is probably more likely to be looking to buy one than any other.
A return rate of say 1% may lead to more people looking to buy a drill who have just bought one in the last week than people looking to buy their first drill.
> In fact there’s a calculation to be made - what percentage of people return or dislike their drill?
If that's true, they left something out of their calculation: What percentage of people will install an adblocker as a result of feeling like they're being hounded for a few weeks? This scenario was mentioned specifically by Tim Cook when he introduced the Safari anti-tracking features.
I haven't consumed significant amounts of ads in a long time, only some logos in sports and the occasional visit to family or the rare times adblock fails (YouTube premium user too). So I can only imagine how hilarious that must be.
I think you're right. I'd like to see an analysis of the effectiveness of personalised advertising based on tracking versus ads based purely on local context. The latter being if you're on a web page about birds then you get ads for bird seed and bird houses. No tracking involved.
It's always fun watching an ad system try to figure out nonbinary people. Spotify ads can't decide whether I'm a successful businessman or Spanish-speaking housewife.
It's always fun checking Google's ad settings and seeing what they think I'm into.
Apparently now I'm into baseball, flowers, boating, celebrities, country music, credit cards, geology, event ticket sales, fishing, and windows OS. Among a couple hundred other things. It even gets some rather basic facts (marital status, company size, education) wrong. I seriously wonder how they generate this profile?
Well... It actually got some categories right, but I don't really feel that's very impressive considering that it put me in every category by the looks of it.
It fascinates me to see how the ad algorithm responds to people who watch content in multiple languages. I study a lot of languages as a hobby, so I often watch YouTube videos that are in Mandarin, like news broadcasts and niche hobby channels. YouTube has now started showing me ads (in Mandarin) which seem to be targeted to Mandarin-speaking immigrant parents of young children who want a way to teach them Mandarin despite my, and my spouse’s, very busy careers. I find this amusing because I am a single, pasty white man in my 20s.
That’s just Spotify. Many years ago they had a little tool that actually reported what demographic slots it pegged you at based on your listening preferences. The top two hits for me were 1.) early 20s, college educated, White, woman 2.) 60+, blue collar, African American, male
At the time I was a late 20s, college educated, South Asian male. I’m very cis and very straight. And yeah my musical tastes are pretty eclectic, but that was a weird profile to settle me on.
Some people fit in convenient buckets, but lots of people don't, and assuming all people do, will make the ad system useless to a lot of people. Even if you're not non-binary at all, you could still be a successful businesswoman or a Spanish-speaking houseman (househusband? stay-at-home dad?).
Better to just follow people's interests, instead of using their interests to incorrectly pigeonhole them and then drawing incorrect generalisations from that.
Something about my actual interests and activity apparently makes youtube think I'm into Fox news and all the crazy shit found there.
Now, who else has this same value judgement about me? This assessment that I neither declared for myself nor even ratified.
It's annoying but ultimately harmless that youtube shows me conservative wackjob stuff.
But is that same profile in someone else's database that marks me as someone to watch or something? Does it affect my insurance rates, my liklihood to get extra scrutiny when travelling, my ability to purchase or register a firearm, my access to jobs that might be extra sensitive or responsible, basically any of the things where someone either private or the state does any sort of background or credit check on you for any reason, and there are really many of those when you think a out it.
I'm guessing, today, it's probably not really affecting my life in any real way, but, there is no way it makes any sense to say that will still be true tomorrow.
There was that infamous case of a retailer figuring out someone was pregnant before they did based on what they were buying and mailing a customized flyer...to their dad's house. I don't remember the exact situation, but it probably wasn't the only incident.
That btw is an anecdote from the association mining community.
I spent a lot of time learning about association rule mining in my AI courses, including the implementation details of competing ways to mine them. The technique seems extremely useful and fascinating (I jury rigged it for on the fly league of legends champ recommendations to maximize calculated win rate change given limited information), but I almost never see it used in the real world or even see it talked about anymore.
I believe you're referring to a rumor (which may be true, I just mean it in the sense that it's out there and not something you or I have verified) about Target.
My experience working in a similar domain (NLP summarization, which leverages methods like text rank which are identical to pagerank but for text summation) is similar.
Personalized page rank is not significantly better at summarization in my experience, even "queryable" summarization, but that also could be a pure implementation problem or a problem of hyperparamater selection...
Does it really work so well in politics? I've read in various places that a lot of political advertising in America functions basically as a means for channeling donors' money to a few K Street firms belonging to party insiders.
I agree with you. I highly doubt that our economy has enough (product, message) combinations to justify the need for personalization based on more than a dozen attributes.
I will buy X, if I need X. And once I buy X, it's done. For example, I wanted a cordless drill last week. Did the "site:reddit.com" thing (I actually have been doing that almost subconsciously now, as Google results are all trash), chose a drill, and ordered one off Amazon.
Then, after that, what's the point in showing drill ads to me for two weeks?
There's a well known effect in advertising that advertising a product to a person that has already bought that product generally increases their satisfaction with the product and the purchase, and may cause them to recommend the product to others.
Probably that's what they are going for if they're doing it on purpose.
> There's a well known effect in advertising that advertising a product to a person that has already bought that product generally increases their satisfaction with the product and the purchase, and may cause them to recommend the product to others.
Do you have a link for further reading on that? That's fascinating if true.
It's not supposed to feel good. If 9/10 people have a brief negative thought about the advertising experience and nothing else happens, but 1/10 people happen to have their friend on the phone at the time and makes a referral, then overall that is a win for the brand.
It's pretty straightforward to understand that when the vast majority of your sites income is generated from ad revenue, that data is being used to optimize for generating ad clicks, etc., rather than actually giving users the best/most relevant/useful/desireable information for their purposes.
From the behavior of ads (that I imagine are highly optimized), all that knowledge is useful for front-running an specific TV model all over your internet once you decide to buy a TV.
It seems to be completely useless for anything else, and specifically harmful for product discovery, that is the one way ads add societal value.
I worked for a healthcare recruitment company in a capital city with some large hospitals and a number of universities. I can't for the life of me understand why they chose to spend so much money on trying to track healthcare professionals online when they could just advertise it on-premise where they actually hang out.
I can't understand why every company want me offer “personalized” service. It never works, and if I can't manually set preferences, then it's not personalized (because “personalization” means exactly that).
Given all of the data collected about Google users, ought not one of the applications of that data be some way to give users specifically what they are searching for...
You're missing what "personalization" has come to really mean. It means knowing enough about the user to give them an experience you can profit from and which they will accept. If there isn't something you can expect profit from, there's no reason to give them anything.
This used to be solved by allowing queries like `Class Inheritance +ruby' to require results to include "ruby". They killed this for Google+ by changing it to quotes, so `Class Inheritance "ruby"' but now they interpret even those. When I use Google, which is less and less, I am not looking for a fight with a computer to express my intent, I'm looking for the answer to a question. That never seemed to be an issue until recently.
I work for Google Search. If you put a word or a phrase in quotes, we will only find things that have that exact word or phrase. Nothing has changed in this. When it happens that people feel it fails, it's often that they don't realize we've matched that word or phrase appearing in ALT text or text that's appearing in a less visible part of the page -- or in a few cases, the page might have changed since we indexed it.
> If you put a word or a phrase in quotes, we will only find things that have that exact word or phrase.
I'm sorry to tell you, but this is flat wrong. I commented[1] about this a few months back with a random phrase as an example. I see it often in my day to day also.
I searched for "eggzackly this" and all 10 results on the first page contain the phrase, although most have punctuation in the middle.
Looking at all 22 results (without opening the "omitted results" section or image results), the phrase became harder to find off the first page, but I tended to find it in the source code, or the DOM, or the cached version of the page, or by disabling JS (sometimes requiring a combination of those techniques). I found it in 21 out of the 22. The one page I couldn't find it in (the dolls one) looks like a frequently updating page, and it was highlighted in the snippet, so it looks like it was there at the time of crawling but not now; the cache isn't there for that page.
Full disclosure I work at Google, but not on Search.
Punctuation matters. I explained this in another response, and I should have mentioned it as part of my response here. But to repeat, these are typical reasons why it might seem that quoted search isn't matching when it is:
1) text appears in ALT text
2) text is not readily visible on a page (maybe in a menu bar or small text)
3) there's punctuation ("dog cat" will match "dog, cat"
4) page has changed after we've indexed it (so view the cached copy, if available)
In the [eggzackly this], you found matches of those words separated by punctuation -- which we interpret as a space, so the phrase is matched.
I wish we'd fix the situation with punctuation. I get that's confusing. But that's not a new change; quotes have operated that way for ages.
The article has been updated with a response from Danny Sullivan, the person you're responding to, which is worth a read.
I just tested "eggzackly this" and every result on the first page contains the string "eggzackly this", albeit usually with different punctuation ("Eggzackly. This [...]", "Eggzackly, this.", "Eggzackly! This [...]")
I'm sorry but this has absolutely changed. I'm not sure why but quite often we are suggested results in queries that ignore quotes. The engine is even telling us that if omitted those terms.
We don't have control over this and it's very frustrating.
We haven't changed anything. Promise. Honest. Not at all. But we definitely want to look into any cases where people feel this isn't working, so actual examples (if people are comfortable sharing) will really help.
What you're talking about is probably a case where there's a quoted word or phrase as well as other words that aren't quoted. In such a case, we're going to absolutely look for content that matches the quoted parts. That's a must. The other words, we'll look for them, but we'll also look for related words and sometimes, we might find content that doesn't match one of them.
Because those other words aren't quoted, we'll tell you if we find a match that seems helpful but doesn't contain those non-quoted words. That's what the message is about. But it should never be telling you we omitted a quoted word or phrase because we won't -- with one exception.
If there's literally nothing on the web we know of that matches a quoted word or phrase, then we're not going to show anything at all and say we couldn't match any documents.
I tried to find a counterexample and I couldn't! I believe you that quotes really are working. What's confusing though is that a quoted word or phrase often doesn't show up in the Google results snippets. This is certainly the reason why people think quotes aren't being respected.
Though, why does enabling "Verbatim" (in tools) on a search reduce the number of search results if all my terms are already quoted? Enabling Verbatim often does it make it feels like my queries are interpreted more literally, but if quotes are already being respected I don't understand how Verbatim would reduce the number of search results.
I agree, it would be easier if it were in the snippet. That's something we're looking at. I believe it used to, but sometimes the quoted part might not have been the best overall snippet to use. But as said, we might revisit that.
On the counts -- basically, it's all really rough estimates. We make a rough quick count, you go deeper into the page, we make a fresh estimate. It can change, and it doesn't always make sense and personally, I'd hope we just get rid of counts because of this, perhaps more confusing than helpful.
Interestingly I tried my pet peeve search, and it worked for the first time this year! It is for a specific recipe, and I search for 'ocau slow cooked balsamic beef'. I have had to manually find it in the archive of the overclockers.com.au forum for the past year, as Google seemingly forgot it existed no matter what search terms or operators I used.
The main difference is I am using Firefox on Manjaro and not my historically typical environment for searching. Normally I would either be using Chrome on Windows or Chrome on iOS if I am in the kitchen.
I will play around on some other devices and see what evidence I can find.
P.s. I'm being referred in for a role at Google currently, is the Search team only in a specific area like Silicon Valley or is it a global team? Most of the jobs in Australia seem to be commercial facing, not product facing.
Glad to hear that works! Search has teams around the world. I'd suggest if you see something relevant, apply even if it's in a particular location. Remote work has changed a lot things.
Hey, thanks for much for this comment. I've personally experienced this and looked myself for an example when I first read your comment, but I couldn't find one.
Today I stumbled upon one. Here's a broken example query: linux next hop "[::]"
This is not the case in my experience. I type a query with some parts in quotes and often get lots of results that have in small letters at the bottom something along the lines of “does not include <word in quotes>”, with no in bold highlighted part showing the phrase in the page context. This was not the case in the past and google made sure the word I put in quotes is absolutely mentioned somewhere
I’m guessing this happens when there are less results matching my phrase
I would love if you or anyone who ever has this happen can share an example, if you're comfortable doing so. We'll debug. But if you quote something, we shouldn't show anything but that which matches the quoted material.
Now, if you quote something and put in other non-quoted words, then we'll look for stuff that matches the quoted part and the other things are optional. So when you see that strikeout message, it means basically "We found this page that has the exact words you quoted, and it probably has one or more of the other words or related words you didn't quote, but heads-up, it doesn't have one of those non-quoted words at all."
And we do this because sometimes there might be a useful page that doesn't contain all of your optional non-quoted words.
Totally agree it would help if we did a better job bolding the sections of a page where the quoted terms apply. Often we do, but sometimes the snippeting won't include them if there's better text to describe the page overall. But we're looking at maybe improving here.
Unless it's changed in the past 56 minutes, the SERP is pretty up-front that it can't find any pages with all three of those.
It literally says `No results found for "kitchen boss" "g320" "disassembly".` right at the top, and then shows you the (properly explained) results for the non-quoted version of the query.
Mobile doesn't show that, at all. It also doesn't show that bit if you use the brand name "kitchenboss" instead of "kitchen boss", which I originally intended. My phone might have auto-corrected that without me noticing.
Anyway, with the terms I posted earlier, on desktop it says:
Results for kitchen boss g320 disassembly (without quotes):
It looks like there aren't many great matches for your search
Then it goes on to show me lots of unrelated results, instead of showing me the 'not many' great matches for my search. If my search doesn't turn up many good results (let alone great), that fine. Show me what you've got, let me decide how to broaden my search to get more. All of the unrelated results being shown are marketing spam sites. That's not helpful.
On mobile and desktop, I'm getting a message saying that when you search for ["kitchen boss" "g320" "disassembly"] that we have no results. And there's not much we can do if there are no pages we find that match all those words you required be present. There just aren't the pages.
What we can do is try searching for all of the words, so perhaps you'll find something useful that way. But even we can tell when doing this that the results might not be what you want, which is why that automatic warning about maybe these aren't great matches.
IE: we can't show you what we've got for a query where there are no exact matches. It's impossible. We can show you what we got if we don't require all the word be present. And we can tell you what we're doing. And you always have the choice to restart the query in another way if you don't like that.
Now here's something else. You probably used the quotes because I'm guessing you figured it was better to tell us exactly what to do than trust us to look at all the words and analyze the context and so on and see if we could make matches generally. If you had done that, just searched for [kitchen boss g320 disassembly], then the first web page result is the instruction manual for your sous vide machine. It has cleaning steps, which I'm guessing also might be what you're after? (It looks like it's just take off the outer casing).
Those results, doing it directly like that, are different than when we gave them to you after your quoted search failed. and that's probably because when you gave us quotes, and there were no matches, we might have tried some stricter matching to keep closer to the original requirements rather than use our general ranking.
To wrap up: maybe don't try the quotes at first. It's totally fine to type in a long natural language query like [how do I take apart a kitchen boss g320] and if you do that, the instruction manual is right there.
I added the quotes because I wasn't getting what I wanted without them. I don't need the user manual; I already have that and I'm not just trying to clean it. I wanted to find a guide to fully disassembling it, like the videos you find for phones when you want to replace a cracked screen yourself. Knowing that there are no matches at all is good information. (Disappointing, but good.) Trying to give me results for a different query, apparently assuming that I accidentally used the exact-match-only quotes, is not useful and kind of condescending. I think that's what leads people to think the quotes don't work the way they used to. Finally, the wording about not many good matches makes it sound like there are some good matches, but they've been mixed in with these other irrelevant matches. That's probably just a wording issue on the message, trying to soften the "We can't find anything" result.
Thanks for the additional information. It really helps understanding the situation.
Showing results for non-quoted words isn't intended to be condescending nor an indication that we think someone made a mistake. Apologies that it comes across that way.
We think we're clearly saying there are no matches with the "No results found" message. If we stopped there, the page would have nothing else. And I get that for you -- and perhaps others -- that might be preferable, a further reinforcement that there's nothing out there.
For others, not so helpful. They potentially might give up with no real way of going forward, when just losing the quotes perhaps could get them useful information.
And I get the trade-off concerns. I've seen many comments here that things done to support less "pro" users are annoying. And yet, we do need to find a way to support everyone. I think that's why we've probably gone with the message about no matches with and showing the quotes.
Perhaps we should consider making that an option -- "Would you like to try this search again without quotes."
As for the message about not good matches, we always try to show the best stuff we have first. That's the point of our ranking. The warning is meant to indicate that even though we'll list the best we have, for that query, none of it is particularly helpful -- not that here are a bunch of results, and there's some good matches mixed in with poor ones. Ranking that way would make no sense.
But it might be also that we're so close to it -- that we always try to rank the most useful stuff first -- that we didn't consider the interpretation you had. So thank you again, it's really helpful to get that feedback.
Sorry the information doesn't appear to be out there. I hope if you disassemble it on your own, you'll post the info out to the open web. I'm pretty sure you'd end up ranking well for that and helping others who might have a similar need.
Weird. On a computer, `"kitchenboss" "g320" "disassembly"` returns exactly six results, all of which appear to include the quoted terms. Plus the "It looks like there aren't many great matches for your search" message at the top. Which sounds like exactly what you want. I wonder why it's different on mobile.
So, besides the possibility that you are flat out wrong (as another commentator claims you may be), let's assume that you're right.
1. This is still horrific UX/UI
2. The culture internally at Google seems to have a "we know better than the users" attitude in all things.
3. Query rewriting is a horrific technique in general with almost zero value to life/society outside of fixing spelling errors. Whatever your A/B testing says about it's purported utility is polluted by Google's own dark patterns and political whims of the managers who run the internal search organizations.
It helps to actually know better than your users if you want to take the attitude in number 2. I don't believe that you or Google knows better than it's users, for many reasons previously enumerated in this thread and others.
One day Google search is going to be displaced and it's current utilization of query rewriting techniques will be one of the fundamental reasons for this.
You should take the absolutely massive amount of recent criticism and the fact that users repeatedly claim it's happening in the face of your claim it's not seriously rather than literally blaming the users writ large for a problem that is fundementally with the behavior of Google search.
I routinely see queries with quoted keywords where results don't have them highlighted in the snippets on the results page (but do have other, non-quoted keywords highlighted!).
That doesn't mean the quoted words weren't present in the content. It just means our system didn't think creating a snippet around those words was the most relevant snippet. Which I get, in some cases, actually would be better. It's something we're looking at.
Your system for snippet creation is so bad that according to you it's created a situation where many, many users believe that quotes don't work because of how bad it is.
Please fix it, like now. Displace your teams current sprint priorities, or the anti-google search backlash will turn into a situation where in 2025 Google is on the defensive for search market share from a Phoenix rising yahoo or something like that.
The fact that anyone at Google ever okayed this behavior at all in the first place is simply rage inducing and you should see that with the magnitude and persistence of the "actually it really doesn't work bro" kind of comments.
If a snippet doesn't contain the words I searched for, I don't click it, because I assume Google has fucked up the search again and given me some irrelevant thing. If that is what has actually changed in the last few years making people think results are bad, the snippet algorithm, please revert that. I want to see the exact context that the words I typed appear in the page.
In some cases? I'm literally asking for a specific word to be included no matter what; how could it be irrelevant to the quote used to describe a page?
Snippets we show tend to focus on full-sentences or enough context to describe what a page is about or relevant to a search. If we have a match that's strongest simply because a quoted term appears in ALT text or some obscure menu item, that probably doesn't generate a compelling snippet in how we normally would measure things. But given that for someone doing that quoting, seeing the quoted area might be the most important thing. So the regularly snippet process isn't as helpful -- and it's something we'll look at.
I don't have any recent information on how google search works, but years ago it looked at the expertise level of the searcher. So newbies received newbie results, advanced searchers received advanced results (and more visibility into filtering functionality). Today... they're hiding the advanced features and also seem to be reducing personalization of results to save compute resources. It's horrible.
You: Class Inheritance +ruby
Google: searching for "cash inheritance..."
I work for Google Search -- we never operated like this. We don't know that someone is somehow a "newbie" vs and "advanced" searcher and change (nor did change) the results somehow.
This is very helpful if I search for a name I didn't quite pick up or don't know how to spell, or if I only remember fragments of a quote or topic, then I just blurt out my stream of consciousness and Google will mostly point me in the right direction. That being said, I wish I could explicitly tell Google to treat my query more literally. Ideally you would be able specify the search query in some kind of grammar. They have these kinds of prompt mechanics for GPT3, so I doesn't seem too unrealistic, even if it's all ML nowadays.
I work for Google Search. We have several ways for you to do this. The easies is to put quotes around a word or a phrase that you absolutely, positively want to be present in content retrieved. And yes -- it still works. It really really does, but if you or anyone finds an example where you believe it doesn't, please let me know. We'll debug it. The reasons people sometimes think it's not working is because the text appears in ALT text, or it appears in text that's not readily visible on a page (maybe in a menu bar or small text), or there's punctuation ("dog cat" will match "dog, cat") or sometimes a page has changed after we've indexed it (so view the cached copy, if available). You can also use verbatim mode from the toolbar so that we search for only the exact words you provide.
I don’t think op meant it literally, but the fact that the results are so keyword stuffed that despite “appearing” on the page they are actually irrelevant to the page and thus useless.
Have you tried this? It immediately turns verbatim off. For everyone! It's been like this forever. Years and years. Gaslighting won't help here!
In terms of quotes, again, this does NOT work like +, like verbatim. If it did, then that term would absolutely show up, just as it is, in search results. Yet, over the years I've seen:
- aliasing happen from within quotes (EG, bob -> robert)
- quotes entirely ignored (eg, those terms NOT showing up)
Yet searching with verbatim on, immediately causes those quoted words to appear!
You are absolutely gaslighting people on this! Right now, in this thread. And if it isn't intentional, if you aren't gaslighting, then how can you not even notice that verbatim turns off, the second you select a date range?
I want to say so much more here, but it's filled with such ... vitriol, that I think my terminal would melt.
I'm not trying to gaslight anyone. And what would be the point? To say something works if it demonstrably doesn't work?
Yes, I tried this for some of the presets before I replied. It worked. It still works.
My sincere apologies for not specifically testing custom date range option as well. I should have; you are correct. That won't work. I'll pass it on to see if there's a way it can. My apologies again.
If you need to do this another way, what I also said works. Do it in the search bar using the before/after command. Just quote all the words, and that's the same as verbatim.
It's like you're just spouting Google propaganda, without validating what you're saying. Just like with vertabim/date range, which has been a thing for close ot a decade, which Google has received endless reports about, this too does not do what you say.
Your responses are akin to those canned responses one gets when you post a bad review. "So sorry, please contact us here at this email address.", which of course is all about impression, and results in nothing ever happening.
This has been going on almost a decade, yet oh what, no, haha is your response.
Verbatim mode means we search for exactly the terms you put into the box. Quoting terms means we search for exactly the quoted terms. That's what I meant by them being them same.
So if you did this search in verbatim:
[search for this]
It's the same as:
["search" "for" "this"]
in terms of the instructions we're getting on what to retrieve -- look for content that has all of those words and only those words. No spell check. No synonyms. Just those words.
The ranking of the results might differ, because we probably use slightly different ranking systems when using verbatim versus quoted words. But in either case, the retrieval requirement is the same. Results should have all the required words.
I don't really see what the link above is saying to somehow refute all this. That link is about a quoted search for ["eggzackly this"] and nothing to do with verbatim mode. And it says that it found those two words in that order with punctuation...
Which is what I explained elsewhere in this (now huge) thread. But to give it again:
Typically the reasons people believe quotes are not working when they are is because:
1) text appears in ALT text
2) text is not readily visible on a page (maybe in a menu bar or small text)
3) there's punctuation ("dog cat" will match "dog, cat"
4) page has changed after we've indexed it (so view the cached copy, if available)
In the [eggzackly this] example, that's what was happening as the poster saw -- we found those words separated by punctuation, which we interpret as a space, so the phrase is matched.
Personally, I wish we'd fix the situation with punctuation. I get that's confusing. But that's not a new change; quotes have operated that way for ages.
Most important -- quotes SHOULD work as you and others are expecting. We WANT them to work that way. That's why we spend time looking at these reports saying they're not. I have spent lots of time doing just that. We find the matches. But if anyone believes they aren't working, and the reasons involved above aren't happening, let me know. We'll get on it. We want them to work as expected, and we want everyone to feel they're working that way.
For additional clarity, quoting has always, always, always been different than +, and verbatim. When you(Google) removed +, so that 'google+ searches' could work without interference, quoting was already a thing, and people were just told by some airhead googler "Oh, but quoting is the same! Just use that!"
It wasn't. It isn't. I never has been. Ever.
Verbatim was introduced to replicate that lost + functionality, after massive outrage at the inability to find search results. The fact that you, and other Googlers still think "" is the same as verbatim/+, when it doesn't even show the same search results, is highly, highly questionable.
To be beyond blunt, you're wrong. You are completely and totally wrong. +/verbatim and "" are not the same thing.
Please go away, and learn how your own product works, before commenting on it, ok?
At the time, + was used to require that something be present. Quotes were used to require words appear in a particular order. You could do a search where you quoted a phrase, but that didn't necessarily require it to be present (as I recall). If you absolutely wanted the quoted phrase to be there, you had to quote and put a + in front of the quoted phrase. So yes, they were different things.
When + was eliminated, quotes took its place. Quoting a single word was the same as when you used to + a word -- find the exact word. Quoting a phrase still meant find the phrase, but that also meant it was required to find the phrase.
With +, then with quotes and with verbatim, it's about what you retrieve. Verbatim says get these words or words and only those words. Quoting says get these words and only those words (and only those words in a particular order, if you indicate that). Just like + used to mean get these words and only these words.
The ranking of results might vary when you quote versus verbatim, but what you're asking to be retrieved is the same to us.
At the time, + was used to require that something be present
When + was eliminated, quotes took its place. Quoting a single word was the same as when you used to + a word -- find the exact word. Quoting a phrase still meant find the phrase, but that also meant it was required to find the phrase.
No, quoting did not take its place. At all. That's why verbatim was introduced, after outrage. Google claimed it did, but it still aliased. It still decided to provide results without quotes.
Again, this is why verbatim was born. From that "Google no longer gives precise results, ever" angst.
I can confirm what bbarnett says about date ranges and verbatim. Turning on a date range toggles verbatim off. Turning on verbatim toggles the date range off. It's impossible to enable them both.
As for the quoting, it should work. And if you have an example where it's not, please let me know. Quoting is designed to exact match. It shouldn't produce synonym matches, correct misspellings, find content that's not on the page as we saw it when indexed (I think I did say one confusing part is that if there's punctuation, that gets dropped when matching).
I read the article, and the HN comment it links to, but didn't find an example in either, and it doesn't match my experience. Does someone have a concrete example when using quotes results in pages not containing the search terms?
I work for Google Search, and as I shared elsewhere, quoting still works. It really does. If you or anyone finds an example where you believe it doesn't, please let me know, and we'll debug. Typically the reasons people believe it is not working is because:
1) text appears in ALT text
2) text is not readily visible on a page (maybe in a menu bar or small text)
3) there's punctuation ("dog cat" will match "dog, cat"
4) page has changed after we've indexed it (so view the cached copy, if available)
I believe you! (see my other comment in response to the original one).
But it seems people don't (my original comment is being heavily downvoted because of this). And although they can't submit even one example, the fact that they don't believe you is obviously a symptom of a bigger problem.
For some reason, Google is losing the trust of power users.
No results containing all your search terms were found.
- few results with quotes, not more without quotes:
Your search did not match any documents.
It looks like there aren't many great matches for your search
Tip: Try using words that might appear on the page you’re looking
for. For example, "cake recipes" instead of "how to make a cake."
- no results with quotes, but results without quotes: Google says that the search with quotes didn't find anything, and that they searched without quotes instead.
I have yet to find any instance where Google corrects the inside of quotes without any warning.
Google has been regularly ignoring quotes for at least 5 years, probably longer. That was one of the biggest factors for me dropping it as my main search engine.
That's like speaking to little children, that are learning to talk, reproducing their errors. Some adults believe that it's cute, but it's idiotic, confuses the babies and make their progress more difficult and slow.
I don’t think this means anything for the point you wanted to make about search results, but please note you’re exactly wrong about baby talk! It’s not a good analogy.
Baby talk (or CDS, child-directed speech) helps engage their attention and provides valuable feedback. Kids who experience less CDS develop language more slowly.
N = 3, we intentionally never baby talked to our kids, and spent a lot more time reading them novels and other things without pictures or simplified language than (I'd guess) most people do (we did also do plenty of picture books), and their language development was in all three cases way ahead of schedule.
Could just be luck (well, genetics, probably) I guess. Maybe they'd have developed even faster if we'd used baby talk. One shitty thing about parenting is it's really hard to tell what helped, what hurt, and what didn't matter at all.
>...their language development was in all three cases way ahead of schedule.
I would put a lot more of it to having (seemingly) engaged parents. Even a backwards strategy enacted by a loving parent who is consistently trying their best is likely to outperform the result that most can manage (owing to time/money/education/etc).
That is my belief as well: being there, listening, interacting lovingly, paying attention is overwhelmingly more important than a particular technique.
I've never baby-talked to our son, but I do coach him to say things that are within (or almost) within his speaking capabilities. So for instance, this evening we were reading The Gruffalo, and he pointed to the fox and said, "Fox eat!" I said, "The fox wants to eat the mouse?" He said, "Yeah!" So I tried to coach him to say "Fox eat mouse". He got as far as "Fox eat there"; maybe he'll get to "Fox eat mouse" in a week or two.
I did this as well with the same results (but also have reason to believe genetics played a major part). But I'm not sure we're optimizing for the right thing. I'm far from convinced that accelerated language development is a good thing. I think development may suffer in other areas.
Why would you believe genetics plays a part? There is minimal evidence for that. You have actual evidence for things like your higher than average time engagement, nutritional indicators, as well as health and dental care. You probably live somewhere with decent air and water quality. Then of course the likely fact that parents have relatively prodigious vocabularies, fluency and articulation. This is why your kids are smart.
Lots of evidence that genetics have a large influence on early language development, especially speech, if you care to look. E.g. this study finds genetics contributing over 60% of variance: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3851292/
No doubt when we start looking at impairments and deficits that you can find plenty of situations where a high proportion of the phenotypic variance is heritable. But when you restrict the study to the top 60th to 95th percentile (which we are discussing here) I don't think you will find particularly strong relationships.
It's not genetics or a language strategy, just engagement.
Compare a toddler living in a ghetto concrete jungle to one who takes daily strolls through Central Park or Suburbia. The enrichment of parents teaching about the trucks and the trees and the lady with a purple hat pays massive returns.
I did goo goo gaa gaa for the first 6-8 months since you can tell there’s really nobody home up there yet and it’s cute and engaged my other kids to play with the youngest. But yes. Mine seem fine as well so I doubt CDS is going to make/break a human being.
Fathers descend to "baby talk" when the child is learning and slowly bring them up to par instead of trying to just force perfect talk from the start. They do this instinctively.
There's some great comments on this from salman khan, I think. He recorded the first years of his kid's life at home and documented this phenomenon
You think the baby really wants to hear googoo gaagaa? Now, they are trying to say "I'm hungry. Feed me!" Babies must look at adults doing the googoo gaagaa, and think to themselves that these adults are absolute morons.
The sites that Googs returns are basically the internet's version of googoo gaagaa. I look at the websites returned, and often think that the site's owners must be morons. Useless drivel clearly designed to game the Goog search results. I think think about how moronic it is that Googs allows this.
You think the baby really wants to hear googoo gaagaa? Now, they are trying to say "I'm hungry. Feed me!"
I suspect they know the sound they want to make but they don't know how to articulate it. They make an approximation and we can encourage them repeating the correct version, so they realize we understood what they're trying to do: "you're half way" but repeating their approximation is misleading.
In my experience waaaaaaaAAAAaahhh meant wet diaper or something hurts. Leeeeeah, leeeaaaahhh was being hungry. Phonetically is similar to the beginning of a polish word mleko (milk)
I honestly find it pretty helpful. You can type "russian murder painting" into Google and it will come up with Ivan the Terrible and His Son. All that hinting may be annoying if you know exactly what you wanted, but I'm not a specialist in everything I ever search for.
Then again, both DuckDuckGo and Kagi also give that result for that search phrase. As well as being more generally useful for more specific searches as well.
What would be nice is if you could toggle this behaviour. Sometimes I know exactly what I'm looking for, sometimes I don't. Assuming I never do is at least as silly as assuming I always do. Just give me the option.
I am frankly baffled that after all this focus on "personalised search", they still don't actually allow you to personalise your search like that.
Right, although piping junk into the search box and expecting it to bring back something useful is trained behavior.
I've been using DuckDuckGo a lot more recently and the thing that surprises me isn't the kind and quality of the results, it's that I actually need to use my brain to search.
It's not about whether this is a good or a bad thing—I kind of like the precision in a way, it's just jarring how different it is as an experience.
Do they? I see this stated all the time, with no references.
They type whatever unprocessed half-second thought they have into Google and expect Google to lead them to the water
Perhaps if Google didn't try to fix things for people, they would be more thoughtful with their searches.
Take away the junk food, and people will resort to real food. The same way some cities limit parking at big events so that people have to take mass transit. It's for their own good, but they have to be shown the way.
Google has optimized for working 'most of the time' for 'the most people
This may be Google's goal, but it hasn't happened yet.
I don't have very many friends or acquaintances in the tech bubble, so I base my observations around real people in the real world. More and more they're giving up on Google entirely.
Their primary search engines these days seem to be Instagram, Pinterest, Etsy, Amazon, and other non-Google sources.
When I ask someone why they're searching Amazon reviews for tech support information, they tell me because it's not on the web. That's Google's failure.
You can only be thoughtful with your search if you know what you are searching for. But oftentimes i'm not really certain what i'm looking for, or i don't know the exact terminology that should be used, so i'll just enter some related terms, in the hope that google leads me in the right direction.
A truly thoughtful search requires an understanding of:
- What you're searching for, which as you mention means terminology and knowing that information exists. (If you don't know that there's a country called Burkina Faso, it's never going to occur to you to search for its capital)
- How each of your search tools works, its benefits and drawbacks. It's similar to selecting a programming language or framework: If I need to know a holiday date (e.g. I can never remember when the fuck President's Day is), I'll Google it because that's something even a normal person would notice if they screwed up. On the other hand, when I'm looking for current events information, I use a search tool that specializes in news searches for journalists and researchers because I don't want my search results biased by what Google thinks I want to see.
- The domain in which you're searching, so you can evaluate what the search tools provide for you and use the tools iteratively.
- Your own abilities and desires, which requires self-knowledge. A search is only a success if it produces something helpful to the searcher, and something they can't understand or won't use = not a successful search
- What information is and is not available. It sounds like a silly thing, but this is how a lot of scams work: They're testing for people who lack a certain subset of common knowledge. For example, I've seen articles talking about local elections that imply nefarious intent behind some information not being provided online, and they're obviously written by people who don't commonly work with local election data. Because if they did, they'd know that when working with local election data, the default is 'idk we have it in a file cabinet or on a computer somewhere'.
Search is HARD and Google has figured out one tiny, tiny part. It's just the part that was the easiest to build with what they had and that was easiest to monetize.
> Take away the junk food, and people will resort to real food.
Many people already resort to real food, even with plenty of junk food around.
“Problem” is unfortunately, that it comes at a price, that many are simply not ready or able to pay.
Who should step in is a good question, and probably governments should make access to information a right and have high quality public service available (in this case a public web search engine). Public libraries used to fulfill this role for centuries.
> Their primary search engines these days seem to be Instagram, Pinterest...
Why would someone want to search Pinterest? Every time I've gotten a search result to Pinterest it's been some scraped image completely and frustratingly devoid of the context I was originally searching for. Pinterest is one of the worst offenders on the web.
I see what you are saying but it seems to me that it used to do a much better job at that. These days I feel like I'm fighting the search engine constantly and it is certainly not magically finding what I want anymore. It feels like some crusty unmaintained tool that I have to know how to use.
It's funny to observe my stepson learning his way through Google. It's happening mostly through the assistants on TVs and locked cellphones. But he's learning to do exactly what you said: half-second thoughts and brute forcing many queries for the same subject. He's 7.
A less charitable interpretation --- and unfortunately one that could be true --- is that Google does not want you to think. It wants to keep you stupid because it's easier to deceive those who can't think and bend their thoughts in the direction that gives G more $$$. I'd say it's not merely optimising for the stupid; it's actively encouraging it. It wants to be your brain, control your thoughts and life.
Google has optimized to whatever sequence of behaviors achieves the most profit. The search results are not chosen for utility to the user but as nudges in a cycle of influence intended to drive you to attend to an ad, purchase something, or consume particular content.
They should not be engaged in non-consensual manipulation of social or political behaviors, and the ethics of market manipulation at scale through advertisement are far from clear.
I use market manipulation to mean just that, someone manipulating the market in whatever way. I'm not familiar with the legally-oriented meaning of this term.
Market cornering is classic market manipulation. Google uses every asset at their disposal to maintain their 98%+ death grip on the search markets. The list of competitors bought, stifled, legally crushed, or absorbed is probably endless. The search market is thoroughly cornered.
I used the phrase intentionally and specifically. Advertising isn't always market manipulation, but it can be and is used to that purpose.
Google uses advertisement and content "curation" to manipulate consumers. This results in product preference, purchasing behavior, and market conditions favorable to Google and/or unfavorable to Google's competition. This includes siloing consumers in political bubbles and manipulation of narratives through the deliberate selection, order, and pacing of content exposure based on the intent of Google's shotcallers.
The reinforcement cycles inherent to their algorithms are used to manage the information made available to vast numbers of people, with highly detailed behavioral profiles used to achieve behavioral outcomes, whether it's buying something, voting, or preferences for or against particular policies or candidates.
Another factor that isn't being fully accounted for is a new SEO/marketing technique where many people are asking scripted questions publicly on sites like reddit and then stealthily providing answers that market a product or service. This leads to reddit results not being exactly authentic as well. Pretty much most online reviews cannot be trusted as we are begged to do positive reviews of companies (and when companies outright purchase positive reviews, which is also very rampant) also as a factor.
Though Google is at fault for letting their service falter to the "payola" race, many other factors are in play all across the Internet since data quality has faltered almost totally. For major-cost and non-refundable purchases I need to trust, I go to brick and mortar stores and inspect what I am buying. I am thankful not everything has shifted to an online-only model. It's going to be a very bumpy ride on the Internet until Congress and consumer protection laws wake TF up and do their job.
Is Google unique in that, though? Amazon reviews are worthless now because companies pay customers to leave positive reviews or pay review farms to leave positive reviews. Even if someone reports them to Amazon, though, the companies just close their accounts and open new ones with different names and sell the same product. It's so trivial for them to pivot when they get caught that I'm not sure there is a solution to this problem.
There is a perfectly good solution, and some people use it: don't buy from unknown vendors. If you buy a product from one of a few well-known companies in that space, they have a big investment in their brand, and are much less likely to engage in behaviors that might diminish the value of that brand.
The price is that you pay more -- effectively, you are paying for that branding.
I've had so many bad experiences on Amazon that I am increasingly doing just that. It doesn't apply to everything, but it is a useful strategy.
So how does that apply to the conversation around Google, though? We can't only ingest information from a small group of sources. That's antithetical to the entire concept of the internet.
Not really because the content is coming from a variety of users who are creating the content on the site. Reddit itself is not creating the content in the same way that sellers are trying to sell you their products and Google is trying to serve you their ads.
Reddit does have astroturfing, but a lot of communities are aggressive about identifying and banning shills, so it's not as widespread as in google search results.
A ton of reddit communities are run by shills themselves. That's why users outright get downvoted and banned when they even are posting potentially winning content. We cannot act like everything is totally normal these days on Reddit. They're hemorrhaging users fast.
Just look up posts about people getting banned from reddit on Twitter or on black hat forums. There are tons of people complaining about it.
The thing about users getting entirely banned from a platform is that they aren't able to complain anywhere near to the forum about unfairness. It's all done in stealth.
Platform algorithms now often do a lot to keep dissent about platforms themselves and partner companies out of sight. It's pretty crazy how companies that publicly advocate equality and fairness serve to undermine speech when it's about their business.
There are such posts about each and every platform out there. In addition to not knowing what's the ban frequency, there's also no info in those complaints if they got banned for a good reason.
Last winter people got fed up with antivax-dewormer-nomasker bullshit and a lot of subreddits just took the trash out, so those complaints you saw might just be be those dumbasses.
I'm not saying there aren't annoying powertripping mods on Reddit, e.g. r/Linux, but there always tends to be an alternative subreddit one can use.
IMHO it's not so much of a problem with google search, but the internet as a whole.
Most genuine discussions have moved from open, publicly accessible web to places inaccessible to search engines and general public. Smaller niche forums, blogs and personal websites with no financial incentive have died out. People have moved to Facebook, Discord, Whatsapp, Instagram, Slack, Twitter and other places behind logins. Online newspapers and portals are increasingly using paywalls. Most of the genuine human interactions and quality content is not indexable anymore. Instead we have a million affiliate marketers fighting for the top positions in search results with every possible seo trick.
Reddit is one of the last places with huge amounts of publicly accessible online discussions.
It's because private communities are the only ones free from mass abuse. Public forums moved to private discord groups with hard to find invite links / etc because public forums take an army of anti abuse workers to keep alive. While a discord group just needs a few people to kick the trouble makers and maybe revoke the invite link for a while.
If that's the entire problem, then why not make these forums read-only by default (while still keeping them publicly viewable) and hand out invite links as in Discord?
Lobste.rs, for instance, has an interesting "invite tree" concept where your reputation is bound to people you've invited and who they've invited and so on.
But your overall point is valid, I think. My pointing to a single active forum doesn't change the fact that many of other enthusiast groups have moved to facebook groups and the like.
Feature, not bug. If there's no public search, it can't be gamed for money. The problem TFA identifies is one of discerning that the person you're getting your info from is an actual person, who cares. The best way of doing that, until we find some way of creating institutional trust in these matters, is talking to the sort of person that spends all day in talking to a chat room about whatever it is you're asking about.
I too have found myself searching more in Reddit. Not to throw shade on Reddit, but even if I find exactly what I’m looking for in there, it’s depressing that it’s all bound up inside of another walled garden who will eventually have the same incentive as Google: squeeze every last advertising dollar out of the produc… I mean users. Like Google, it’s just a matter of time before they too lose their balance.
A question worth posing to this community: how can we build an internet that’s hostile to advertisers? Secondarily, how can said internet also be much more accessible to content authors so they won’t have to learn a css, html, and JS to publish some stuff? Finally, how can that content be discovered from within this network?
I "search" reddit a lot, but all my searches are always through Google. Reddit search is notoriously bad, even after multiple attempts by them to fix it. Suffixing my Google searches with "reddit" though gives all the results I'm looking for.
A factor in there has got to be 'who pays?' If it's hostile to advertisers, then there's got to be money to pay for the infrastructure from somewhere.
Maybe a tax on ISPs? I think I'd happily pay $10 extra per month for access to an ad-free interrnet. Maybe $20. But how many of the people that are already happy with the ads and poor google results would do so? Would it be sustainable?
I think you're grossly underestimating how much is needed to replace the ad revenue that feeds today's Internet. Remember, majority of people wouldn't pay for getting rid of ads, because they don't have the disposable income. So they're also not really worthwhile to advertisers. You have to divide the revenue by some small fraction of current users.
Yep. It's really weird if you think about it. The first time I saw a company being upfront about their ad revenue [1] I was surprised.
> The Premium fee is basically about $7 per year, which is less than what a free user generates in ad revenue. Thus leagues that pay for Premium and use an ad-blocker are generating less revenue than free users.
They charge about $20 per user per year to remove ads. I pay for that, so I'm not sure what kind of ads they have, but I'd love to see what they're advertising and what the click through + conversion rates look like.
What you said makes sense to me. I wonder if advertisers are paying a fortune to acquire users with a lot of disposable income.
Agreed, Facebook makes something like 8 dollars a user per year? That's just one of the mega services, imagine replacing that money for all of the players that power your internet.
It's very relevant to consider that the value of users is very unequal.
IIRC for Facebook American users the average revenue was 50+ dollars per year. Furthermore, if you're the type of person for whom saving a few dollars doesn't matter, you're likely worth more than that average; and for FB to break even on your ads an appropriate price is likely to be closer to $100 per year.
There must be a difference between paying for the infrastructure and paying for the content (including the code).
It's easy to imagine a future where the infrastructure is something analogous to a public good, like clean water. The infrastructure is agnostic to the content.
The content, on the other hand, is a product of a massive, churning, never-ending process of intellectual work. Ads, for the most part, provide the underlying economic incentive for that work.
And the dirty little secret of all that ad-supported creative content? Its value is minimal. Truly minimal. Often low quality, incoherent. If it all went away tomorrow, I would hardly notice.
That it is nevertheless so popular is something of a paradox. I know I sound ridiculous, but at the core, I'm beginning to believe that the fact that advertising is as powerful an economic force as it is, is a reflection of the fact that people's lives are more boring and devoid of meaning than we often admit.
Ads will always exist because they work. Only way is to ban them explicitly but you can do that with AdBlock for example but you still get SEO spam, placed content, inauthentic "recommendations".
> how can we build an internet that’s hostile to advertisers?
You have to reify "trust" into concrete, computer-representable data. Maybe borrow the "web of trust" concept from PGP, but do some sort of multiplicative thing where the amount you trust someone's recommendation online is the product of the trust relationships between you and the recommender. That's really the best you can do - even legislation against online advertising will be subverted by companies that go through layers of proxies to buy influence.
One small thought — having the search engine be configurable, so that the user can specify which sources to give priority to (e.g., Reddit, NYT Wirecutter, Wikipedia, etc.), would be an incremental improvement.
I, too, search "<search term> + reddit" often for product reviews and such. Thing is, the results on that front have started to slide as the paid review side of the internet catches on. I'm finding that it's getting harder and harder to trust the reddit search results - lots of shill accounts and obvious junk. That's not a google problem, specifically, but it's another degradation of a workaround for declining search result quality :(
Usually I trust Reddit threads where users give pros/cons of multiple competing products. Things like running shoes users have usually tried a lot out and liked them for different reasons. If those match with the one pair I've tried out it seems like a useful data point for decision making.
Yeah, a lot of subreddits are clogged with the same bad info that's gotten all over Google's front page. The stickied "list of recommendations" on an enthusiast sub is just the same as you'd get from clicking the top result of "Best X 2022" on Google, complete with affiliate links
That's not the point - the point is that paid shills are astroturfing reddit enthusiast subreddits, so that operator doesn't shield you at all. Once a hobbyist subreddit gets big enough, it attracts a lot of attention from shady types trying to capitalize on the captive audience. I've seen it happen numerous times (only when the offender is caught) in the /r/watches subreddit, the /r/overlanding subreddit, etc.
About the "dead internet conspiracy" - I've worked in writing how-to articles for a fairly large "help" website. They paid very little attention to the quality of the articles. I was paid for each piece and thus had about 30 minutes to write an article and later integrate feedback from internal review. Otherwise the payment became too low.
The most important factor was cramming SEO terms and links to keep people on the website into the articles.
The result is trashy articles that could well have been written by a bot but aren't. This could possibly be done with the help of curated bot-content, but I think we're far away from the point where this is really more profitable than getting students to do the work.
>> This could possibly be done with the help of curated bot-content, but I think we're far away from the point where this is really more profitable than getting students to do the work.
It may be becoming borderline. I expect that sentence/paragraph completion is already becoming useful to people who churn out quick content for a living. In any case, the important part isn't whether or not it's bots. The important part is whether or not it's authentic. The precise meaning of authenticity gets squishy, but it exists nonetheless.
IMO the sentiments are correct, whatever the details. Part of why google sucks is that the internet is worse, for a bunch of the things we use google to search for. The internet becoming a larger, more profitable industry changed it. Instagramming for influencer perks, SEOing, or selling targeted ads like FB do... it does not lead to the same places that earlier iterations of the WWW produced. Times change.
My friend briefly had a copywriting job writing weed strain descriptions for dispensaries. He was never provided the product he was describing, just told to make it up.
I have the very same feeling concerning electronics. Searching for a particular product does not even popping up 5-10 comparison articles but the content of all seems to be based on technical specifications of the manufacturer only, which I already have a hands on.
Significantly more time required for consciously choosing a product to purchase (which in my case is critical because I am like Sheldon Cooper trying to choose between PS4 and XBOX One normally, to the horror of my wife, she wants a new TV and it is months long project based on accurate and quick data, and now this, with Google, which makes our family atmosphere even more tense : ) )
In case you haven't encountered it yet: https://www.rtings.com is a good site for TV specs/reviews specifically. I know someone who works there and their methodology seems legit (I like it more than Wirecutter).
Most reviews are useless from the start because most reviewers are totally dependent on manufacturers or dealers providing samples (yet will generally claim to be "totally independent"). That's before you get to reviewers who can't or don't know how to test the product in question and so end up narrating the manufacturers specs to their faux testing.
Niche, probably outdated, but indicative: The only way to purchase a laser printer with high printing quality these days is to buy something expensive and hope for the best. Magazines used to do actual reviews of these things.
Brother MFC laser printers have delivered for me for over 15 years now. I have bought many for various small offices and my home and relatives’ homes, etc, and I have not heard any complaints.
I especially like the scan function where brother web connect OCR’s the document and saves it as a pdf directly in your Box/Dropbox/OneDrive/GoogleDrive folder. Just wish it worked with iCloud Drive.
Oops, my bad, I did not properly take into account high quality printing. It is definitely too cheap to be anything high quality, they just get the job done for everyday printing at a low price.
I look back at the newspaper stories I wrote a few decades back. I could get the score from the coach, find out hits from who and when, and after that 20 second interaction, I could write a news story in maybe three minutes, which told you everything you needed to know.
Somebody who worked at a winery once told me that the flavors they mention on the bottle are actually what the wine is missing, and they name them in the hope that the power of suggestion give you a more balanced impression.
It would be interesting to see some sort of study, to see the impact on wine labels on: normal people, vs wine "experts", vs sommeliers.
I've seen a few of those wine documentaries about sommeliers, and they certainly made it seem like it was a legitimate ability to identify stuff. I'd be interested to see how close they are in a more neutral, measured environment.
>Tasters can't even tell reds from whites with a great deal of accuracy. [2]
Actually, this one is bullshit. While it may be true that the average person without blind tasting experience cannot do this, anyone who has actually done blind tasting seriously should be pretty accurate at this. Purely structurally, red wines generally have much more tannin and more alcohol than whites.
Sommeliers are professionalized, with courses & exams. But I don't believe they are actually judged on their ability to taste wines and detect flavours. Designing such tests would be simple - rate of successful flavour identification based on data from other Somemeliers.
Instead Sommelier exams are subjective - candidates are judged by another Sommelier on subjective criteria rather than objective measurement. In my opinion they judge it as a dramatic performance: how quickly can the candidate rattle-off various flavours? How "high class" is the language they use to describe the flavours? How does the candidate present themselves? Sommeliers dress in fine suits, but this should have no impact on actual wine tasting ability. Yet I am sure if I showed up in a draggy old t-shirt I would fail regardless.
The whole industry seems allergic to objective scientific measurement. I am sure Sommelier's do have some ability to identify varieties, but are they actually tasting all those subtle hints they list off? I doubt it.
What is this comment based on? To me it feels like this is what you imagine the exams are like. The MS exam (the highest and most famous one) has three parts: theory, service, and tasting. The service part is all about presentation, absolutely (that's the whole point). The tasting part is not. In particular...
>But I don't believe they are actually judged on their ability to taste wines and detect flavours. Designing such tests would be simple - rate of successful flavour identification based on data from other Somemeliers.
That is exactly how the tasting portion of the exam works. And
>In my opinion they judge it as a dramatic performance: how quickly can the candidate rattle-off various flavours? How "high class" is the language they use to describe the flavours?
This isn't really how it works, especially the "high class" language comment. Actually there are a variety of different things that they are supposed to identify about the wine, known as "the grid" [1]. You can take a look at it yourself. They are essentially judged based on how well their grid matches up with the consensus grid from the master sommeliers.
This is something I notice in advertising generally: whatever they most emphasize is least likely to be true about the product (e.g., the "great taste" of McDonald's).
The other day I looked up the wordle answer (I know I know). The first result was a site where I had to scroll through about 19 paragraphs of SEO vomit to get to the answer. The page could literally have one word on it and serve it's purpose. If that isn't a sign that the the internet, or at least google search, is dead, I don't know what is.
There's a whole industry of sites like this for NYT crossword answers, to the point that it's often impossible to get any organic results at all for something that's been clued in the Times, which is frustrating because I don't just want the answer (there's a button in the app that does that already), I want to learn about the thing I’m unfamiliar with. Luckily, most topics that come up have some content on Wikipedia so I go there instead.
Ugh yes that’s super annoying! I don’t want the answer spoon-fed to me. The point of searching it is to actually learn a little something which will help me remember in the future
lol the same happens with WSJ. It’s very disappointing to try and do a bit of research on a clue and get the answer on a bot site linked to the puzzle you’re doing. Interesting how they do it though.
It’s like a DDoS attack on your mind (distributed because everyone is doing it). The attention span economy at its finest.
Reminds me of those ways to catch spammers or bots by occupying some of their resources with meaningless tasks for as long as possible. Except it’s turned around.
Sometimes I wonder if there’s even any real money in ads anymore or if it’s just a giant circle jerk that slowly destroys society…
Speaking of bots, I'd be interested to know the percentage of articles on major traffic content sites are authored or co-authored by AI.
My suspicion is this is rife given how many articles read poorly and are almost entirely fluff. If this is true it would appear we are doomed to algorithms shaping our online experiences, which is worrying given the existing shrinking diversity of opinion and content. It's like a entropic gene pool in nature, but with information.
Sports stories are frequently written by bots/AI/whatever you want to call them. Sports stats makes it relatively easy to create written text articles about games. When first rolled out, there were some obvious tells such as an excessive amount of "for the first time this season" or "set a record for the season" in articles about first games of the season. Unusual plays or quirky behavior by participants tend to be missed in these articles but otherwise they're serviceable though somewhat dry and bland.
Stock analysis articles are very commonly generated automatically from templates. The selected template is based on some real characteristics of a described company (whether the stock went up or down recently, what is the P/E etc.), but the content is generic and reused across all companies with similar characteristics.
Apparently, using a machine translation as a basis and working through correcting it to read or write foreign languages is a growing trend and that’s a form of computer assisted literacy if I were to guess.
Bar that, it’s humans outrunning AI in the race to the bottom with a head start. Human people can be forced to be incredibly machine like.
> Human people can be forced to be incredibly machine like.
Amazon has a site for humans to pick up small tasks and get paid for them called mechanical turk [1], which is a reference to a fake chess playing machine with a human inside [2]. With the great resignation and the workforce otherwise pushing for a reasonable standard of living, I'm not sure how heavily mturk is still used as depending on tasks and speed it's really sub-minimum wage work for many people. [3] But as The Atlantic article says, sometimes it's the only work people can get.
I think the quality of searches like “best TV” will improve dramatically once language models are used to generate SEO spam. Anything would be an improvement over what today’s human spam bots produce.
Lately, half the results I get are pages that I cant view without paying to some service or signing up for a free trial.. Google literally serves up results that are unreachable ..
Can we also talk about how Google allows top organic results to LinkedIn, Instagram, and Facebook which literally present you with a LOGIN page before you see anything related to your search?
Absolute worst UX ever, yet they allow it because it's their SV buddies.
> yet they allow it because it's their SV buddies.
At one time, Google used to ban sites for "cloaking": offering one version of the page to the crawler, and another to the user.
But over time they got into trouble from these sites, getting sued left and right. Being accused of putting up a wall, and abusing their monopoly. Eventually, these sites won out and Google dropped this requirement.
Similar story with Image search. "Blah blah something something copyright something". And voila - now you can't get direct access to images on the search results page.
This might be the case for news sites which show limited content until you sign into a paywall, but I don't recall the biggest offenders which show NO relevant content at all (LinkedIn and Facebook) saying anything.
Classic example of this kind of content... Try searching "How to use X to get stains out of Y".
You will find a page for almost any X and Y combination. And they will all have wording like "Put some X on the stained Y... wait a bit... rub it in... and then put it through the washing machine. Hope it works!".
I mean then is DIT really a conspiracy theory? I know that people in HN have already started doing replies with AI in some threads. (only because they tell people). Wait a few more years and there will be no way to know if forums are just bots generating content.
I guess HN would be a weird outlier, because there are no ads. Except maybe the bots would be useful after all when various "Show HN" or "Launch HN" and have the bots cheer those companies and get random publicity.
The idea that a genuinely Dead Internet might be an improvement over the current internet experience is a fun one, and one I don't entirely disagree with.
Within the last 6 months I’m seeing a rise of very good NLP content farms that almost have what I’m looking for but not quite. I think what they do is start off a transformer based NLP with example queries that others have searched for and it generates a realistic looking answer that’s usually wrong. The scale and breadth of these could only be done by machines.
Writing SEO content isn't necessarily one of them anymore. Check out AI content services like https://www.frase.io/. You can also generate things like product descriptions inside the GPT-3 playground.
Reminds me of an old The Parking Lot Is Full comic.
> Little-known Fact #839: There are only twenty-three people alive today, and you're one of them; everyone else you know just looks human to lull vou into not searching for the other twenty-two. Lonely? You should be.
I would expect an automaton to be able to spell; so perhaps the presence of spelling errors is a mark of an authentic page. Maybe one could force Goo to spit out authentic results by including a strategically-misspelled word in the search terms.
Totally agree on the reddit point, I've also noticed the same occurring to me. The girlfriend recently got Pokemon Arceus and sometimes asks me to Google something she wants to know.
It's completely pointless, you just get a bunch of articles from news sites (??) that transcript the quest but not tell you anything more. I miss a nice community wiki like I'm used to from playing Dark Souls etc.
I've just started appending site:reddit.com to everything, works a lot better.
Yeah, Fextralife saved my ass multiple times while working through the dark souls series. It's a shame that type of community resource isn't more popular.
Google could fix this by making the algorithm take into account searches that often end with "reddit", thus applying more weight to Reddit results to similar searches where the user didn't include Reddit in it. Clearly it's an indicator that those are the better results.
Take StackOverflow for example. Almost any programmer will find a SO result as the top result and it's usually exactly what you're looking for. Since there's no money to be made by companies writing blog posts on debugging a compiler error, Google's algorithm works as intended.
Question is: Why hasn't Google done anything about this?
It's the organic results that are terrible, so they're not losing ad revenue by placing these garbage sites at the top. Perhaps its to intentionally make better websites pay for ads to get better placement? But those won't be the ones to ever pay to begin with...
I think it would be foolish to assume google hasn't spent hundreds of hours in meetings talking about what they can do about everyone having to type reddit. Problem is they are facing an army of SEO experts who are one step ahead of google. As well as legal issues. Imagine if it was found google was artificially boosting reddit in an unfair way.
This is Google we're talking about. They could very easily conquer the SEO experts the same way they basically eliminated spam and most web browser exploits.
I did software dev at a marketing firm for about a year, and it was pretty soul sucking, so I know what you mean. I won't work at one again unless it's literally my only option.
I’ve been there too. The sad thing was that the people I worked with were genuinely great and smart people but you’re spot on about the actual state of the environment being a depressing and soul draining thing. I quit that job on moral grounds, I just couldn’t be a part of that any more.
I agree google is bad, but I think reddit is rapidly becoming equally as inauthentic. I'm sure every major player at this point understands the gains that can be had by astroturfing reddit. The real problem seems to be the internet is inherently untrustworthy and going back to finding people you trust in the real world is the only fix I can see.
We've also got to address the people and corporations that are gaming the system that google has created. Google is by no means off the hook, but Marketing practices have also taken a very bad turn to deception and in reinforcing a payola systems recently that we may never be able to recover from trust-wise.
> - "Why are people searching Reddit specifically? The short answer is that Google search results are clearly dying. The long answer is that most of the web has become too inauthentic to trust."
Haha, the noobs. I use HN instead sunglasses cool face
A bit more seriously: I fully agree with this. And if HN doesn't have what I'm looking for then I use Reddit as well. But if HN has some info on the topic with a few highly upvoted threads, damn, it always impresses me.
When I'm looking into some project or piece of software I'm unfamiliar with, I really do search for HN posts on it. Fastest way to cut through (enough) of the biased material and get something genuine. Even hyped stuff usually has enough contrarian posts to give you an idea of where to look for the skeletons.
HN has now been around a while, is really popular, and is starting to catch the attention of ad/marketing companies now, though - how do you know what here is genuine?
Even the heuristic of "only trust accounts older than n years" isn't perfect, as eventually a few people will undoubtedly sell their old accounts on a dark web market for a little extra cash...
And it's not just the spammers. Any topic that touches domestic or international politics in any way almost instantly brings out a lot of bad-faith actors, here or anywhere else on the internet.
> Even the heuristic of "only trust accounts older than n years" isn't perfect, as eventually a few people will undoubtedly sell their old accounts on a dark web market for a little extra cash...
Yikes, I hadn't thought of that angle. That would explain some of the long-dormant accounts on Some Other Place that suddenly start spewing out-of-character garbage, assuming they weren't password-guessed, data-breached, or keylogged by some rando.
Hmm, I think for now it's relatively easily distinguishable. I've noticed that HN'ers feel "like me". So I look for that specific signal. I look for signals that care about curiosity and an insane hunger for the truth. Also, it's tough to mimic "Oh yea, I was at Xerox Parc programming this language that was posted about, let me give some nuanced insights into what was stated in the article" (that quote was made up for the sake of example).
The only marketeer that would be able to consistently fool me might be a marketeer that was a developer. But last time I applied for a junior marketeer position I was asked: given your resume, don't you want to be a developer instead? So devs seem to get pidgeonholed into always being devs.
Me too I even have a bookmarklet in Firefox so that I can use a prefix (hn) and the search is rewritten as site:news.ycombinator.com, to make sure all results are limited to HN.
I also have the same kind of bookmarklet for Reddit and Google Scholar.
The reason that the quality on Reddit is higher is because there’s people moderating those quality subreddits. Without those moderators it would all turn to crap and be just as useless as Google.
(This isn't an argument against your point, just a bit of additional context that increasingly is odd to me as Reddit gains more and more social weight)
Very true. Fortunately I don't think most moderators want to see their community destroyed. That's exactly what would happen if they started taking money. As a group, people will catch on fast and either force the moderator to step down or just leave for a new subreddit.
> - "There’s a fun conspiracy theory that popped up recently called the Dead Internet Theory..."
> I hadn't heard of this. Now that's some sci-fi level of conspiracy but in today's world it seems totally plausible.
I never believed in conspiracy theories, and after I read "Media Control" by Noam Chomsky I understood there is no need for conspiracy theories once you understand how individual incentives are aligned and how individuals always act to maximise profits.
Someone on HN phrased this and I am not taking credit for it but it explains beautifully whats going on: "Google is not making money by showing you the best search result they can, they make money by keeping you searching."
That does not make sense. If searches did not result in satisfactory results, then people would stop searching.
Which they are, evidenced by restricting searches to HN or Reddit.
This is a problem for google, maybe not right this minute as growth might offset dissuaded users, but nevertheless, it does not behoove them in the long run to provide garbage search results to people.
> Which they are, evidenced by restricting searches to HN or Reddit.
You are right but overestimate the number of people doing these restrictive searches. They should be in the <0.1% range. I personally switched (to duck and latly kagi.com) and know quite some people in tech that did so too, but the non tech-savy person (which are the majority of overall users) doesn't even know other search engines.
Eventually, though, even non technical users will stop Google searching if it wastes their time. If they do not find what they want, then they can go back to asking people in person or instagram or whatever, but I would not expect people to keep aimlessly searching.
I view the Dead Internet Theory as Black Mirror style satire. All it would take is liberal application of GPT-3 style transformer AI to content generation and much of the Internet could be fake. You could have fake political trolls arguing with other fake political trolls from the other side, fake blogs, fake review sites, etc. and it would take me a while to notice. Most of the modern Internet is just that bad.
Advertising always creates perverse incentives. It works in traditional media too. Look at what happened to things like Discovery and The Learning Channel when they became subject to advertising based pressure for ratings. They went from having actual educational content to being full of tabloid trash.
The death of the authentic web really chimes with me. I have an almost physical reaction when I occasionally come across a page that isn’t trying to sell me something, that is a labour of love.
A month or so ago, I was trying to help someone retrieve some very old Wordpress for Mac files. I found http://www.columbia.edu/~em36/wpdos and was so touched, I sent the author a few dollars for a coffee
Google's problem is they've virtually nothing (given their resources) to "commoditizing their complement". https://www.gwern.net/Complement
Google's compliment is web sites. What have they done to make a web site easier to make?
They even killed their RSS feed. They have released a bit of web tech, but their offerings are generally a bit sad or only solve Google problems (e.g. Go).
If you want to distribute an .exe or .app, MS and Apple have released some pretty good tools to help. If you want to write a blog or make a simple web app, it's unlikely you're going to think "Google has some great stuff to help, and has awesome tools". Mozilla's web resources are better. Microsoft's web resources are better.
>> "Google increasingly does not give you the results for what you typed in. It tries to be “smart” and figure out what you “really meant" ..."
> This is the most annoying behavior because I really mean what I write.
Yeah I remember this being mentioned in a local presentation at university. As a great thing. Google doesn't search for what you write, but what you want.
The problem is that very often Google don't know what I want. Before they introduced this, I was able to define my query so that I got exactly what I wanted.
Dead Internet Theory is totally believable. I remember back in 2006-7 or so, being slightly curious about putting up a food review site because I was really angry at my local X food establishment. I found places were you could buy complete restaurant database ready to be scripted onto the web for maybe 90 a pop. The data was actually pretty good, but I was shocked by the huge community around flipping these dbs into internet spam. For the very high majority these were hungry business types who could barely open a code editor without asking for help. I only expect the problem has gotten exponentially worse since then now that ai generated content has improved in quality.
I think the OP is weak because it conflates ads and seo spam. Yes, Google went all in with ads, and yes, this hurt its credibility and the quality of its products.
But there is no conceivable universe where seo spam isn't the arch enemy of Google. Google needs to fight spam to survive, it knows it and it does. But that's hard. So hard in fact, that nobody else has cracked the problem, and for all the anecdotal evidence of someone switching to Bing, Google's marketshare is still utterly dominant.
The Dead Internet Theory may have some weight: Google hasn't dropped the ball, but it is slowly drowning into the sea of "content-free content".
That said, the reason the Reddit trick works is that it uses information Google explicitly excludes when ranking content (engagement signals).
Google has a bunch of “objective standards” that it uses to paternalistically shape what the web looks like. Many of these are divorced from what users actually want for pieces of content (https, AMP, a life story in front of recipes to demonstrate authorship, etc).
As explained in another comment [0], the real reason the Reddit trick works is that Reddit content is hand-curated by humans who are dedicated and passionate (and, I think, also work for free). It's a garden in the jungle.
I think there’s actually two dimensions to this problem and the article and your comment only address one of them.
The other dimension is that, in the past, if you searched for stuff your results were likely to be a blog or a forum thread. Today the bloggers have evolved into instagrammers, TikTokkers, YouTubers, or Podcasters. The Forums and community pages have moved to Facebook, Twitter, Slack, Discord, etc.
So it’s not just that SEO and botspam has eaten the Google results page, it’s that this is all that’s left of the open internet that needed search to navigate. Much of it truly is a wasteland. Google owns part of the blame for crippling RSS and privileging recent pages and specific domains or AMP pages over evergreen, self-hosted content in search results. But also users have given up on the open internet in droves. Instead of starting a fansite they start fan subreddits or discords instead.
That inauthenticity comment really hit home for me, too. I realize that I do not trust the internet at large, and haven't for a long time. That's been the real trigger for my retreat from mass social media into smaller, tigher online communities.
Even HN is starting to feel like it wants to sell me something.
The latter feeling may be because HN is run by a startup accelerator. They run literal native ads on the front page whenna startup they sponsor goes live.
I don't think that's where it comes from. It comes from the same places as Google, where people know the value of this community's attention. Posts at the top of HN are designed to get to the top of HN.
This fits well with my own worldview. I've been griping about Google results for years, and jumped for DuckDuckGo when it became usable. I'm sure that fifteen years from now, DuckDuckGo will be ad-infested crap and someone new will come along to replace it, just as Google replaced AltaVista.
Even DDG knows that it can't handle everything, and so it has its bang shortcuts. I've used the !reddit one, and I'd use !w (Wikipedia) except I do those from the Firefox search bar.
I've heard the "everything's a bot" theory before, but never saw a name put to it before. I'd have to guess that 99% of all SMTP traffic is spam at this point.
>I'm sure that fifteen years from now, DuckDuckGo will be ad-infested crap
In terms of direct ads, perhaps. But for SEO spam, in many cases, DDG already seems to be there. For example, things as simple as "python datetime", "python json", or "python datetime.now", where it would seem obvious that the top result would be the documentation for the module/function, have spam sites above the actual Python documentation. Meanwhile, search for "matplotlib", and your screen will fill up with ads.
> This is it for me exactly. I search for the following kinds of things on Reddit exactly because results on other sites aren't trustworthy: Reviews are secretly paid ads.
So are reddit-comments and entries. There it's even worse, because most people don't connect the content with manipulation.
> The "best" recipe for pancakes is only what's trending on instagram right now.
So, like reddit? I mean every platform has their hive mind, and reddit is even worse, because the hive mind can be manipulated with paid upvotes, not just reposting and comments.
> The same for trending programmer tools.
Aren't most of them yet again commercial products from companies?
"It is obvious that serving ads creates misaligned incentives for search engines..."
I don’t think this is the problem. The problem is the need for public companies to grow exponentially. If you take away the need to constantly grow exponentially, then ads on search can be both balanced well and make an insane amount of money.
Based on some of the April Fools' Day experiments that Reddit has done in the past, I'm not sure why you wouldn't have the same hesitation and mistrust of Reddit posts and comments. So much of the content, even on Reddit, is made by bots or copied by bots from older, legitimate user-generated content.
There’s something sad and ironic about using Google to search Reddit. One, I mostly dislike using Reddit - I only want to see specific discussions very occasionally. Two, what is the state of the internet if I have to use the best search engine to find content on a website I mostly dislike? Haha.
Reddit's search engine is kind of crap (not terrible, but also not great). That's why I use Google for Reddit, to have a better Reddit search experience.
I figured that's why it's so high, is Reddit's UX keeps slowly getting worse so the best way to find stuff on Reddit is by searching outside of it.
Searching Reddit helps but the quality of comments has gotten lower since 2015 or so. It seems to coincide with the wave of subreddit bans and the nakedly politically-driven moderation on subreddits. And with the reflexive attitude—against anything countering the Reddit consensus—that developed during the Trump years. High quality posters seem to have withdrawn from the site (at least in how much they comment) and what's left is mostly ignorant teenagers and bitter millennials with shitty jobs. In turn, that crowd is much less likely to upvote high-quality thoughtful content, so the cycle continues. The decline in quality has trickled even into the less popular subs. Don't get me wrong, the site has always had problems, but the more recent decline in thoughtfulness is dramatic.
The worst part is that despite Reddit getting so much worse, there is no other place that's grown to fill the void. This place is great, and I do search HN when it makes sense, but it's small and narrow in scope. Reddit basically crowds out any competing websites by sucking up all the low-level chatter required to sustain a community, but has also pushed away high-quality posters, who now have no place to go. Very tragic but maybe a good case study in shitty network effects.
Quality on any non niche popular subreddits were already abysmal long before 2015. Reddit is useless for anything which is not highly specific but there are some diamonds in the rough: great subreddits exists about fashion, knives, gardening, coffee, shaving and plenty of other weird interests.
This is true but I find that about 2015 is the inflection point. Even posts about highly technical subjects are not as good if they’re from after that.
I dunno, looking at the growth curve in Paul Graham’s tweet I expect most of the drop in average comment quality can be attributed to the size of the user base. It’s hard to keep high-quality content the norm even in much smaller communities.
/r/nfl had a reputation for high-quality content and wasn’t a particular battleground in the Trump Wars. It’s still a good breaking news feed, and the live game threads are fun, but every post is dominated by joke comments and memes.
I find the idea that banning Trump supporters killed reddit to be pretty far-fetched.
To the extent there is a change in quality, it probably comes from other factors, including having a bigger, broader, and different user base now than in the past (and only a small portion of that change likely came from Trump-related bans).
Parent didn't say Trump supporters. And I completely agree with their observation. Quality went completely off a cliff around that time. I think it just officially entrenched the Reddit orthodoxy and that there is a "right way" to think, and that infected even non-political subreddits. Not all though, I know some subs are much better and open to discussion than in the past.
>most of the web has become too inauthentic to trust
i think that specialised search engines are gaining ground. For example, I am using github search for searching code samples, that works better than google.
You might want to check my side project that tries to explore the subject. I have a search tool / catalog of duckduckgo !bang operators, i am hoping that it allows for better discoverability of specialized search engines.
The latest addition is a description for each search engine, just hover over the name, and you get a description derived from the sites meta and title tags.
I think that specialised search engines are gaining ground, it has become easier to set one up, thanks to elasticsearch/lucene. They can be quite good, for a limited domain, and they don't have to invade your privacy in order to find out what you are looking for. I think that what is missing are tools like this, that would aid the discovery and use of these search engines. I hope that this will allow them to eat into the market from the 'low end'.
> The latest conditions on mountain bike and hiking trails are being shared inside communities like Reddit but not on the web
This point especially rings true for me, but it also concerns me a bit. Reddit has killed a lot of other forums over the years. If something happens to Reddit, we run the risk of losing a large corpus of information.
>> "Google increasingly does not give you the results for what you typed in. It tries to be “smart” and figure out what you “really meant" ..."
> This is the most annoying behavior because I really mean what I write.
I hate this too. I do get typos corrected by Google. But I don't need that - if I put a typo in my query and get bad results, I can correct the typo myself. But if Google decides I must not have meant what I actually said, there's no way for me to correct that. It's a ridiculously bad tradeoff - we eliminate errors that are trivially fixed by introducing errors that can't be fixed at all.
I have similar feelings about phone input autocorrect, which automatically converts typos that are very easy to read into (mostly) correctly spelled words, plus (sometimes) completely unintelligible nonsense.
I agree. Often, what I am after online is to see what other realpeople are saying about something. typing 'reddit' into google is basically a proxy for "please google for the love of good, can you start indexing actual human discussions again?".
I have a similar habit, I often search "[topic] forum" - I'm not fond of Reddit specifically due to accessibility issues although in some cases I still go there because it's the only good source.
>because results on other sites aren't trustworthy
Reddit is gamed way more than google. Paid posters, moderation of anything against a narrative. Google search may be dying, but reddit ain't doing much better.
> The long answer is that most of the web has become too inauthentic to trust.
It's only a matter of time before reddit too becomes too inauthentic to trust. Not only is it directly funded by advertising, its audience is mainstream enough for advertisers to invest time and money posting fake opinions in order to make it look like it's coming from real people.
I seriously hope I never see comments or news about people appending hacker news to searches. I don't want advertisers to kill this site when they catch wind of it.
They have filed for an IPO last month with the SEC so they should go public very soon.
Last valuation was at 10b$ which is ridiculous for a website that can literally get its most popular subreddits shutdown arbitrarily whenever a small group of extremely online volunteer mods decide to "go on strike" by locking the subs because they don't like something/someone else on the website.
It happened before and the admins yielded to them so I don't see why it wouldn't happen again, especially since it's not like they can run the website without that weird cabal of (mostly delusional/psychotic) power mods doing their work for free.
>>is ridiculous for a website that can literally get its most popular subreddits shutdown arbitrarily whenever a small group of extremely online volunteer mods decide to "go on strike" by locking the subs because they don't like something/someone else on the website
I have a story about this - and its worse than just mods -- Admins intervene and set narrative on for whom is allowed to mod and make mod decisions...
I wont reveal the details - but I have seen Admins literally come in and fuck up Mod orders because (my suspicion) is that the Admins have MANY accounts that /appear/ as mods - but are actually Admin shill accounts...
I had this happen to me first hand and I was appalled.
Reddit's ethics are absolute garbage in this regard.
I've also thought that. The reddit shutdown made the frontpage of (reputable) news sources, I can't imagine that investors aren't going to be asking the admins how much control they really have over the user experience on any given day.
regardless, what is reddit's current value? (I've been on reddit for 15 years - but I have recently deleted my accounts due to censorship and ban-hammering for the most ridiculous reasons.)
It's perhaps a little bit early in its creation to be sharing this but I am working on a new search that should help to fix the problems mentioned in the article, https://namusearch.com/. It allows you to build (and share with others) a curated list of websites that you want to use for searches
Yup, when I want to search anything I use a combination of reddit, HN, and Discord. My main use of Google these days is to find a website I forget the name of but roughly know what it's called. In the olden days, I used bookmark aggregation sites like del.icio.us to search for relevant content, which was generally more fruitful than a Google search.
I didn’t really even think about this properly until just now.. these days I am looking at Reddit, Facebook groups and if needs be, YouTube (videos not by ‘creators’ as far as possible) to find information I used to google. Ads and referral links have totally ruined the usefulness of so much information.
I agree that Reddit remains a good source for info, but I’ve found that Google usually does a good job surfacing Reddit results — usually in the top half. Though this is most notable when I’m googling for esoteric info about TV shows and video games (e.g. “best build mass effect 3”)
"The latest conditions on mountain bike and hiking trails are being shared inside communities like Reddit but not on the web."
I just wanted to mention that a friend of mine made an app for user-reported trail conditions that might be worth taking a look at:
https://trekko.app/
> I search for the following kinds of things on Reddit exactly because results on other sites aren't trustworthy: Reviews are secretly paid ads.
There is so much shilling on Reddit if you knew it would blow your mind. I wish more people realized this. Reddit is the best place to shill because not only is it ridiculously simple, people also automatically assume you’re not shilling, and then once you seed the idea, everyone else will do the shilling for you indirectly.
The healthiest way to use Reddit is like Wikipedia: assume the information you’re reading is highly compromised and biased in one way or another, but use it as a starting point in your further research and it’s a great tool.
Reddit posts are not your friends. Upvotes do not mean the contents of the posts are legitimate or not shilling.
Reddit is the best place to shill and the sooner the non-shillers figure that out, the better off the entire internet will be.
Reddit also - in my opinon - actively enables shilling and botposting. Why do they have an API?
A forum that's meant to be 100% about humans talking to humans doesn't need an API, so why does it expose one?
Also the model of user-created and user-moderated subreddits actively enables the creation of shill accounts. It's trivial to create a subreddit and use it to farm karma with a ton of bots. If you can keep real users from ever entering your walled garden of a subreddit (of which there are many) your bots will never be detected until you wipe their comment history and set them loose on the rest of the site.
This is nonsense. Google is not particularly woke as a whole, certainly not to the extent that concerns about "woke"ness drive massively important product strategy decisions. And certainly the government isn't forcing anyone to be more "woke" or, to my knowledge, really influencing search results at all. Unless you have evidence to the contrary?
Your case might or might not be true on political topics, and I won't engage on that, but I think OP mostly mentions impact on product reviews and other more 'down to earth' topics.
I don't think woke activists would care too much about shoe brands, so I think your response is missing the point at hand, which is that search results (including but not only political ones) are deteriorating fast
I was curious about this, so I decided to think about what Google's image search probably does and apply Occam's Razor...
It appears the image search is closely tied to the overall web search; the images returned are images from pages that match the search terms.
What would I expect a search for something like "white inventors" to return? Are there lots of websites cataloging inventors who are specially white, or discussing inventors specifically because they are white? Probably not, why would there be? But what about websites discussing non-white inventors, or talking about white vs non-white inventors? That seems much more likely to me since discussing and/or promoting a minority is more notable and common.
So it seems like the algorithm has decided to categorize "white inventor" and "black inventor" in very similar ways - both search terms are returning results relating to inventors of different races, whereby a non-white inventor is more interesting since it's the exception. I don't think it is surprising that these search results are as they are, and this seems much more likely to me than the whatever the alternative is that you are suggesting.
On a related topic, if you search for "black people" one of the image results is King Kong, and it comes from a website discussing comparing black people to monkeys. Another one of the images is (white) Mitch McConnell surrounded by other white people.
If you want an image search engine that is returning images whose content matches your search terms, there are tools that specifically focus on this.
For "white inventors:" Most likely if you include a race or ethnicity in your search, it is pulling from articles discussing race, so the results should be expected to be diverse.
For "American inventors:" probably it is keying to the second half of "African-American"
White American inventors are typically just referred to as "inventors," have you tried searching that?
>White American inventors are typically just referred to as "inventors," have you tried searching that?
when I search "inventors", 7/39 images are from articles specifically about overlooked black inventors. all searches relating to inventors are poisoned by american (i.e. foreign) culture war topics.
>probably it is keying to the second half of "African-American"
I refuse to believe that google can be so stupid as to do this after over two decades of development in natural language processing.
> I refuse to believe that google can be so stupid as to do this after over two decades of development in natural language processing.
I don't. They might have spent those decades speeding up and scaling their ability to be stupid. Stupidity at scale describes a pretty significant chunk of the internet.
Anyway, I don't think misinterpreting your intent on this type of search really costs them much, it seems like an edge case example to me.
It's well-known that Google search prioritizes newer content over older content, and history-related content created over the past few years tends to focus pretty overwhelmingly on race, gender, etc. So a sensible null hypothesis, is that Google's just showing you a representative sample of the new-ish content that's out there.
Could entirely be explained by google's algorithms capturing a zeitgeist of sorts, and presently, discussions of "white inventors" and "american inventors" have a lot to say about black inventors and their presence or lack thereof in the historical record. No conspiracy necessary.
If you google Donald Trump you get a bunch of news results about his legal troubles. Not because Google has an axe to grind, but because Donald Trump has legal troubles, and it's in the news.
Doubtless the image search is using the same algorithms and pulling images from articles. Google doesn't know what you're "really" looking for. It just knows you searched for "white inventors," and the reason you're seeing black faces is that for pages with have high pagerank for the phrase "white inventors," there are images of black people.
Not sure what woke activists have to do with it. It's not hard to guess whom Google is beholden to from recent search results: advertisers. They are increasingly an ad platform, not a search engine or informational resource. Makes me yearn for the days of the public library, which just might make a resurgence to fill the vacuum Google has left.
Its not about reddit as a search, it's about using reddit to validate your search because the alternative would likely yield poor results. You could trust the 10 listacles that came up as the first results that all look oddly similar, or you can try and filter through reddit by including it in your search terms
An important thing to realize, too, is that this is a problem that keeps getting worse. The article talks about product reviews and recipes, but it's been spreading a lot further than that. Recently I was trying to look up a technical error, and found a lot of web pages that seemed to be auto-generated with "How to solve [error_scraped_from_the_web]", complete with a list of generic things unrelated to the error (IE, "Step one: try turning your computer off and turning it back on again. This is usually a good first step, and you'll be surprised at how often...").
Likewise, I wonder how long appending "Reddit" will work. As others have pointed out, Reddit shills are already relatively common, and it's becoming increasingly common for bot accounts to create lots of random comments to appear to be human (such as finding a thread with thousands of comments, then copying and pasting the comment to another place in the thread or to another thread, or auto-generating a simple sentence based on other comments in the thread).
Sometimes the advertising hordes move so fast they kill something before it even takes off, like what happened with Clubhouse.
Free project idea for someone with more time on their hands than me:
Classical search engines determine trust automatically, based on various factors including "link neighborhoods" where trustworthy sites link to other trustworthy sites. These automated strategies are clearly breaking down; the spammers are winning the arms-race.
So maybe we need to go back to human-based trust.
People used to curate lists of websites, which partly solved this problem but didn't necessarily scale. I wonder if that idea could be supercharged.
Consider a browser extension that people install, which:
a) gives users a button to mark a site as trusted/favorited
b) tracks domains visited (and frequency)
Then, separately, you can manually add people you know personally to your "network". You trust them, so anything they trust is also something you might be able to trust. Manual favorites could be weighted higher than frequently-visited sites, and both could be displayed inline next to links on all pages you visit. You could also see which people the trust in a given link comes from, in case some of them consistently have bad judgement about these things and you want to remove them from your list. Then, finally, you could create a personalized search-engine that only indexes the sites determined to be trusted by your personal network.
Of course this would require placing a great amount of trust in the extension and service themselves, so maybe they would have to be open-sourced or self-hostable or something (a profit motive might create a huge amount of temptation to abuse the data). That's a stickier problem.
Edit: There was a little ambiguity left here about transitive trust; “friends of friends” type stuff. I think if this went on for unlimited hops, we’d be back at square one. So maybe it only uses direct contacts, or maybe some small N of hops (where longer ones are weighted lower?). Maybe this would be configurable, not sure.
Also re: privacy, maybe you could come up with a clever way to E2E encrypt the site visit data, even though it’s shared with many parties?
I've just had the deja Vu reading this post! Took me a minute to realise, I did actually come across this post and your blog some time ago, and really enjoyed reading all the posts then and now again! Hoping to read more from you!
This is very much in the spirit of what we were trying to do with trove.to [1] — give people an easy way to curate & annotate lists of websites, and layer a social graph and endorsement system on top of those lists.
The problem we encountered is that the vast majority of people are not hyper-organized list makers — the 1% rule of the internet [2]. To create a "human curated search engine" with any utility, you need a massive amount of manually-categorized data — data which most people are simply not interested in generating. This is why no social bookmarking site (e.g. delicious, pinboard, etc.) has ever taken off to hundreds of millions of users.
I still think there's something exciting to be built here, but it will likely need to take a more "automated" approach as you suggested.
Reddit effectively has this information buried in its database. But they don’t really use it, preferencing issues of the day and recycled tik toks apparently.
The biggest problem, doing that, is that you increase the pressure from bad actors to websites that are trusted by lots of people.
So, for instance, the more weight you give to sites that are quoted by Wikipedia in your search rankings, the more content farms will have incentives to sneak edits that link to their sites.
There are ways to counter that (eg moderation), but in general, defense is more expensive than offense.
Right but the idea is that you don’t use sites as sources of trust, you use people. Content farms won’t be paying your friends and family to shill websites to you.
I think you are applying a solution that would work in a cooperative game. The problem is that this situation, like spam, is an inherently adversarial one - and one where one's adversaries are very motivated and have substantial (substantially more?) resources than you at the outset.
I like this idea of incorporating trust and reputation. As for the curation of websites not scaling, some time ago I thought about the possibility of a search engine where the user supplies a list of trusted websites (for example, university websites, blogs of people they admire), and the search engine ranks pages based on link distance to these websites.
This reminds me of Cory Doctorow's "whuffie" from Down and Out in the Magic Kingdom (man, that title takes me back!).
Whuffie is, roughly, money determined by your social interactions. More importantly, others also have a queryable score that's weighted according to who you esteem highly - this sounds like what you're proposing!
I was thinking something similar recently, and I also believe it's an idea worth exploring. Something else to add to this conversation. There's an obvious difference between two cases in which a trusted person trusts a url:
1) Single contributor website (blog, personal page...): It seems that we could spread the trust the whole website in the algorithm (at least more than for the next case)
2) Multi contributor website (forum, newspaper): It seems the trust should be given at an URL level
Something worth delving into if we are designing this trust based search engine in real-time here at HN ;)
In that post, I don't address the reputation management aspect as much, but it's central to making the whole thing work, and I think crowd-sourcing and a well-conceived reputation management system that can influence results are good next areas for exploration.
I think the Keybase project would have been great for providing the "authentication" part of this solution. To bad it died on the vine of the Zoom purchase.
> Then, separately, you can manually add people you know personally to your "network". You trust them, so anything they trust is also something you might be able to trust.
It doesn't matter how many sockpuppet accounts a marketing company creates if you don't click the "trust" button on any of them, and if the system is designed so that your local trust network only consists of what you trust directly and (weighted) transitively.
I've been wanting[0] more or less the same for a long time now. Ideally the dataset should be public (and distributed) so that you could find bad actors and eliminate them from your network (i.e. when you see a bad result, you can figure out where it came from into your feed), as well as run your own tweaked algorithms. WoT alone is great, but it gets better still when you can tweak the algorithm.
At that point, it doesn't need to be people you know personally; it just needs to be identities with a proven track record. If you can manage multiple identities, you could make ones e.g. for the explicit purpose of presenting a blacklist of bad sites you've found, or for providing a "front page" to a certain subject. Anyone is free to trust your identity or not, and choose exactly how to rank its actions.
This whole thing could be expanded to tagging and other metadata as well as comments, and so on.
A system of distributed trust like this is the makings of decentralized moderation. Decentralized curation. I've also had this idea on my mind for a while: https://news.ycombinator.com/item?id=21975294
You could also incorporate "reputation" of the author. Basically have a real person, the author stake their real identity on the quality of the blog post they wrote.
This feels like the underlying issue. Google may have stayed the same, or even slightly improved.
But the web, in the sense of quality:crap ratio, has gotten substantially worse.
This flood seems like the ultimate manifestation of turnkey hosting solutions.
Imho, we could do worse than reviving an idea from email's early days vs spam: negligible per-use charging. The idea was to tax emails at $0.0001 (or somesuch). Insignificant for actual users, but financially decimates high-volume, low-value spammers.
The web is like that because content farms are optimizing the pages to be found by Google and Google doesn't know how to filter them out, so we really can't treat it as a problem independent of Google itself.
There is real content being drowned out by autogenerated SEO crap. I tried looking for rice cookers which didn't have non stick coatings, they do exist and some blog posts which talk about them but the top results are all stores which have just a generic category for rice cooker but generate 200 duplicate pages with the title changed to exactly match whatever your search term is. So it says "Ceramic rice cooker" but shows their generic listing of PTFE rice cookers.
Google search is constantly improving but the SEO spammers are improving faster.
Fair. The fact that Google exists + the fact that Google serves a huge amount of traffic + the fact that Google is unable / unwilling to filter out content farms = incentive to content farm.
If there were no Google though, we'd likely have the same thing.
So I guess the only reality that avoids incentivizing them is one where (1) there is a massive traffic generator & (2) that massive traffic generator severely disincentizes content farms.
This has been happening a lot with StackOverflow and GitHub pages lately. A lot of the times, the actual GitHub or SO link won't even be on the first page.
I'm surprised they haven't done some kind of manual pruning of junk like that, or maybe they have and it's not working... but on the surface it totally seems like they could implement something that says "GitHub has content X, and these other 10 sites are 99% the same, but we've flagged GitHub as an authoritative source so they'll always outrank the clones".
Maybe it's a fear of appearing unfair. Or maybe they secretly want to hurt Microsoft by turning a blind eye. Or maybe this is actually a much harder problem. If I had to guess it's probably #3. But as a user of search it's frustrating to find the clones ranked above the real stuff.
Yup, just found this morning that an article my wife wrote on a very obscure legal topic was stolen, reformatted, and posted on some "life hacks" sort of site. It shows up #3 in the DDG results. At least her originals are still #1 and #2.
Meanwhile I have in my inbox in the last 24h at least a half-cozen emails looking to do SEO work for my company website.
Web = untrustworthy? YUP
I'd happily pay for a serious version of 1999 Google, but updated to filter out anything advert based, and search for exactly what I want.
Search is such a fundamental function, and we've done the experiment and the advert model fails - it needs to be just another utility.
Interesting! I get all Reddit results on my laptop (Safari), but on my Android (Edge and Chrome) I get Good Housekeeping and NYTimes before Reddit. Some kind of ad? Though when I click on the three dots next to the result, it says it's not an ad. Odd.
Yes, there's a particular bug that sometimes happens on mobile with some queries. It's relatively recent; we're aware and working to resolve it, because site: really should only show content from the indicated site.
Just as bad as the auto-generated pages are company blog pages whose SEO rigged post pretends to give "help" for a problem where the main solution is of course, using their product.
Yeah, looking up technical stuff now for anything outside of very major tech is an absolute nightmare. It's nothing but auto-generated pages made of random parts of random forums posts smashed together under some weird url like tech-helb-4-yuodbajdasdasd99234029242.co.xyz.com.org.
I've clicked on a few out of curiosity, immediately recognizing they were garbage from the description text, and it's just endless SEO links and completely random text.
You'd think one of the richest companies on earth could make a freshman intro to CS-level spam filter. If they can't, then they truly do hire the most incompetent people on earth. If they won't, then they hire the evilest people on earth.
Yeah, those pages are definitely auto-generated. Static site generation makes it possible for those types of pages (I call them "shims") to jump to the top of the results list. I wrote about it here: https://zestyrx.com/blog/nextjs-ssg
I don't see what static site generation has to do with it. You can spin up a huge number of shims even more easily with a dynamic site and a DB with a list of all the messages you want shims for.
SSG gives you the best SEO because web crawlers can understand the structure of a static page with pre-populated content better than a dynamic site that relies on Javascript. The pages also load much quicker and score better on other metrics that Google uses. I would go so far as to say that SSG is a must for major publications or anyone serious about SEO. This sort of data-driven SSG is exactly what you are thinking of but it happens at build time so that the build output is static content that can be put behind a CDN.
Absolute nonsense. You don't need a static site generator to serve up an HTML page instead of rendering with JS. People have been doing dynamic pages since before JS even existed.
The difference in speed between running a simple script and reading a file is not large enough to noticeably affect SEO.
Most of these sites are probably just using PHP, not any static site generator.
I concur on the looking up technical errors bit directing you to auto-generated sites!
I was recently trying to troubleshoot a very basic error message for Linux and was getting results and webpages that would list the error message in the title in some way, but then give instructions on "First, open up device manager", "Click Win+R to open windows command prompt", etc. Lots of untrustworthy ads. Different URLs, almost line-for-line identical webpages.
This was something like the top four search results (that weren't sponsored ads).
1) if this is from the project's own site, it's good.
2) if it looks like an archive of the project's mailing list, it's good.
3) if it looks like an internet forum, it might be good, or it might be just another poor soul asking the same question.
4) if it's on StackExchange, it's like on the forum, except your chances are slightly better. Karma must flow.
5) if it's on Reddit, it's like on the forum, except your chances of getting an answer are worse.
6) if it's a blog of some geek, sometimes it can be better than 1) and 2), or you might just get a straight answer.
7) in any other case it's most likely a SEO farm. Run.
If I have a linux problem these days, google usually gives me the relevant piece of source code on Github from which the error message originates. Like, you're a big boy now, go figure it out yourself.
It seems we're back to the early 2000s, when search engines not so much specialized in topics, but leaned heavily towards one type of content or the other. Holy hell, maybe one day Reddit's own search will be good enough so google can be ditched for good!
It still exists, the last app update was uploaded yesterday, but it seems most of the users didn't stick around after the initial hype died down. I guess the early focus on monetization and trying to turn it into payment app didn't help.
I want Google to allow me to specifically include/exclude mirrors from my search results. "Only show my the original source of this content", or "only show me mirrors of this content".
I don't want to see the same result repeated 5 times across different stack overflow mirrors.
It almost immediately upon the real adoption curve (this seems like October November 2020 anecdotally to me) just absolutely filled to the brim with NFT shills, cryptocurrency pumpers, and “self actualization with this one weird trick” promoters of their MLM. But early 2021 it was impossible to find any rooms with anyone discussing anything but these things.
Yeah, and unfortunately the only rooms not run by scammers were filled with people talking about anti-semitic conspiracy theories. What a cesspool it became.
I have noticed that a lot of tech and startup clubs on Clubhouse were created by users from India and Iran. Today the clubs are still dominated by users from these countries.
Nothing wrong with that but that also indicates the usual clickfarm spammers from developing countries had unfiltered access to Clubhouse from day 1.
This might sound elitist but it's probably a good idea that C2C apps get their first batch of users and community leaders from high income countries before branching out to the rest of the world.
India is a young country with a 1.4B population, with a lot of engineers. I know the quality can vary from gemstones to tombstones but there is an active tech community.
>This might sound elitist but it's probably a good idea that C2C apps get their first batch of users and community leaders from high income countries before branching out to the rest of the world.
> it's probably a good idea that C2C apps get their first batch of users and community leaders from high income countries before branching out to the rest of the world.
Unless they're targetting local communities with some sort of culturally relevant format, its good for their bottom line to target the most profitable customers of advertising anyways.
Google used to be better at filtering out garbage content like this. They have resources for detecting low quality content (e.g. all pages on this domain follow the same content-free pattern).
I suspect that doing that wouldn't drive ad revenue up, so they don't bother.
I had to search crates.io . Let me tell you. It's not the pinnacle of search.
What I searched was `fast bitset implementation`. My results consisted of drill bits a stack overflow questions and a Baeldung article on hashSet vs long[]
> "Step one: try turning your computer off and turning it back on again. This is usually a good first step, and you'll be surprised at how often..."
This seems like a natural result of optimizing each search for revenue. Think of a search to solve an error message on your computer. There's a very small number of vulnerable people who are going to spend money as a result of that search, so optimizing for ads would mean tailoring the results specifically for those people, pushing them to sleazy sites where they might spend money on some kind of antivirus scam. The results are worthless to you, but who cares? You're worthless to Google when you're doing that kind of search. Try searching for something that people in your demographic spend money on, and the results will likely look better to you.
Might the inevitable arms race between bot writers and bot detectors be the missing accelerator for a general AI that has a predilection for top 10 white label brands of generic consumer products?
Post truth society has arrived, trump was a symptom, not the cause.
Combined with AI imitating speech and deepfakes, and technology of inplanting false memories, we will have the matrix, just not the wau we expected ^ ^
Google used to be really, really good at finding exactly what I told it to find. Nowadays, it's turned into the yellow pages; sponsored content from businesses trying to sell me goods and services.
Can people suggest good alternatives or search patterns for certain categories of information or search types?
Some of the search patterns I currently I use:
* Youtube for product reviews and demos, entertainment, music and educational material.
* Google with site:reddit.com at the start for questions best answered by other humans; crowd-sourced answers, authentic replies from mostly real people.
* Google with site:news.ycombinator.com if I want to find "forum-like" discussion on topics I'm interested in.
* Google Image search with site:amazon.co.uk when looking for niche products I need to buy, because Amazon's search is so incredibly broken and game-ified.
What I'm having a heck of a time finding is technical content; long-form programming tutorials, deep dives into academic concepts (I do a lot of signal/audio processing and search for blog posts related to these topics), circuit schematics, electronic engineering content. These used to exist on enthusiast forums 10-15 years ago, but Google often no longer surfaces hits from these forums, both because the content is old and the forum model is dying. Reddit is the "replacement" but it plagued with low-effort "look at my thing" posts that help nobody.
In my experience, the forum experience is far from dead, but it's effectively impossible to surface in a search engine - any search engine - unless you know the name of the forum.
Oh, and the content must also be "fresh". If the content isn't "fresh" (which most of the best forum/blog posts are not), nobody shows it anymore. I can search for a specific blog post using a verbatim quote, but the result (if it exists) is buried under 10+ pages of "fresher" content, no matter how disconnected it may be from the search.
The forum experience is dying. I spent about 4 years of my time in-between Google stints working on a searchable feed for forum sites. Finally gave it up when I realize the extent to which the forum scene had died and moved to Reddit & Facebook while I was working on the project.
The root problem is that attention has gone from abundant to scarce, and people already have their habits. That makes it really hard to build a new forum site and attract an audience that's willing to type your URL in every day (and if they don't visit daily, forget about building a viable community). Forum hosts like Facebook and Reddit don't have this problem - you can view your Buy Nothing Group and Moms of Springfield posts interspersed with your feed of friends, or your r/factorio content interspersed with a steady stream of r/AskReddit.
There's also emerging technological barriers. If you don't sign up for CloudFlare, as a new website, you're going to get hosed - but at the same time, CloudFlare makes it basically impossible for any new search engine other than Google to spider the site. Ditto security patches, and keeping software up-to-date. Most people don't want to deal with sysadmin stuff at all, particularly if they're trying to build a community as a hobby. So that pushes people further toward hosted solutions with a turn-key secure software stack, which is Facebook and Reddit.
> The root problem is that attention has gone from abundant to scarce
I don't think that's necessarily true.
I think the root problem is that running & using a forum is too difficult. That is why centralized forums (like you mentioned, reddit and facebook) that handle it for you won out against decentralized forums run by forum members.
Even before facebook/reddit/etc forums tended to live or die by individual effort of one passionate system admin dealing with all the hosting, updates, accounts, and spam until they get fed up and the forum closes because they can't find someone else to take the keys.
One of my favorite niche forums is https://archboston.com. It has years of deep-dive discussion from passionate users about the history and progress of Boston area infrastructure and real estate development projects.
For a while there the site was up but not allowing new accounts to be created -- someone was paying the hosting bills but didn't have time to do any admin tasks. Thankfully, someone else stepped up and people post new stuff every day (albeit with banner ads at the top of each page now, which is honestly not too bad)
Perhaps the experience is dying, but the wealth of curated information in forums is still there, is still incredibly valuable, and in some cases is still being added to. Here's one example I used extensively recently; it was sent to me by a colleague, since I never could have found it via a search engine.
Aside from prioritizing Ads, I actually think the root of the problem is sort of the opposite: information has simply gone from scarce to too abundant. Finding information you need is like trying to find a needle in a haystack. To solve this problem, search engines like google search came into being. Initially, having multiple search engines caused a problem in itself: if you have 700 search engines, which search engine do you use? So the industry naturally ended up coalecesing into a near monopoly, as having 1 (or a few) engine(s) to use is simpler than having 700. However, the root of the problem is still growing. Now it's not so much as "which search engine to use?" but more: "what's the precise combination of text to feed the search engine needed to find my needle?" And as more information gets created, the problem just gets worse: your information gets drowned in the sea of other information and thus gets harder to search for...
Amusingly, appending "reddit" to your search is like a pseudo search engine in itself, instructing the search engine to act as a search engine of only a specific domain of information. Almost like we're back to having multiple search engines, and no one knows which one to use...
If you find a forum for a given subject, it is almost always an authoritative source filled with experts. This is especially true in engineering disciplines.
It's unfortunate that Reddit and social media took over and led to their decline, because it's suboptimal setup in so many ways.
- Reddit in the large is a high noise, low signal monetization chamber. Some subreddits have good moderation, but that doesn't stop the spill over and drama.
- You can't assume much about any given Reddior, and you won't typically form relationships or associations with them. It's pretty much pseudonymous.
- Reddit doesn't focus on authorship. It doesn't allow inclusion of images, media, or carefully formatted responses in threads.
- Reddit corporate is the authority and owner of all content. They can change the rules at any time, and that's a fragile and authoritarian setup for human discourse.
- Reddit corporate is constantly changing the UI and engaging in dark patterns to earn more money. This flies in the face of usability.
Forums should make a comeback. It would be better if each community had real owners and stakeholders that had skin in the game rather than a generic social media overlord that is optimizing for higher order criteria that sometimes conflict with that of the community.
But forums have problems too. They should be easier to host, frictionless to join, easy to discover, and longer lived.
Another way to think of this: every major subreddit is a community (or startup) of its own and could potentially be peeled off and grown. You'd have to overcome the lack of built-in community membership and discovery, but if you can meet needs better (better tools for organizing recipes, community events, engineering photoblogs, etc.), then you might be able to beat them. Reddit can't build everything, just like Facebook couldn't.
This is depressing. Good information is useful for far longer than a carton of milk in your fridge! And a lot of that new "milk" is apparently made of chalk and bilge-water.
Yes, of course they do, the alternative milk market is growing fast but most people still go for cow milk if they want milk. That's why grocery stores still devote a ton of space to cow milk.
I'm relatively well informed but I always just assumed oat "milk" was an inferior substitute marketed to the actively or wannabe lactose intolerant. [EDIT: and vegans of course.] (No idea if "wannabe lactose intolerant" is really a thing, but i'm thinking of the way gluten sensitivity became a faddish self-diagnosis for a while.)
I still don't drink the stuff, but it's only dawned on me in the last year that there are other reasons, such as environmental concerns or ... actually, I'm not sure. Opening two tabs to Google "oat milk why" and "oat milk why site:reddit.com" now, which conveniently makes this relevant to TFA :)
I use cow milk in my coffee and almond milk in my bland cereal of choice. I’m not lactose intolerant, but too much milk definitely feels “heavy”. The almond milk plus some sultanas substitutes nicely and is very cheap. Mostly its not for the taste, it just makes breakfast easy and efficient so I can focus on fancy stuff later in the day.
Here (mid-tier city in the US) it's more like 80/20 or 90/10 at any normal grocery store, and I suspect there's higher product turn-over for dairy so the actual sales figures favors dairy more than that suggests.
Of course it depends on where you in Sweden you are. The super market in the the more affluent part of a large city where I live is about 50/50. Out in the 'sticks' where my parents live it's much closer 80/20
The entire information ecosystem has internalized a bias toward "freshness." It's even really strong in software. Evidently code is more valid and correct if it has recent GitHub commits.
Almost no software just works if left unattended for years. If its a library it means it will likely not work with the latest versions of everything else. Your bug reports will go unattended.
People also have a lot more tolerance for missing features or issues if they see it improving regularly. While getting something unsatisfactory as the last and final version is not nearly as acceptable.
I have built Unix/Linux stuff from the 1990s with no code changes. Programs that do well defined things generally have a long shelf life. Even X stuff often works, though it can look bad on modern displays.
Math kernels, codecs, and so forth can more or less live forever.
Software is the most toxic environment possible for freshness bias. Every month it seems like there's a new framework/ecosystem/whatever and everyone's migrating to it and you're behind the times if you're not using it.
This is only if you bother chasing the absolute freshest trend. Things like React, Ruby on Rails, Node.JS, etc have been the standard and most popular tools for close to 10 years now and aren't going anywhere.
Anyone know the origin of the fresh rule, and the purpose? It makes sense in some niches but in others it is so obviously bad I wonder why Google added it
Doesn't cloudfare do this? (extremely extensively imo - just about every single click is a captcha of some type sometimes)
As much as HN despises blockchains because of crypto, (no, that's not the only use of that tech) it does seem like verification of websites could be a decentralized consensus - along with DNS as well -and user verification could be a portion of the consensus of website quality.
This could even help with making self hosted search engines more accessable
Hmm, or you outsource the trust problem to your users. Let them select which domains to trust or who to trust to weight trustworthiness of domains for you.
Imagine that user A can create a list of domains with trustworthiness score. User B can then use that list by going to user-a-awesome-curation.koogle.com?q=nice+shoes
It might create filter bubbles but it would be transparent filter bubbles. You could even wikify/open source the curation.
A lot of this could be solved if we could signal intent or context before searching. But that would require that you know how to use the tool which is a gargantuan user barrier from Google's point of view. Meanwhile, we have to hack about trying to signal context.
the forum experience has effective been totally replaced by either discord or subreddits, or any other kind of self-moderated social media group you can think of.
Its a plus in minus in a lot of ways but the biggest con is that its just straight impossible to search a discord log effectively.
I've been on the Kagi beta test for a few weeks now and, for the kind of searches I mostly do, it seems to be a massive improvement on Google. Strongly recommended.
I've also been using Kagi for ~1 month and god can I testify for how fantastic it's been. You have to TRY to find blogspam and the allowance of blacklisting domains plus some other handy search customization features make it an absolute joy to use.
It may lack "instant answer" widgets or other fancy search engine features but it gets the actual "search" part of the equation so right that I find it astonishing how I ever used DDG/Google in the past.
Yeah I've flicked it on a couple times but rarely notice much change, though honestly it's not a search engine feature I've come to desire, as even if you get a widget the majority of the time it may parse incorrect data or not even show what you wanted (which happens a lot with google's widgets IIRC).
Same. Over the years I've trialed most search engines out there, but always find my way going back there after at most 2 days of trying them, because I end up adding "@google" before every query anyways because the results are bad.
With Kagi most of the results are what I'm looking for. If they are not, I'll still try "@google", but so far with very few queries Google's results were actually better. The biggest drawback is worse "smart cards" results, but I hope they keep those optional/unobtrusive anyway.
The strange thing is that the feeling Kagi gives me, isn't even unknown. It just feels like Google circa 2010.
I've signed up for the beta, but it's hard to shake the feeling that signing in to a search engine is a mistake. "We respect your privacy", "we'll never sell your data", I've heard these claims before and they've almost always been lies. They can tell me that they don't maintain an eternal history of all my queries, but how can I ever verify that?
Kagi ultimately will be a paid service. As I've noted in the context of other services (email), by providing revenue in the form of paid services (and only paid, no freemium tier) the service doesn't have to implement ads at all, and thus can skip the pressure to deliver more and more data in exhange for better rates.
I guess it comes down to trusting that they'll sit on that big stack of user data, exactly the same stuff Google used to build a trillion-dollar company, and continue to decide day after day not to sell it. Will they continue to hold out if VCs get involved?
Yes, I'm aware that all the same arguments apply to DuckDuckGo, and that Google & Bing already explicitly do sell this stuff, but the stronger the promises, the more I demand to see proof.
I think part of it is also an implementation pressure. If you put your dev work into building a system around managing subscriptions and processing user payments, its less easy to flip a switch and start siphoning data to 3rd-parties when they come calling.
In theory, this would be visible to end-users as a halt in feature roll-outs, because the dev team has to pivot to building ad-tech.
I hope to god VCs don't get involved; if they do I'd be the first to bail. I'm hoping that the revenue model makes VC money allergic to them in general. Bootstrapping is preferred for this type of service (see also: Pinboard, sr.ht)
Umm... Google and Bing track the hell out of you whether you sign in or not. DuckDuckGo claims not to but the sites you hit through it certainly employ tons of fingerprinting.
Write and ask them. GDPR requires that users can retrieve all stored information associated to their profile/person. Most large internet companies have this functionality.
I love this search engine, it gives me the same feeling that Google did when it became a thing. Their business model after beta will be that users pay to use it, and it has no ads. This is a very encouraging sign, and personally I'll be willing to pay for quality search without ads. I hope enough other people feel the same to make Kagi profitable and functioning for years to come.
Looks interesting, but am I crazy for thinking that $10/month is an insane price to pay for a general purpose search engine? Surely Google wouldn't making anywhere near $10/month off of me even if I disabled adblock.
Those two sentences have no connection. You can gain a huge amount of consumer surplus, even as the seller reaps almost none of it. Google's gain from ads bears little relation to your gain from Google. (This is why people are so much better off in markets: it's actually very hard for a business to get more than a small fraction of consumer surplus.) As it happens, when people try to estimate your gains from a general purpose search engine, it usually comes in at like $100+/month (search 'willingness to pay search engine' and think about how long it would've taken you to find that in a physical library, and how you wouldn't've bothered in the first place because such a search would be impossibly expensive). So, $10/month would be a steal compared to not using a search engine at $0/month.
> So, $10/month would be a steal compared to not using a search engine at $0/month.
But that's a fantasy dichotomy as long as free search engines exist, and there's plenty of them. If you could somehow change the world and wipe them all out in an instant, I guarantee that people would scramble to provide alternatives. We will never live in a world where your only option is $10/month or no search. (Free & open source search engines already exist and you could host one at home or on a cheap VPS; there are also P2P search engines)
Of course people would scramble, but it has nothing to do with how much value you can screw users out of using advertising, because even if ads were worth $0 revenue, search is so valuable you could just plain charge users. This is why there is essentially no relationship between the value of search and the ad revenue. The value of the ad revenue could be $0, and the value to the user would still be $100+/month.
And because the consumer surplus is literally an order of magnitude or two more than the subscription fee quoted, that is prima facie a case that a subscription search engine could have a marginal benefit of >$10 compared to the free ad-supported engine. It, or the subset of searches you opt to use it for, only needs to deliver a little more value to be worth it. Quite aside from the problem of Google Search being increasingly jammed full of ads, wasting your time, or any distortion of ranking, people just plain dislike and avoid ads (https://www.gwern.net/Ads).
The value proposition is in surfacing a result that you wouldn't already have from the free search engine. Your personal calculus will of course vary but lets do a basic business case with the following assumptions
Free search engines work 95% of the time for your employees searches
Kagi can get a result in half of the remaining 5% (this is definetly the biggest assumption and I haven't had enough experience with kagi to say if this is realistic)
Your employee does 1 search a day and 30 days in a month (so kagi gets you 0.75 more completed searches a month).
It takes an employee 15 minutes to search manually through documentation or come up with a solved algorithm from first principals when the search fails.
In that situation your employees time needs to be worth less than $53.33 dollars an hour for the $10 dollar plan not to break even.
So play with the numbers how you want to make up your own mind but it does seem reasonable to argue there's a market for it at that price. Personal use where missing a result could have no cost is ofcourse another question.
Sure, I'm not contesting that there's a market for it. Going along that line of reasoning, there's also a market for a group of experts you can phone and get an answer from at $100 / hour, and so on. But let's not push the goalposts too much :) My personal calculus says it's not worth $10 for me (I don't rely on search much for my work).
The price is crazy only because you're used to not seeing the price you're paying (ads). I spend $10 on things way less valuable very often. A good search engine is at least as much value as intellij to me, and my company pays 5 times more than that per month for intellij.
The problem is that ads price discriminate. Google may not get much money off of you, but there are other users that are very valuable (think of a manager in a billion dollar enterprise searching for a subscription product to buy). Would you be okay with it costing 0.05% of your income?
Of course it will be hard to compete with ads as business model if the alternative doesn't allow for price discrimination.
> Perhaps I'm under-estimating how computationally heavy search is.
It's because they aren't rolling their own search, they pay Google and Bing to do the search for them (via the Google and Bing Search API's which are charged), mix in a few results from their own crawler, and then reorder the results.
So they will always have a higher cost base than both Bing and Google, because they are paying for 3 different search indexes (including their own), plus Bing and Google's margins on the API, plus their own infra costs.
(Now if this is a sustainable model or not is another question...)
Kagi has a "consumption" section where they show how much your searches cost to perform.
I've done 20 searches this month (I haven't switched it to default) and it says I have incurred $0.25 (between $0.01 and $0.02 per search) so it would seem that it's very expensive for them to provide results at the moment. It would absolutely be unsustainable at a few dollars a year given these consumption numbers.
I'm not sure how they are counting the searches, mine is indicating more than 100 searches per day and I certainly don't access their website that much (maybe search suggestions count too?). Their results and overall product are much better than Google but I won't pay 30 usd per month for that.
Every time you make a 'search action' it charges you, including changing categories on the site or changing the filters on the search.
Lets say you search for "Photo Of A Mother" * Then you click "images" * Then you click "Sort by Recent" * Then you click "Licence -> Public" * Then you click "Size -> Large" * Then you click "Size -> Extra Large" *
Every time I have put a star in the above is a time where you would be charged a search (so the above would be charged as 6 searches). It's the same thing with switching to news, applying filters, blocking a site or boosting a site etc - I've validated this on my Kagi account by clicking actions and seeing what it does to the billing, and just by using the search engine and using the lenses feature for example you can quickly rack up loads of searches.
Now let's say you search for an error while you program, visit the first site, it's not got what you want so you click the back button and then visit the second site, that's not got the right answer so you click back and visit the third site... That's currently counted as 3 different searches rather than 1 search. If you open them in different tabs it's counted as 1 search though.
And with all this then you are suddenly over 25% through your daily allowance on the $10 plan. Even choosing to 'block' a site in the search charges you to block it. They are talking about charging $0.015 per search if you go over-quota (which is something like 20 searches on the $10 plan), but as far as I can tell if you are using it moderately heavily you will blast part the 20 searches and could end up with an eye-watering bill.
I think the team are great, and when I was on discord they were really receptive, but I ended up giving up on Kagi after trying to use it as a daily driver as I figured they wouldn't be able to find a good enough monetisation strategy for my level of usage (after hearing discussions in the pricing channel). The product is good, it's just that they can't offer it at a price I can accept (and I suspect they can't offer it at the moment at a price that the market can accept, unless they can reach a deal on API licencing or roll their own search).
That pricing model is insane. I think it will remain a niche product only used by the wealthy unless they can get it down to somewhere like $5/month for unlimited searches.
Yeah, it's not that hard to get results as good as Google when you are paying Google and Bing to give you search results.
It IS hard to get results as good as Google at a price where people are willing to pay for it.
IMO I think the issue with Kagi is that their current architecture has made it easier for them to get great results (as they are just taking their results from Google + Bing and adding their own special sauce) but it makes it much more difficult in the long run to be cost-competitive.
Their nearest competitor (Neeva) does offer $5 a month search, has a free tier, and rolls their own engine, so I don't really see how Kagi's current model is sustainable unfortunately (but maybe they will be tremendously successful and I will look back at this post and feel silly!).
Presumably because, as they move out of beta, they want people to start paying to use their search engine. Thats the only way they can afford to be ad-free.
I'm 100% happy to pay for an ad-free search engine that doesn't sell my data. I don't really like having all my search terms being linked to me via an account, but I suppose that's already happening anyway. I guess I picked the right day to stop looking at clown porn.
They could do the SaaS model and give you an API key or unique URL allowing up to 10000 searches per month which you could share with friends/your company
That's probably a worse business model, but it would be really interesting to hear what other monetization ideas Kagi considered or is considering.
Works at an enterprise level (e.g. exchange rate providers) but would be a bad fit for a B2C product.
Managing customer expectations on api key usage (esp. if that key is publicly visible e.g. URL parameter vs. HTTP header) is not worth it unless you have higher-priced products.
Also api keys would mean you might have to prevent re-selling, etc. Furthermore, they could still analyze api key usage to get the same historical data on you as if you logged in.
That's missing the point. If the same API key is used by 1000 people at a company, you don't have the expected 1:1 mapping of search terms to individuals with an account.
An individual account which isn't permitted to be shared is different.
You could sell tokens that get used up after each search or expire after certain amount of time since first use. Browser extension could store tokens and provide them to website as needed in random order. Tokens could be resold so no tracking by payment processor
If you buy from reseller they have no way to correlate payment details to search queries. With subscription, however, service provider have your payment details and knows everything about your queries so it can build a profile on you.
As a bit of a weird hobby, I like to read up on right wing conspiracy theories. That means I do a fair number of searches for specific terms and people mentioned in fake-news facebook/forum posts.
Google seems to slowly oscillate between thinking that I am a right wing loon, and thinking I am Joe Public who must not be shown misinformation. That is, sometimes google is perfectly willing to vomit forth results from the propaganda mills, even when I'm not specifically looking for it, and other times I can't get conspiratorial-minded results even when I am making an effort to find them.
This most frequently manifests itself when I am looking for sources for claims that I know exist. Like if I remember reading an earlier conspiracy that has just been invalidated, or someone posts some a video of someone reading a blog post. If google has decided I am an innocent bystander not to be shown conspiracies it can be nearly impossible to track down the original blog or posts about the conspiracy.
Recency bias is another huge problem with google results. Older content gets heavily de-prioritized, even when it is clearly what you want. Google is willing to give up on terms in your search before it is willing to show you old stuff. For example, if you tried to research early Ukrainian political corruption during Trump's impeachment, your results would be nearly entirely Trump-related content even if you tried to use google's date-filters and exclude terms like -Trump.
I noticed this recently when trying to find primary sources for flat earth claims. They don’t exist on Google, for me at least. You can still find them on duck duck go if you search for something like “flat earth ice wall” but Google just returns generic debunk articles.
This sounds like filter-bubbling. From what I can tell, Google doesn't have user specific filter bubble but user-category filter bubbles, and it's constantly updating the category of users it thinks you're in.
`site:reddit.com` has been worked poorly for me recently, although I've used it many times in the past. He's my most recent search (I was traveling and trying to watch Netflix, but geo-block was preventing some shows from appearing):
The entire first page is for NetflixViaVPN subreddit (not linking to avoid SEO). They have a stickied post that seems to shill two VPN providers I haven't heard of. This is plausible, as maybe Netflix hasn't either... The stickied post has comments disabled, so it's hard to tell. Then if you click other posts, a bot auto-links the stickied post, but everyone is making different suggestions that may imply the stickied post is wrong.
Interestingly, the same search on DuckDuckGo only has three posts from that subreddit. This better matches what I wanted! The first one I'm seeing is:
This seems much more plausible. All those comments suggest a provider I've heard of and that I've heard other people mention IRL. Google seems to rely too much on the URL or the page header, so it's stuck in a single subreddit.
imo, google is still king, but you have to be a bit of a power user. You're already using `site:` which is good if you know exactly where you're looking. If not you can use `related:` in the same way. I find using `-something` to remove terms the most useful. I'll search for something (usually an error message) then add `-react` (and mumble "ffs not everything is react"). Then if I still see things I DON'T want add more `-` to the string.
It's not GREAT that you have to do that, but it's pretty functional and certainly better than going past page 1 of search results.
The major problem with this that I've experienced is even if I use operands like + and - to specify or remove terms--more often remove--Google ends up using a synonym in place of that word that means the same thing.
So if for instance I'm looking up info about ADHD meds as an adult, I might get tons of articles about childhood ADHD since that's where all the research is. I search Adult ADHD meds, I still get articles about childhood ADHD. So then I:
and I still get crap blog spam that's probably related to teaching or raising children or some other bullshit like warning about the dangers of addiction or something, and never information about my ADHD or the meds for it.
It's not GREAT is the understatement of the decade.
I've had some luck inserting "forum" into search terms to find real human content. Mostly when trying to find technical info about cars, but may apply to other fields.
>What I'm having a heck of a time finding is technical content; long-form programming tutorials, deep dives into academic concepts
github search is good for that. search for 'list of awesome anytopic'/'curated list of anytopic'/'list of anytopic' and you might find a repository with a curated lists of links on anytopic. (search box on the main page of github)
You might also want to check my side project: I have a search tool / catalog of duckduckgo !bang operators, i am hoping that it allows for better discoverability of specialized search engines.
Since there's only a handful of sites you target your searches at, it would be nice if you could just have your own search engine that focuses on those few sites, and perhaps crawls a little deeper.
I've sometimes thought the death of Google will be the self hosted search engine.
I've found my jobs' internal (social) message boards/mailing lists/Slack channels/etc to be great resources as the only contributors are those who work/worked at the company. Your (ex)coworkers presumably met/meet a certain competency bar and are less likely to spam. At larger companies there are message boards/mailing lists/Slack channels/etc for nearly every topic.
For local information, I've found forums for local sport teams to be great resources during the off season. Posters are often happy to engage in any sort of chat during the off season. Even if you haven't gotten to know the frequent posters during the sport's season you can use the (usually highly visible w/o any additional clicks) account age/# of posts/"karma" as a proxy of posters' trustworthiness. note: If you don't normally contribute on-topic (i.e., about the team and sport) posts, I would only search the forums for your questions and not post off-topic questions as that'll get you quickly banned.
It's particularly awful on mobile where you get Google's "smart" cards which can be ads, followed by ads then the actual results which are mostly SEO trash. Trying to find support for Google Fiber routers was nearly impossible because Google just tried to interpret what I wanted as signing up for Google Fiber and just overwhelmingly suggests that. It gets even worse on Youtube where after like 10 results for what you typed in they just show you "things you might like".
For technical content I had great results from using safari books online in the past. Having most tech literature an easy web search away was super convenient, because typically the best treatment of any subject is in book form. The downside is that it is expensive, so when I switched employers I lost access and I wasn’t willing to pay for it myself.
Change your search strategy. Most forums require a membership to view them, and most long form posts are on personal websites. Google can't or won't serve those. You have to navigate like it's the old web. Find a good place to make landfall, read old posts, ask around, and follow all your leads.
> There’s a fun conspiracy theory that popped up recently called the Dead Internet Theory
I think we're well on the way ...
Was recently pretty shocked, searched for "gas heating repair" and got back at the top some sites with my suburb name in the title. Naturally I thought, wow, if there is a local place I should go there. Clicking into it, it has everything about my suburb - a picture of the local park, and whole paragraphs of random text containing bits and pieces about the local area interspersed with odd sentences about gas heating ("Cold mornings in XXX can be confronting without effective heating" etc). The text kind of makes sense but also reads like it was generated by GPT3.
Of course, then I realise, this is all SEO. They have generated a page like this for every suburb in my city. There are tens of thousands of such pages they are hosting. The most shocking thing is this is a small time gas repair dealer. They clearly don't know how to do this, they've gone with a low budget to an SEO firm who has effectively generated a giant plume of toxic content into the web atmosphere, all to create a marginal benefit for this one small company.
If a small time low budget unsophisticated company can do this, then I have to assume it's happening everywhere. On a mass scale we have giant smoke stacks all over the internet spewing toxic plumes into the atmosphere. And the humans are gasping trying to find the small bits of remaining breathable air.
Yes you are right, almost every business with an online presence is generating vast amounts of garbage which exactly targets a huge range of specific keywords. From the search engine perspective, the page is exactly what you are looking for.
I am curious, did this small business actually provide the service you were searching for or not? If they do, then it sounds like it was ultimately useful.
Ah, young whipper snappers, everything old is new again, and clearly the world is always getting worse.
Well, some things are (reverse image search, ease of accessing 'Cached' pages -- now I have to go to archive.org Wayback, etc), but forum search has always been bad.
Long before Reddit was big, USENET/DejaNews and forum software like PHPbb/UBB ruled supreme (and before Markdown there was UBB Code). Google, despite owning DejaNews, did not often surface links into USENET content, and a lot of forums, for whatever reason, were not indexed by Google. For example, I used to spend a lot of time reading the latest on PC/3D Hardware stuff on Beyond3D, Overclockers, Rage3D, etc and I almost always had either use site specific search (dejanews.com or say, PHP BB's built in local search), or I had to add site:beyond3d.com for example.
And is a large amount of confirmation bias going on in these Google threads that appear. Some people make assumptions that their search patterns are representative of the billions of searchers ("argh, I searched for pytorch k-means and a GitHub wrapper site appeared!") and that their experience is a representative sample, while others focus only on what has gotten worse, and not what has gotten better.
What's clearly gotten worse is webspam. But while it has degraded the Googlee experience, it's not clear any of the other search engines are any better at filtering it out, except by luck because perhaps they don't crawl as many sites as often.
I think this is very important to note and I agree.
Whether or not google search right now is as good as it could be should not be the main point of discussion.
We have to remember to acknowledge that the web google is indexing now differs drastically from the web it was indexing 20 years ago. Web pages are now less likely than ever to be freely accessible plain text put forth in good faith for public consumption. Google (in addition to dealing with big walled gardens designed explicitly to hide content from google) is trying to sift through basic spam, industrial scale SEO exploitation, and nation-state cyber warfare.
Bitching about google search being bad almost feels like yelling at the canary in the coal mine when it passes out.
The issue (at least for me) is that google is no longer actually searching for the thing I ask for, and it's being blatantly disrespectful of users who cared enough to learn how to actually use the search features.
Quick example from today? I did a literal two word search - gulp admzip - and while the result are okish, an increasing amount of space is taken up by results with this handy little blob at the bottom:
"Missing: gulp | Must include: gulp"
"Missing: admzip | Must include: admzip"
WTF are they smoking? I asked for two fucking words, and the top result doesn't include one of them. Then the second result doesn't include the other.
So then I add quotes around the phrase I want "gulp admzip" because I'd really only like to actually see results that include that EXACT phrase, and... drumroll... IT DOES IT FUCKING AGAIN: "Missing: gulp | Must include: gulp"
And that literally has nothing to do with the quality of the items it's searching, and everything to do with Google deciding what I meant - Clearly I meant the npmjs.com package adm-zip, because that item gets vastly more views than any of the real search results.
I couldn't have possibly meant to restrict the search to the actual fucking phrase I told it to search for, because there aren't that many results, and they don't get many views.
I highly agree that "Missing", "Must include", has to be one of the worst hijacks of search functionality I've ever seen. Please just respect my search terms, as it doesn't fully respect them even when I put them in double quotes! It takes far longer to scan my eyes across the results dump and then retroactively see that my results are absolutely not what I am looking for, than to just see that there aren't many results for my exact query.
Decreasing the feedback loop time is essential to modifying my query quickly so that I can eventually find what I am looking for.
The "Missing", "Must include" pages have always, always, ALWAYS, in every case, never given me what I am looking for. If it did, then I would have just taken out my search term.
This is the exact angle of discussion I'm saying we should try to avoid because this behavior from Google is an effect of an underlying problem with the entire web.
Most sites that include the exact phrase "gulp admzip" are empty spam of regurgitated word lists of every build tool or dev package designed to attract errant clicks that help boost ad metrics. That is why Google can't "just grep the entire internet".
Incidentally, I agree with Google here that whatever problem you're trying to solve is more likely to be an issue with either gulp or admzip and you will be better served by content specifically about one or the other.
We should avoid discussing the fact that the search engine is now hijacking my search to show me things it would prefer I have searched for?
You know what - I'd much rather just see the spam sites.
The spam sites are useful feedback that my search is either too generic, or there aren't many good hits.
Further, some of them aren't actually spam sites - I'm not afraid to click through 5 or even 6 pages of results, and I can usually visually distinguish obvious spam from content very quickly.
I can't do that if Google has removed my ability to actually filter the results to the relevant search terms, and just keeps showing me the freaking link to npm over and over again.
And the problem with the entire web is a result of incentives -- Google's algorithms don't discourage people from creating spam and clone sites. You might even say the algorithms encourage spam because they have been terrible for so long.
Have you considered that there may not actually be any good results for the exact phrase "gulp admzip"? Especially considering "admzip" is a misspelling as you admit?
That's meaningful feedback that my search needs to be improved.
Getting all spam back is actually ALSO GOOD! I can visually distinguish spam pretty quickly, and it's also meaningful feedback that my search needs to be improved.
Removing my ability to search for exact phrases is fucking BAD! I'd much rather get spam or nothing when I search for a directly quoted phrase, rather than google just start returning bullshit.
The problem is that the bullshit google returns is actually very hard to visually parse out - they're real sites that get lots of views, those views are just ENTIRELY unrelated to what I'm actually searching for. That's really hard to filter out quickly.
It's entirely possible that a person could be querying the exact variable name "admzip" for a variety results that should only be code snippets of the by-convention "AdmZip" variable name with no concern for the package name "adm-zip", which, by the way, Google will interpret hyphens as just whitespace, so it's the equivalent of searching "adm zip".
I know this could be a case because I do this kind of programmatic search all the time, and in fact I remember specifically searching the web for "AdmZip" and not "adm-zip" a few years ago.
And if there are no good results for "AdmZip", that's fine! At least I can, at a glance, quickly know that there are not many code snippets across the web lying around with that conventional variable name.
> I asked for two fucking words, and the top result doesn't include one of them
You are aware that it has always been like this? That's why the "+" operator even exists. Even 10 years ago or more (before Google+ stole the "+" operator), you could get back results that don't actually include your search terms and you'd have to do +gulp +admzip to force their inclusion>
Try it yourself, it does JACK SHIT - they still show me what they'd prefer I have searched for.
---
Edit - ok, I'm guessing you are, since on the second read I see you're joking about the google plus timing. I don't understand how that makes this better? Tooling is being stripped away in favor of showing me "popular search content" that google would prefer I see, and ads...
I'm not saying it makes anything better, I'm saying that Google has always been able to return results that don't include the search terms, that's why the '+' force inclusion operator existed. It is not a new phenomena for Google to return results that don't contain keywords, since one of the defining features of Google vs Excite/Lycos/AltaVista et al is that it wasn't a simple TFIDF (textual frequency inverse document frequency) search, many of the older search engines used a variant of that. PageRank, even in Google 1.0, probably allowed a very low ranked TFIDF result (doesn't include most of the query) to be boosted, although I'm just speculating.
I'm just saying, in the same theme of my original 'young whippersnappers' post, that a lot of things people are saying is new behavior is in fact, old behavior with a different UX.
Woah that's a weird test case! fwiw now that you've posted this, this link shows up for that search. But similar searches like "gulp adimzip" show similar issues. Is this simply a bug? Clicking on Missing: adimzip" | Must include: adimzip" now makes Google search for gulp "adimzip""
> What's clearly gotten worse is webspam. But while it has degraded the Googlee experience, it's not clear any of the other search engines are any better at filtering it out
The problem as I see it is Google has created a bunch of perverse incentives to make your page rank higher. One big problem is Google gives higher rank to "comprehensive" articles. On the one hand that would seem like a good thing right? But what you end up getting is endless affiliate articles that don't seem to be written for humans. And they are really easy to spot if you know what to look for.
A great example is webhosting reviews. Search "best web hosting" and click any of the 1st page results and you will almost always get an article that just rambles on and on and on with headings like: best web hosting for email, best web hosting for blogs, best web hosting for email marketing. To a human, it's an incredibly disorganized mess, but to Google's bots, its "highly comprehensive and authoritative".
While that may be true, it’s also true regardless of the ranking algorithm or which web search is the winner.
Webspam will seek to game whichever search company has dominant market share and they will structure their spam to overcome the filter and ranking specifics of that engine.
Considering tools like GPT-3, one could easily imagine in the limit, a spammer running a large number of searches through a search engine, finding out what ranks high, and the training a generative model on that dataset to produce similar articles. Auxiliary signals like inbound links and DNS records they can also usually work around by purchasing domains or buying inbound links.
It will always be a war and there is never going to be a victory over webspam. Even with something like web3 where posting content costs money I can imagine ways spam.
I guess in the perfect magical world where the algorithm detected "suitability for humans for the given query" then if gpt-3 spat out a bunch of stuff and it ranked highly, it would mean this wasn't really spam, because it was useful to humans, even if it was generated in a spammy way.
Well, it could still be spam, just that is very closely resembles non-spam, but the end result is you didn't get the information you were looking for, just something that was closed to it, but misinformation.
Imagine you're searching for vaccine information. GPT-3 generated spam provides pages that highly look like authentic NIH medical reports and expert opinion, only every piece of factual data was replaced with the spammer's product (let's say a competing Pharma vaccine that is less effective)
Search engines are not arbiters of truth, they can only approximate it via consensus knowledge, and if consensus is distorted by vast amount of spam, well, there's a problem.
Think of it like a blockchain ledger. If someone steals more than 50% of the hashing power, than all bets are off. Well if someone steals a non-trivial amount of a subject area with spam, than consensus algorithms break down, and the 'ledger' for that subject looks to be weighed towards the spammer.
Just heavily penalize presence of affiliate links and ads on the page and the 90% of the problem will go away. But of course it's something Google will never do. It all comes down to their perverse incentives.
I think the biggest sentiment shared with everyone against Google search at the moment is the eagerness to see better competition.
While it may be true (and certainly easy) to paint everything with this large brush of 'nothing has changed, we've seen this before' - it misses the point just a tad, which is (imo) aimed at the disdain for constantly being tracked, and wanted to have a much more hacker friendly web experience (like the old days).
Things such as https:/serx.cf/ (https://searx.space) or https://github.com/benbusy/whoogle-search (https://whoogle.dcs0.hu) should prevail on a site such as HN - and furthermore, we should be setting the path with these tools for the future use of internet by non-power users.
The other day I was searching for a specific kind of jewelry and realized I don't know of a search engine that can do what I needed, which is to just find good results for my search. Searches for jewelry-related keywords triggered Google to go 90+% ads, and their results (and other search engines' results) were so junked up with spam and the same couple sites over and over that they were useless.
We're back to the Web needing a search engine.
[EDIT] I should add that the ads Google was showing me didn't even do a very good job of showing me the very specific kind of thing I was looking for, even though there must be thousands of stores around the world selling pieces that fit the keywords. The ads were for jewelry, but most of them weren't anything like what I was trying to find. In this case an entire page of ads but all from different sites and mostly the thing I was looking for would have been better than nothing, but it couldn't even do that.
I've found Brave Search[1] and Kagi Search[2] to be great alternatives to Google. I know exactly the sort of thing you're describing and both of them are a breath of fresh air in the space.
Am I the only one who just skips the search engines and go straight to the source? If I want factual information, I just go to Wikipedia and use their search. If I want to shop I'll go to respected online stores and again use their inbuilt search feature.
Obviously I've just built up a list of good sites in my head which I trust... Google search is good for discoverability if you're new to the web I guess? Although in the old days that's what web directories where good for:
Wikipedia is the perfect contrast to google search results.
- It contains 100% signal- no noise- and provides helpful related links if you need more information.
- Pages are organized and brutalist.
- Every page has a steward who (thanklessly) keeps the information accurate, up-to-date, and ad-free.
Contrast this to Google:
- 50-100% noise (depending on the query; more information requires more queries and therefore less signal)
- SERP pages are disorganized and absolutely riddled with UX dark patterns (modals, banners, autoplaying video, etc). Many pages with good info are over-styled/over-javascripted/over-languaged, and finding the one or two sentences you're looking for is a chore.
- One-off SEO spam plagues everything; ads and affiliate links are pervasive. Stewardship is a waste of time.
As people turn to Wikipedia for information the same momentum that ruined Google is sure to follow. It's already happened to Wikipedia pages that even tangentially touch anything political. If you go on the talk pages you see a slow motion battle between good faith editors that want an encyclopedia of information with a neutral tone, vs bad faith editors that want to use Wikipedia to represent their idealogical narrative. It's even spilled over into historical pages that don't carry the "correct" judgement on the past.
It's probably worth noting that Wikipedia breaks character occasionally and does the banner panhandling, intentionally nosediving UX to put an ad front-and-center.
This is the behavior that should be penalized by search engines- however difficult it is to quantify. Wikipedia is some of the highest quality info on the internet, so they should be able to afford the penalty themselves.
> It's probably worth noting that Wikipedia breaks character occasionally and does the banner panhandling, intentionally nosediving UX to put an ad front-and-center.
They are asking for donations to fund their best-in-kind free project. I know ads aren't popular on HN but if you don't want ads, and you don't want a donation banner, how do you expect sites to be funded?
Wikipedia's expenses go up year after year seemingly needlessly. It is making revenues far and above what it needs to maintain the project, but its expenses seem to go up each year anyway. https://en.wikipedia.org/wiki/WP:CANCER
>If I want factual information, I just go to Wikipedia and use their search.
Wikiepedia is a very good resource for a lot of things, and a good jumping off point, but you shouldn't assume that you are getting "factual information", especially when it comes to hot button social or geopolitical issues.
It's much easier to type "searchterm" or "searchterm wiki" in your web browser address bar and then click through to the Wikipedia result than it is to first navigate to every individual site and use their non-standard search bar.
> Google search is good for discoverability if you're new to the web I guess?
This is almost certainly not true. I don't think kids (who make up the vast majority of 'new to the web') care about using Google for discoverability. They'll use YouTube, Twitch and Instagram to find things they care about. Google is for answering questions, not finding new things.
And honestly, it's not generally that good at answering questions.
I feel like that this has not changed the last 20 years. Yes - google was at some point like a miracle that seemed to solve lots of problems around searching the www for information.
While google "refined" its search and monetarized it the web still evolved and is evolving to something.. different. Many of websites most people already know, competing around google top rankings and ad revenue; there are even people dedicated to "make $website more visible to the web (what they really mean is google)" for lots of money while the real internet goes on in the background.
We need more ways to search the web. We need lots of different search engines that are competing and working together also. The web is still young and no one really knows what it will be in the future. (I fear it has to do with ads. Lots. Of. Ads)
>The web is still young and no one really knows what it will be in the future.
My fear is that walled gardens might win in the future because who guarantees you that websites won't move to Facebook Pages, Facebook Groups, Slack and Discord channels etc. Open web is weaker than ever just look at LinkedIn; walled garden, throws you Register form in the face when you try to access it and won't let anybody crawl or scrape their content except Google who drives more traffic to their walled garden.
I know normal people would never use it but I sorely wish there was a way for me to just grep the web instead of using "search" as offered by Google et al.
They have the buyer ready to buy and still missed the sale because they wanted to make a profit on ads. How ironic. And this kind of experience probably turns a lot of buyers off from using Google search.
We don't need your unrelated ads, we already know what to buy, don't patronize us. We need help getting to the product page from keywords. We need real reviews. We need a shopping experience we can trust.
That was the craziest part to me. I was interested in both products and information, and despite deciding it was a good idea to show me almost nothing but ads, they didn't manage to show me anything worth clicking for either purpose. I was practically their ideal target for getting someone to click an ad on purpose, and they still dropped the ball.
Could you clarify what you mean by 90% ads? I would assume there are always organic results right after the top 2-3 ads, has this changed somehow?
Also you say the results were "all from different sites". Is that a good or a bad thing? I imagine having too many results from the same site would be less informative, no?
I'm very curious to try the same search query myself, but of course I understand that it may not be something you'd want to share.
Google's ideal would be that every single search result would be both relevant and an ad. And it's going there, somewhat at least. Because the people who have the most time to write articles are employees writing/researching 8h a day. Someone doing it on their free time has no chance of competing. The problem is that obviously the people paid to write are biased. In some cases maybe it's a problem, in others maybe not.
It's already dead. Google mined all the links that were curated by the initial internet communities for all it was worth and turned them into profits for Google's earliest employees and shareholders. Now that no one is curating useful links anymore their search quality, unsurprisingly, is deteriorating. Without human curation there is no signal for Google to use anymore and whatever signal is there is just SEO spam that is optimized for serving ads. It's like an ouroboros eating its own tail.
It’s not just the links. After the links, google mined facts, like “how much does a german shepherd weigh,” so no on gets those clicks, and the incentive is gone there too. They’re even mining the snippets of the content, lowering the incentives for creating that too.
It's essentially a machine for printing money and people don't really understand what they're giving up in exchange for "free" search results. Google is beholden to market forces, it's no longer in the business of indexing useful information because the market doesn't value useful information, it values ad revenue.
This is a structural problem and anything that gets large enough will succumb to the same forces. If the incentives are for optimizing ad revenue then that's what all corporate machines will do at scale, regardless of their initial motives and incentive structure. It doesn't help that Google is also an ad network, hence the ouroboros aspect.
You say it's a machine for printing money in a topic about people complaining that it doesn't work any more. It may have been but it won't be forever if things keep going the way they're going. Quality content is already being locked away.
We're in agreement. I don't think quarterly earnings are the right way to design and build products. Maximizing profits is not correlated with value and is often inversely proportional to it. Google was so successful that they changed the incentive structure of all content on the web. Now they're like the yeast drowning in the byproducts of their own metabolic processes. They exploited whatever nutrients were available (hand curated links) to make them initially successful and now there is no more worthwhile content being generated that is not designed to rank highly on Google (which is not the same thing as quality content and is instead content optimized for generating ad revenue from the Google ad network).
It's the same with social networks, upvotes and likes skew the the type of content that is generated to be liked by the people liking and upvoting instead of being insightful. Popularity is not the same thing as insight so the social web is full of mostly useless but popular content.
This sounds almost like Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure."
Google made links on the web the measure of how good a page was. That became the target of everyone trying to do SEO. As a result, it stopped being a good measure of how good a page was.
But in the long run, nothing will work in that environment, because every measure will be gamed as soon as people figure out that Google is using it. Google's only choice is to try to stay ahead of the SEO crowd, and I'm not sure they can do that (well) for too much longer. In fact, if the article is to be believed, they're already starting to fail.
Yes, it's very similar with the added caveat that Google has an interest in serving results that have ads from their own network. This is why Google's metrics can be hacked. Anything that is barely above being classified as spam but serves ads from Google's ad network will be prioritized over other results simply because they have to hit their quarterly revenue targets. SEO hacking is not possible if a search engine is just a search engine but Google is also an ad network so they will always be susceptible to being gamed.
This is also the case for social media platforms. They're incentivized to surface content that generates engagement and ad revenue. Basically ads are at the root of all problems when it comes to the internet and the content on it.
Here and further down this thread you repeatedly allege Google upranks pages if they participate in their ads program. Google explicitly says they do not do this [1]. Do you have evidence to contradict this?
> SEO hacking is not possible if a search engine is just a search engine
Uh, yes it is. If the owner of the site being searched is generating profit from that site being searched then they will game the search engine's algorithm to get them the most clicks.
I'm not interested in a pedantic argument. If you didn't understand what I meant then you should have asked for clarification. A search engine designed to surface useful information is not gameable if it is not in the business of generating quarterly profits from its own ad network. A site designed to drive traffic to itself can still try to hack the system by generating spam but without Google's incentives for surfacing such content because it serves ads from its own network there will be fewer such sites and useless content to go along with it.
At the moment Google is incentivized to uprank spam because the spam comes with ads from its own ad network.
A search engine finds 50 pages that are exact matches for the search. Which one does it present as the top of the list? How does it decide? Unless it decides literally by a random number generator, however it decides, someone will try to discover the algorithm, and exploit it. This is true whether or not the search engine allows or displays ads.
> I'm not interested in a pedantic argument. If you didn't understand what I meant then you should have asked for clarification.
I don't think it was a pedantic argument or a misunderstanding. I think Lascaille understood your position and disagreed with you, not just pedantically but over the substance.
This is actually what google does. They find the 50 sites for your query and then do a multi-armed bandit test to see which one gets the most clicks but with a bias towards sites that serve ads from the Google ad network. A search engine without that bias is not gameable because it will converge on the results that is most popular for a given query and not because it also serves ads that affect the search engine's bottom line.
Popularity is gameable but not the same way as Google is currently gameable because as soon as a site becomes popular and starts exploiting its ranking it will be easy enough to add a decay factor to prevent such sites from dominating the top results during the multi-armed bandit stage of ranking.
In any case, the logic of why Google is going to shit is obvious. Arguing about fixes is not going to change their underlying business model and why spam is dominating their results. As long as they are a search engine, an ad network, and a corporation that must maximize profits their results will continue to deteriorate until the top results are all just spam.
> I don't think it was a pedantic argument or a misunderstanding. I think Lascaille understood your position and disagreed with you, not just pedantically but over the substance.
Then that wasn't clear and seemed like a pedantic point since it's obvious that any algorithm is gameable and I should have made it clear that I wasn't talking about a perfect search engine but one that was not susceptible to profit driven spam (which is currently the reason that Google results are going to shit).
In my mind, if the web page tries to exploit knowledge of the search engine's algorithm, that's "gaming". This is done by the web page, without the deliberate co-operation of the search engine.
If the search engine is the one doing the funny business, to increase their own profit, to me that's beyond "gaming". That's... "corruption" might be the right word.
That's reasonable. Substitute "corruption" wherever I used "gaming" when referring to maximizing profits at the expense of serving useful search results.
>I don't think it was a pedantic argument or a misunderstanding. I think Lascaille understood your position and disagreed with you, not just pedantically but over the substance.
Yes, I was a bit disappointed with the reddity response. The sites admin has an incentive to have their site appear first. The quality of their content may not suffer but they will additionally try to identify how the algorithm works and 'game' it.
I honestly don't think this is the problem, like at all. There are human websites made by humans, still. There's more crap, sure, but the good stuff is largely still out there.
The problem begins and ends with the conflict of interest that Google both sells ads and selects search results. If they didn't have a vested interest in people visiting sites with their ads on them, they could decimate the number of spam results.
Which websites are shown in the search results is not influenced by whether Google has ads on them.
The only thing that influences Google search results is Google's desire to keep as many people using Search as often as possible, since nearly all of Google's money comes from showing those text ads at the top of the Search results. This is all public information, you can read it in the 10K etc.
So if Search sucks, it's not because Google has the wrong incentives but because they can't solve the problems Search faces.
Hard to proof this but anecdotally lots of energy that used to go into people's hobby or passion websites now is going into digital platforms like reddit, pinterest or Twitter, or if lucky substack or Medium. Some of those are walled gardens and Reddit as discussed here is where people now search. Much less out on the actual net
Well, since I actually run a search engine looking for this sort of stuff, I think I'm able to demonstrate that a lot of what was out on the net is still there, just exceptionally hard to find on Google.
Hah, I didn't catch that I was responding to you. Definitely would defer to your experience. Love your search engine btw. It brings the fun back to the web!
If anything I think I'm only scraping the surface, since a lot of my methods involve ruthlessly discarding data that doesn't live up to a fairly blunt set of criteria. I think with something like a headless browser (and a lot more processing power), I could probably use laxer standards and find even more good stuff.
Right, but that's pretty trivially identified. I've had great success doing that with my search engine. Here's a thousand domains that are low in farmed contents in no particular order:
So why do you suppose Google doesn't surface the domains and results you presented? For example, I searched for "game reviews" and gameboomers was nowhere to be found.
I have no insight in their search engine, but I do know it would hurt their ad revenue to surface results that have no ads. Lends itself to speculation. But it could just be some confluence of other factors, of course. They seem to aggressively favor recent content (I do the opposite).
Which is why I said it's an ouroboros eating its own tail. The scale it operates at and given that it's also an ad network guarantees that whatever results it finds will favor its own ad network. It doesn't even have to be intentional since all they're doing is optimizing some metrics and running ML algorithms. There is no single person that could be blamed for the deterioration of the results. There is no way around this conflict of interest and they will continue pushing the envelope to increase their own revenue at the expense of useful results for as long as possible.
If a site is hosting ads from the Google ad network and is barely above being spam then Google will prioritize it over other results in order to maintain its quarterly revenue predictions.
That's a reasonable analysis. I do think Google has grown about as it can legitimately grow, which means the only way it can maintain an appearance of growth is by cannibalizing itself. The increasingly aggressive use of search ads are probably an example of this.
When are you going to have a VC fork over $10,000,000 and start your empire??
I use your crazy search engine all the time, and it's fucking great. Last time I checked, though, you're getting like $30 a month from Patreon and that's it, haha.
You're too much of a well-adjusted normal human being, that's your problem. You need that touch of Zuckerwellian sociopathy to get to the next level
Well I'm actually up to a $100 now, which is about twice my burn rate, so you might as well say I've broken even already. To the moon, baby.
To be serious, it's notoriously difficult to actually make a profit off search. I guess we'll see if Kagi will be able to eke out a market for itself, that may change things. Right now, I think investors will look at Bing and go "yikes".
Doesn't go to say I couldn't build a fantastic search engine with more resources than my current shoestring budget, the hard question remains: Why isn't this just a waste of money? Would take one hell of a sales pitch to gloss over that concern.
Wonder how we could set up an alt-web without the incentives that cause this problem. Delist any for-profit site? How would the sites keep the lights on without ads?
To me it's more a sociological problem than technological. Also networks have changed.. somehow the decentralization idea is spreading fast. For ideological, technical, cost .. or other reasons. Some people start neighborhood wireless networks etc.
It also seems to me that internet has somehow became a middle man and is not providing human deep enough interactions, especially outside chat-like website (basically any exchange, business)..
I could envision a whatsapp like system with quality control for producers and transparent transaction/tracking/accounting management offered by the network so people spend less time on side-loads and just focus into helping each others and doing what they need to.
> Making them turn off the lights is the goal. Good riddance I say, once we get there.
I wish there was some middle ground. Think of all the useful Youtube videos showing how to play an instrument, do woodworking projects, fix cars... there is a vast amount of knowledge there. Maybe YT should be nationalized :-)
Youtubes early success was due to being a free video hosting platform, the monetization just led to the rise of 10:04 long videos. In any case most larger creators will put sponsorships in band like the good ol days. I’m hoping decentralized alternatives can take over like Peertube or Odysee, but I do also appreciate the more traditional business model of Vimeo.
I've had no problem finding sites to index with my search engine. Like it's tiny compared to Google today, but it's about the same size they were when they first started out. Leads me to think there's probably as much, or more now as there ever was. There's just more noise to go with it.
Yep! There's a massive amount of useful content on the web. A lot of it is just a pain to find right now, because the quality of search is bad and the amount of garbage is a thousandfold greater.
It's true that walled gardens have been eating up useful information and that is a real shame, but make no mistake: there's still a ridiculous amount of good stuff on the open web. They're not playing the SEO optimization game so they get buried.
That actually has made me feel a lot more positive than I would have imagined.
I have sort of got stuck in this bubble of reddit and not much else and... the way people behave has really sapped my desire to contribute anything to the world. I'm looking around me at the opinions a lot of people are expressing and I think... no, these people don't deserve nice things. It seems sometimes like the whole world is full of venal, bitter and divisive people that do nothing but snap bitterly at the heels of anyone that makes something of themselves or strays too far from safe consensus.
Federation is the only reasonable solution at this time but the technical overhead of federated search is high enough that most people won't use it so it won't benefit from network effects like Google did in the beginning. There might be a combination of blockchain juju that could make federation viable but all the thought leaders in that ecosystem are too high on their own supply to realize they could use blockchains for anything other than gambling.
There’s no better solution - you either have ads or a paywall, which will result in few users.
If there were a better solution we’d all already be using it. Certainly you can rely on savvy people to produce free stuff, but the total amount of content will be drastically lower and therefore fewer consumers.
The best thing is to just use bookmarks and your favorite sites’ own search.
It's interesting, and quite concerning, that Reddit has cemented its position as a key repository of useful information on just about everything just as its drive towards monetisation really kicks into gear. It's concerning because, as part of that monetisation strategy, Reddit is becoming increasingly walled off and anti-user. I am sure I am only one of many long-time Redditors who have vowed to stop using the site completely once old.reddit.com goes, and it's only a matter of time.
It's probable that a huge amount of useful information will soon become much more difficult to access, and/or diluted by stealth advertising, as Reddit looks to aggressively monetise its position. I'm interested to see if a credible alternative emerges and if there is any effort to move some of the existing useful data off the platform.
It's very interesting to me as somebody's who's been making and burning Reddit accounts since before the Digg implosion. 15 years ago, I trusted what I saw on Reddit when it came to things like products: If I came across a post on the best can openers, I had some certainty that people were just sharing their opinions on can openers.
Now I don't trust a single damn thing I see on that site.
Depends on what I'm looking for. If it's something small (under like 20 bucks for me, substitute whatever for your comfort level), I don't even bother. I just go to Target and pick up the 2nd or 3rd cheapest whatever.
Now if it's a bigger ticket item, it gets dicey, because then the best strategy, in my experience, has been to determine what about said item really matters to you and then find a group of people who share that and ask their opinion. The problem with this is that those people have the best information, but also very exacting standards and aren't very sympathetic to arguments like "But I don't HAVE hundreds of dollars for a coffee maker."
Honestly, more and more in the past few years consumer products and services in the world has just been like a race to 'get something past me' and I've just started making my own stuff, buying vintage, or borrowing because I'm tired of being sold crap.
Australians have www.choice.com.au that use a subscription model to monetize rigorous product comparisons for big ticket items. I'm not sure what the options are in other countries. Obviously there are special purpose publications like DPReview and Anandtech
I find things like CR or choice really good if your needs align with the standard consumer's.
One example is I'd probably recommend you look at office chair recs from them if you were a male between the 20th and 80th percentile in height (or a female in the male ranges) and weight who only uses the computer for work. If you were like me, a woman who's 5'3" with the legs of somebody 5'1" who sits at the computer for hours, those recommendations wouldn't work. (They also wouldn't work for Yao Ming).
Too bad wirecutter is very hit or miss. I have definitely bought their recommendations and regretted it.
Searching for query + reddit is an OK means of looking for recommendations
What would be nice is a store (like amazon or target) that only sold reputable items. I'm not looking for 3rd party marketplace (like walmart or amazon offers). Maybe costco (brick & mortar) offers this curated selection, but even costco's products are hit or miss.
Not really what you're asking for, but I saw this[0] a while back and it's made me actually think about can openers a bit now. You might find it interesting, too.
I just default trust Wirecutter for everything these days. I know they might not be optimal - there's probably a better alternative for my specific usecase if I spent the time on it but I'd rather just let them make the decision on trivial things.
The only real option is to spend more time, cross reference against many sites and forums, and use your intuition as to which comments and reviews are authentic. Also pay attention to negative reviews.
Yeah I have a 6 month timer on Reddit accounts because I tend to post on topics in my home town and my close hobbies and I’m afraid of being doxxed. At least in HN I just talk about programming languages and stuff so I keep my account.
I also use reddits account name generator. I think that also helps that my name there wasn’t thought up by me. Least a little level of abstraction.
Has anyone here actually tried the "<search query> reddit" search lately? Click on a reddit link and it takes you to a page that forces you to open it in an app. This has made me stop using reddit completely.
I'm the Google Rep, and there were two more parts to my reply which explain this:
Here's why people often think quoting with Google doesn't work when it really does (I've looked a huge number of these reports).
1) We match ALT text
2) We match text not readily visible, such as in a menu or small text
3) Page has changed since we indexed it
4) Punctuation...
Punctuation comes into play if you did a quoted search like "dog cat" and there's text that says "dog, cat" then we'll see that without the punctuation. That doesn't seem a major issue but we're looking at if we could improve there.
And...as noted in another reply below, in the first result, if you look at the cached page that shows the ENTIRE page that we indexed (rather than the paginated version you land on), there's this text:
quotes, don-t-give-up-the-fight
with when you remove the punctation, is the match:
quotes dont give
And I get that this can be frustrating, that we don't consider punctation in a quoted search. That's not a new change, however. It's been that way for ages. But as I also said, it's something we might revisit.
Yes, as someone pointed out, there are tokenized versions down at the bottom, but that exact string does not exist.
So first of all it's just not true.
Second of all, if your consumers are all complaining the results do not match what they are searching, it might make more sense to listen. Adding quotations marks like that to find an exact string should be a huge red flag that the user is only interested in that string – why then would "it's in an offscreen meta tag of a cached version of the actual site" be anything other than a bad result?
Regarding point #2 - why would you ever match text that cannot be CTFL+F'd?
I can understand why you would match it generally, but why would you ever serve those types of results to users in a browser? Why would users ever want this as a feature?
What's interesting to me is that if I search for "You said in the post that quotes" the first result is the Twitter thread, with that quoted phrase bolded. the second result is some other site which is mirroring this thread itself. There are no more results, I only get those two.
However, if I search like you did for "quotes don't give" then I get the behavior you describe. So the quoting works sometimes, not others. Maybe based on length or something else, I'm not sure.
The difference might be due to Google A/B testing their algorithm. That is, giving users different results for the same query trying to infer which results were preferred based on whatever happens next (user keeps searching, or user goes and stays on some site).
Ignoring full page replacements ('Showing results for "eggzactly that", to see results for "eggzackly that"...) I think this is all just punctuation related.
For instances, on the ["quotes don't give"] example, the first result I get is
Thanks to the wonders of tokenization, that "don-t-give-up-quotes, don-t-give-up-the-fight" gives you the string of tokens, "don t give up quotes don t give up the fight", which contains the exact phrase "quotes don t give", which is the tokenization of the phrase "quotes don't give".
That's actually from the visible text of the page -- well, desktop version as I saw it; when you click through on mobile they only show half as many quotes and you need to load more to find it.
I may start using another search engine just because of this. I can't tell you how many times I've searched for something in quotes, but the results almost always lack a word or two. They just want to show you some results; they don't care if that's exactly what you wanted.
So out of 22 million results, I can really only see 198?? That can't be right. Wait, it says, "In order to show you the most relevant results, we have omitted some entries very similar to the 198 already displayed.
If you like, you can repeat the search with the omitted results included."
Yeah, that's what I'd like, I want to see all the results. CLICK.
OK, it takes me back to "About 22,100,000 results," so far, so good.
I always figured this was a performance issue related to sharding in distributed systems. Deep pagination is an expensive operation so most search clusters limit the number of visible results by default. That, in addition to an assumption that results beyond a certain number are unlikely to be useful - how many times have you found something on page 10 vs just reformulated your search query? - means that most applications just leave the default limit in place.
Returning a count of results, however (especially if it doesn't need to be precise), is a lot less expensive. Hence why Google is happy to give you the 22,000,000 number.
Yup, deep paging is a huge problem for distributed search systems. It's not just a Google thing, its every search engine. Here is a section from ElasticSearch's documentation[0]:
"Avoid using from and size to page too deeply or request too many results at once. Search requests usually span multiple shards. Each shard must load its requested hits and the hits for any previous pages into memory. For deep pages or large sets of results, these operations can significantly increase memory and CPU usage, resulting in degraded performance or node failures."
It's not just a Google thing, its every search engine.
OK, I see now. I tried it on Bing and got similar results with two small caveats. First, Bing gave me 861 accessible results, which is a base 2 order of magnitude greater than Google's. Second, Bing's total number isn't nearly as astronomical, it claims only 191K total results, not Google's 22M.
Could it be that Google has just indexed 100x more terms compared with Bing? Maybe, but my anecdotal use of both of them doesn't really seem to indicate that Bing is so deficient. For example, I tried using a phrase that would come up with just a few results. "bioavailable turmeric extract formulation" (in quotes) yielded 24 results on Google, (plus 4 ad results on top). On Bing I got 33 results, plus 2 ads on top. In fact, Bing looks more like "old Google" than new Google looks like old Google.
Number of results (“match set”) can differ even with the same document corpus. e.g. Tokenization, n-grams, language analysis, stop words, synonyms, etc.
The ElasticSearch documentation is actually pretty good documentation of all search engines in general.
I've noticed that I have started doing that recently - appending reddit to my queries.
There just seems to be a load of imitation sites now, like 6 different wrapper sites for GitHub, 8 for StackOverflow, a couple for GitLab, something aggregating a load of forums - so the first couple of pages are the exact same content - just from 15 different sites that copy the originals.
At least going with a community site there tends to be actual discussion and or useful links to the relevant content
Those are infuriating. I hate to see ACTUAL content creators having their livelihoods stolen this way. Why wouldn't Google filter out the worst offenders? It takes literally one minute to get a nice list of a dozen imitation sites that nobody would miss. Maybe Google feels a little inhibited from 'choosing the winners' for all but the largest cases?
One FTE at Google could probably filter out like 99% of the SEO spam sites in technical english querries.
It would be a winning battle, since it is less work to blacklist than to make a high scoring site.
I guess Google Search internally is a mess. Maybe they have no clue what they are doing or have some really bad directors and lower managers messing stuff up.
Maybe there are so much blackbox ML called from 1000s of Perl files that the engineers don't understand what is happening.
I often wonder how much modern IT infrastructure is simply this mess of 'we have no idea how it really works' blackboxes strung together with API calls.
I suspect you're right about how much of a true understanding they (at google) still have of the behaviour of their search engine.
I'm not sure about these days, but historically the engineers on Google search wanted to fix these problems algorithmically, rather than delisting specific sites by hand
> If you can do this, so can Google. This just shows they refuse to.
If they immediately blocked these sites then Google would get a lot of flack for censoring the web.
I don't like these sites as much as anyone. A while back I even tweeted about[0] having a dream where I wrote a browser extension to intercept and redirect these copycat sites to the real site.
In my mind this falls into the same category as phone spam. The phone networks could block these but how would you feel if you knew your phone company was auto-filtering incoming calls without you having any control over that? It's a very thin line.
Hopefully one days algorithms will be smart enough to auto de-rank copycat sites or blatant plagiarism so they don't show up on the first page.
They already de-rank plenty of sites for countless abuses, especially for gaming search. They have been doing this for a long time, and no one has ever called it censorship. This is the first time I've heard of anyone even suggesting this.
Also, their ranking algorithm is extremely complex. To suggest one complex algorithm is censorship and another is unbiased search results is to have a very naive understanding of how search works.
> So... if google creates an algorithm to detect copycatting/plagiarism it's okay for them to deploy it, but it's not okay if they do it by hand?
No, I thought more about my comment a day later. I don't know what a fair answer is. Being ranked on page 216 by an algorithm or de-listed manually is basically the same outcome.
I have found that installing uBlacklist (a browser extension) and blocking these sites from search results as I encounter them helps noticeably. There are only so many of these "clone" sites that rank highly on Google, so I found it pretty easy to keep up with them for the things I usually search for. There are even shared uBlacklist lists for things like SO clones, but I haven't even bothered to use them.
There HAS to be a way for google to detect a site is a copy and de-rank it. I refuse to believe their army of PhDs can't figure this out. Google's incentives are wrong. They make more money from SEO spam with ads than from the original sites.
It’s sad but I also noticed I have to add « wiki » more and more because Wikipedia is increasingly not the first result for searches where it should be the first result. Instead there’s often the stupid Google widget obviously copying Wikipedia’s content without a direct link to the actual page.
Still in shock that they killed reverse image search and replaced it with some useless AI tech demo.
It used to use the actual image and be able to provide context from where that image was found elsewhere. Now it seems to throw the image at AI and the AI will go "Oh that's a street" then they will just show you streets with similar colors as the image you put in.
Completely useless for trying to locate what movie a screenshot is from, or even similar images because the category searching is too general. Yandex image search completely blows it out of the water by being nothing more than a modern version of 2010 era Google Image Search.
I totally agree with you in that it is now completely useless in comparison to what it was.
But to be fair the accurate version could be used maliciously (for stalking someone, etc). I just don't use that tool at all anymore.
I'll echo one of the points in the article: "Google is trying to be smart".
This is the source of many people's frustration, and the source of forced synonyms. A dumb tool that adapts to humans as they use it and tries to be "smart" prohibits us from getting more skilled in the usage of the tool. It becomes unpredictable, and it introduces significant friction each time it does something dumb.
Even if the tool is correct 90% of the time, it is wrong 100% of the time on an emotional/ux level. The successes are invisible in aggregate, but each mistake sticks out like a sore thumb. I guess why this is: modern understanding of our brains (as I, a lay man, understand it) is that they attempt to continuously predict what's going to happen next in their environment. When all predictions are correct it feels good, and there's no tension. A tool that adapts and changes makes our brains predictions turn out wrong, and our brains punish us with tension and attention each time the tool does not do what we want, since it failed to predict the desired behavior.
Previous versions of google felt so nice precisely because our brains, or at least those of hackers, could adapt to its various tricks and shortcuts.
I wish I could upvote this comment a second time. This resonates strongly with me, and in general is why I think the entire “ml driven” approach to contemporary technologies is misguided.
Historically, when we use tools, one of their crucial aspects is that they are deterministic. A hammer only does one (or arguably two) things, and those functions are completely static and will never change. This not only allows users to become experts and more proficient with the hammer much faster, but it also allows users to use the hammer in flexible and creative capacities—when you know with absolute certainty what something does it’s significantly easier to “deduce” what might happen when you apply it to different circumstances.
Contrarily all “smart” technologies move away from determinism. Rather, they all admit of some amount of indeterminacy, even if their basic purpose is fixed. There’s a happy medium where the indeterminacy is restrained enough that a tool’s cleverness really does save us time, but too often the pendulum swings too far and you wind up actively having to fight against your tools to do what it is you set out to do, making all the microscopic time shavings “smart” features net you worthless.
I’d much prefer my tools and products to be, dumb, fast, consistent and ai free.
The irony for me is that Google had a Reddit before Reddit was a thing, it was called Orkut and it was glorious. You could create communities based on whatever the heck you wanted and connect with friends and strangers alike through any twist on any topic you could think of.
But Google being Google, it didn't see the potential because Orkut didn't achieve global dominance (despite being the social media platform in places like Brazil and India at the time). Google's obsession with AI-fying the crap out of its business will likely end up being its downfall now that it's proving to be increasingly ineffective against the SEO-motivated players, who now have an enormously diverse toolkit to game the coveted first SERPs, from black hat to downright paying right into Google's pockets to get their way.
There have been many interesting threads recently about the decline of Google's search quality here on HN. There's zero doubt search results are getting worse, and that ads and spam are the cause. But Google's financial performance has been going from record to record. So there is a huge disconnect building in the market.
Each thread has had some common themes, but what's surprising is how different the problems discussed are. Here are a few of the best recent discussions:
Google no longer producing high quality search results in significant categories (twitter.com/mwseibel):
Disclaimer: I'm working on a search startup, so I have a clear bias, but one of the main reasons I am working on a search startup is because Google's results are clearly getting worse.
The weird thing about these is that they blame Google's search results on spam. I work in SEO and I can tell you that they are much better at ignoring spam than they were in 2010, where a lot of these people quoted still have their heads at regarding SEO.
What's been going on at Google is reliance on neural nets to take care of various ranking algorithm tasks. We want better keyword matching to generate results, but Google is developing ways to match query vectors to document vectors using stuff like BERT. Google is looking at the knowledge graph of entities that emerges out of the content we write and is trying to figure out which relationships between entities are important to a query and which result set has the best coverage and diversity. This incentivizes publishers to write a lot of text that covers multiple related topics and bury the point inside of it.
The other major shift in Google is how they consider links. PageRank is still around in some form, but there could be other link-based algorithms that serve similar purposes. The last few years of core algorithm updates put a lot of importance on receiving links from news websites for any keyword with commercial intent. If you want to rank, go hard on public relations.
The result is a real loss of accuracy and a lot more false positives that are semi-related to the query.
>This incentivizes publishers to write a lot of text that covers multiple related topics
Is it accurate to call organisations that write text according to google's incentives 'publishers' or are they merely spammers trying to maximise their pageviews and conversions?
Yup. IMHO spam has become so good at mimicking genuine content, it's hard to recognize even for a human curator. There's so many websites in the top google results that I'm sure are entirely AI generated, which exist for the sole purpose to propagate affiliate links and ads.
Yes. It's like the results when people realized you could have a classifier trained to match a person's face, reversed to generate a new face based on the classifier. There are a few extra steps, but the web is just recipe sites and product reviews that look like what the google ranking algorithms idealized site looks like.
I wish you great luck and success. This just seems part of a long cycle to me (of course, the older I get, the more everything seems like a long cycle).
Google wasn’t the first search engine, and I expect it won’t be the last. Page Rank redefined search, and now that results are 95% advertising-driven, the underlying “search algorithm” means nothing at all. Someone with a “new” algo that isn’t so completely ad-driven (until they too succumb to the only existing model for revenue) could un-seat the giant, at least maybe in search.
This is the curse of the advertising model for all things. Though it can make a lot of money for a good long time.
Thank you! I think there are alternatives to ads that are worth trying.
A freemium model is viable. You can't have ads or ad-tech tracking, or you just end up another Google. But you can have free anonymous use, and paid pro or business plans (API use etc). And referral link attribution can be done anonymously and with no commercial influence on search results.
I also think you have to share any revenue with the people actually making content fairly. That's one of the worst things about Google, and it's one of the reasons the entire media landscape has become an ad-tech nightmare, because Google and Facebook take the lion's share of digital revenue.
It's worth trying other approaches. Ads are a corrupting influence. If you don't say a hard no to them, they eventually take over.
> But Google's financial performance has been going from record to record. So there is a huge disconnect building in the market.
Of course. I used to get one ad on YouTube from time to another. Now for every 3 minutes videos, I have two forced ads at start and one ad at the end. Heck, the other day, I got an ad inside a 2 minutes video. The fall is going to be legendary.
I think that Google tweaking prices until they find the most profitable ratio of ads-in-youtube to subscription-cost-of-ad-free-youtube is going to be at worst a slight dip. Certainly not a "legendary fall"
I’m glad more and more people, from both here and on Reddit (similar discussions appear sporadically) are beginning to notice. I actually have most of these links already favorited haha.
I hope your startup pans out well. Thanks for doing your part to make the internet less terrible!
Hey thank you. I think one of the good things to come out of Google's decline is that there are now a lot more people working on the problem of search again. There is also the possible if faint promise of a different economic model for funding content online than advertising starting to show on the horizon, with a lot of the ideas around web3. Ad-tech truly does make the whole Internet awful.
Isn't strong financial performance what's masking slow decline in quality, then when competitors take marketshare everyone in Google would be pikachu face.
Just the way Chrome insists on auto-completing searches has seriously damaged the efficiency of my Googling. I'll search something like "Type-97 whatsit making funny noises", get no results, go to search just for "type-97 whatsit" and it adds the rest back on by itself and I get the same useless results from the first time. I don't make the mistake often enough to remember not to make it, and every time I wonder what moron decided that was a vital feature that shouldn't be able to be turned off.
In the category of features that think they know better than the user, I hate whoever decided they should start using word embeddings in searches.
For example, you get the same results for "expand" and "extend" with both words highlighted in results when you search for either. This makes google entirely useless for complex technical topics with decades/centuries of established jargon. Searching for mathematics has become a torture.
I don't think that works anymore, I might be mistaken but I recall trying it and still getting non-word-specific results.
I tried it just now with ' "median" height ' and still got "average" and "mean" results highlighted. (Couldn't reproduce with "expand"/"extend" because I didn't recall the query that produced both.)
The first "result" for me--which heavily focuses on mean--is a Google Snippet, which I imagine could have an unrelated semantics engine and, frankly, too often (I am not saying most of the time, though) shows ridiculous garbage anyway (as it is trying to be more intelligent than computers really can currently pull off).
You probably need to enable verbatim results. Click on the Tools button (below the right end of the search bar) and select "Verbatim" instead of "All results".
As has been discussed and demonstrated many times before on HN, this doesn't always work. No one seems to understand why, but Google ignores quotation marks for some people and not others.
Moreover, how is a regular human being supposed to know this? How is this useful to someone not in the tech bubble?
Google's solution to the problem is to make people jump through a hoop. That's not a solution. That's a kludge.
I wish I could mail a postcard to Google and Microsoft to be put on some kind of "not a goddamned moron" list so they'd stop treating me like an ignorant child.
I'll re-post a comment I made about Google Search back in July 2020 because I believe it's still relevant:
"Call me crazy, but I've been using Yandex a lot more recently. Political FUD aside, the results are pretty good, and completely unfiltered.
It reminds me of how wild and unfiltered the internet was back in 2007. However, I wouldn't recommend it to "casual" users. Using Yandex requires a bit more common sense than Google, because malicious domains show up every now & then. For power users (99.99% of HN), this isn't a problem.
With all things considered, it's totally worth it. I never realized how censored Google Search was until I stepped away. As a grown ass man, I don't want anyone telling me what I "cant see" or attempting to define what's "acceptable" - The freedom to choose is intoxicating almost."
You can select a filter to hide any Russian results.
A few months ago I found myself in need to install Telegram on a new PC and a quick google search (I still had not configured another search engine nor an adblocker) gave 4 ads for scammy websites.
I am sure that they try a lot, I am not convinced that their strategy is optimal.
I am very frustrated with Google search results right now. I do use it to search for facts only and it is my main calculator app now. But that's it.
I have been adding "reddit" to search terms since two-three years ago.
And I have been made aware of two new search providers: kagi and you.com.
Kagi was announced in HN, I believe , and I am now a beta user, and I am surprised at its quality.
It does not pull underhanded shit like Google does, and although it is better than Google in filtering out spammy, SEO sites, I would say- not by much. My overall experience is much better with kagi than with Google. I found myself grumbling towards it after I failed with Google in initial days, but now I go straight to kagi itself.
I like you.com but hasn't used it much. It is nice to see varieties categorised in one single page.
Google will certainly lose any user who can be categorised as "power users" even by a jiffy.
That includes High School students researching for papers they are writing and aquarium users searching about fishes.
There's not nearly enough discussion in this thread about Google's Verbatim mode (Tools -> All Results -> Verbatim).
In my experience turning this on substantially increases the quality of Google's search results. It stops ignoring half the words in your query and seemingly parses "quoted" phases as you'd expect. The biggest problem is that there is no ability to turn this on all the time for your account and turning it on is intentionally a hassle.
Google still has a lot of other problems even with Verbatim enabled, but this makes Google like 30% less terrible even if just using it for site:reddit.com-like searches.
If search worked reliably there is a lot of content that would never be reposted. I've followed several subreddits for a long time dealing with hobbies or skills that I know a bit about and it is usually the same questions being asked and answered year after year. If a user could easily find the response I posted in 2006, 2008, 2011, etc then they would not need to make an account to ask their question. They could simply look at the replies posted with all the photos showing how to accomplish what they need to do and then move on with life.
Another reason is that users can delete their posts and their accounts. If that subject has been well covered in the past but the posters later deleted their content then it will not be available for search to find it. I regularly spin up new accounts and have since I joined a long time ago. I delete all the posts and then later delete the account when I feel like I want a new start.
Reddit regularly needs new eyeballs so search has never worked. With an IPO in their future, search will never work.
However, it doesn't just allow you to necro any archived post. I can't determine any rhyme or reason to which ones are open for necro and which ones are sealed.
Searching reddit with Google is good, but also not that great.
- The date, in many cases, is all wrong. Limiting search results to just last month, for example, will usually still return reddit posts from years ago
- Search works great on the post title and to a certain extent the post body, but is really bad on comments. It's really hard to search for a comment especially if the thread is large and long-lived
- Many threads seem to not be indexed at all, e.g the daily question threads in many big game subreddits
If you're searching for a specific post or comment (rather than trying to find posts and comments about something), https://camas.github.io/reddit-search/ has been my go-to power search. Thanks to its delays in syncing, it occasionally will return deleted and edited results that may have already been scrubbed from Google or Reddit native search.
My firm belief is that what is missing are the librarians. Google used to rely on web archives and inter linkages between sites, but bad actors from blogspam to quora have gamed this system in every possible aspect. There are probably just too few reliable sites compared to the global mass proliferation of unreliable sites.
Google will need to start taking tough, manual decisions on which sites to depritoritise in both what is shown and in what is considered in its algorithms in order to fix its search. And this is not a task you can outsource to whoever is the currently poorest native speaker is.
But do you think there are actually enough good sites left that contain quality content? As has been mentioned elsewhere, are people actually still writing product reviews that are organic and not sponsored content?
I mean you can't blame Google too much. In the early days, a large fraction of Internet users made websites and had hand-picked, curated links. This gave Google a fantastic ranking signal with a high signal-to-noise ratio. This is mostly gone now and honestly I'm surprised their search is as good as it is.
I think you have it slightly backwards. While not entirely Google’s fault, search engines motivated much of this “noise” increase. It’s not as though there were people who just made hand-curated, “high signal” websites and then all those people died. It’s even not that all those people switched to Wordpress or other CMS. It’s that the algorithms used by search engines directly incentivized a lot of the awful practices we now associate with the web. I think the first example of this, which is not nearly as apparent anymore, is it was more beneficial for a listicle to break up its elements into a separate page per element, requiring the user to click and reload an entire page to traverse the list, rather than just have the list on one page. This increased the number of back links if anyone wanted to link to multiple items on the list and allowed per-item SEO friendliness. Another more recent example is the infamous recipe blog. While some people genuinely like adding a backstory to their recipes, most add all of that fluff because Google penalizes short content (likely in an effort to reduce spam I would guess). This results in a weird lose-lose situation for everyone involved which has completely inundated the simple and extremely common search for a recipe. The only ones able to not do this are larger sites like Allrecipes which have enough reputation/clout in the eyes of search engines to avoid the spam classification.
But the problem here really is the consumer. In the past, people would have bought a recipe book or subscribed to a cooking magazine and thought nothing abnormal about having to exchange money for data. Now the consumer expects quality data for free. Obviously something has to suffer.
You can blame google because they made changes that forced sites to adopt these practices. The hand-picked curated links websites still exist but are so far back in the index they will never show.
The death of hand-curated general web directories like DMOZ has also deprived search engines of a hugely relevant "signal" for high-quality content. I'm not surprised that the SEO blackhats have basically won since then.
(You can view the schema.org specification for machine-readable content description as an attempt by the big search engines to partially reverse this dynamic and give white-hat SEO the upper hand again. IMHO, independent website owners should enthusiastically adopt these detailed descriptions if they care to "save" the Web from the onslaught of blackhat SEO junk. But a worthwhile successor to DMOZ (probably based on federation) must also be a piece of that puzzle.)
How does in your eyes schema help against spam? Product rating schema was instantly abused when it Google started using it which makes sense since you markup whatever and however the fuck you want.
If you markup stuff maliciously (ala the old meta keywords spam), you can be banned for it after a simple human check. OTOH, it's really hard to tell wrt. most low-quality SEO-format junk "is this malicious stuff that should be nuked from orbit, or just a clueless webmaster who doesn't know any better". It raises the stakes in a way that lets good content stand out if it chooses to.
Google used to actively fight the spam. Some time around '08 or '09 it was like they very suddenly gave up and never seemed to give it a serious attempt again, as if they'd simply surrendered. Unfortunately, they also made searches much more "fuzzy" around that time or (IIRC) a couple years before, foiling manual attempts to avoid spam by using very specific language or unusual phrases.
Google still has archives of the internet from back in ~2005. They can still use the internet back then as a ranking signal for todays content.
Ie. Imagine a now-dead blog which was very knowledgeable about types of violin and would have ranked very highly for "best type of violin string cleaner". Google can look at what content that blog had, and find a page on todays web with similar content saying the same kind of thing.
Many of these articles/complaints don't compare Google Search to alternatives (Bing / DuckDuckGo / ...), so it's not clear whether web search itself is getting "worse" (in the ways mentioned), or whether the issues are with Google Search specifically.
(For example, the article proposes the explanation that "The long answer is that most of the web has become too inauthentic to trust", which is about the web itself, not specific to Google.)
I try to use DuckDuckGo exclusively, but the results are often poor enough that I have to switch to Google.
Just to be clear, I totally agree with all the criticisms of Google here; it's also awful and I'll often end up just doing site: searches, which Google seems to be better at than DDG.
It feels a bit like the search engineers have lost the war with SEO.
I started switching to !g when I had to finish a project in a hurry and I was using a framework I was unfamiliar with. By the end of the project I had just ditched DDG as my default and went back to Google.
Some years ago, I read an autobiography by Jim Clayton, the founder of mobile home manufacturer Clayton Homes, now a Berkshire Hathaway subsidiary.[a]
One anecdote in the book stuck with me: In the early days of the business, Jim kept getting pestered by salespeople from the Yellow Pages, who told Jim he would benefit from advertising in the Yellow Pages to attract new customers.[b] Jim decided to run a test. He ordered and installed a new red phone in the office, ordered a new phone number just for the red phone, and bought a big ad in the Yellow Pages listing only the line that rang the red phone. The ad ran for a year. No one ever called the red phone. Jim never again spent a cent advertising on the Yellow Pages.
By then, the only consumers and businesses who actually searched the Yellow Pages for products and services were those who didn't have a choice, e.g., out-of-towners needing a plumber who couldn't get the name of a trustworthy plumber from a trusted neighbor.
As regards Google, as its search results and rankings become less trustworthy, more and more people will stop using them to find products and services. Other platforms will benefit, like Reddit. And advertisers will follow, as always.
[b] The Yellow Pages were in essence a low-tech printed-paper version of the search business. Businesses paid to advertise in a thick yellow book, and consumers and businesses searched the index of that book to find products and services. See https://en.wikipedia.org/wiki/Yellow_pages
Just a heads up if someone is searching reddit for product reviews. I believe most of them are inauthentic. I worked in marketing for several companies and we always had some budget for whisper marketing aka shilling. There are third party agency specialized in shilling on reddit and making it all look authentic.
As someone who add Reddit to my query when I search products, I’m not really looking for a review. Generally, I look for subreddits talking about what I’m trying to buy. I then see if they have a good wiki/something pinned about purchasing advice. Then, I take a look at what is posted about the product I had in mind (mostly questions not reviews). It’s more interesting to read about the experience of actual users than reading a review. You learn a lot from what they find frustrating.
Most of them remains user controlled with relatively lax moderation involving somewhat large teams. I don’t doubt shilling must happen sometimes and moderators on the biggest sub might be bought but most of the time it still feel like genuine users talking to genuine users.
You can also semi-automate accounts. I used to use reddit actively and was one of the "most active" posters on a couple subreddits, but really I just automatically posted certain articles and then replied to comments([0]). I was clearly a real person who interacted with people which meant I was trusted and because of that the moderators would not pay any attention to me. This was all done for fun in communities I enjoyed participating in, only contributing from sources that I knew would be appreciated. This was so easy that I know that if someone put a bit more effort into making the automation less obvious and also using enough real interaction you could do massive manipulation.
0: interestingly, I posted so many articles from a small amount of sources that people started to assume I worked at those sites. I had to correct people on this repeatedly. Super easy to get trust and once an idea gets out there it's hard to get everyone to forget it.
I tried to do something similar but couldn't get karma very quick (using a paraphrasing API on other people's comments and submitting news from RSS feeds). I think I had a chicken and egg problem with karma because most subreddits require a minimum amount to post. I ran it for about two weeks before I lost interest. None of my accounts were limited in any way and I was just using publicly available Hola proxies. It seems that very little scrutiny is applied to the reddit mobile API. No captcha is required for registration (unlike the website) and no email is required.
This is all theoretical, of course. I've had a think about running a botnet on reddit for a while because I detest what the place has become. I can literally see the obvious botting that takes place across the entire frontpage and it annoys the hell out of me. It's mass social engineering.
Time and time again I see people on reddit really intensely screaming 'go no contact with your parents, abandon your family for the slightest difference of opinion' and such a toxic viewpoint is not only heavily upvoted but applauded by others that jump in with 'I did the same, it's totally normal, everyone goes no contact with their parents these days...'
I really think the place is being weaponised to disrupt society in a fundamental way. I mean, genuinely trying to break down the bonds of society and turn people against each other by creating this alternate reality. You go outside and talk to the average person at work or in a bar about opinions that, from reddit, you'd suspect are mainstream and they look at you like you have two heads. It's so artificial. I just pray that too many people don't fall too deeply into the trap.
I use reddit spordically - [this](https://www.reddit.com/user/FirstToday1/) is my account. I got my job from something I posted on reddit so I still feel a little indebted to the site. Reading the default subreddits will probably cost you about 3-5 IQ points p.a. I don't know where the family hatred comes from either. It's really common on the advice subreddits. Reddit has always had the punk/atheist type stuff but now that more ordinary people use Reddit what they say can have an actual effect on society now. The increasing popularity of memes and videos over text probably isn't helping the site either. Such a shame.
> I tried to do something similar but couldn't get karma very quick. I think I had a chicken and egg problem with karma because most subreddits require a minimum amount to post.
Doing this on an account that you personally use is probably the easiest way, but that doesn't scale well for obvious reasons. Getting past time gates (must wait a month or more to post in certain areas) and karma gates takes time. This kind of makes me want to go back and try this again but there are moral/ethical issues that give me pause.
Yeah I think I would have been more successful had I just bought some aged accounts with 50-100 karma. If you're "morally flexible" and willing to work with some unsavory types there's probably a lot of money to be made but that's not really for me either lol.
> This was so easy that I know that if someone put a bit more effort into making the automation less obvious and also using enough real interaction you could do massive manipulation.
There have been government studies on this as well, like "Containment Control for a Social Network with State-Dependent Connectivity" in cooperation with the Air Force Research Laboratory out of Eglin AFB in Florida: https://arxiv.org/pdf/1402.5644.pdf
> the automation less obvious and also using enough real interaction you could do massive manipulation
I think this has already been done with r/politics. I've created new reddit accounts to argue with people on there a few times and find them being banned within a few weeks for the most trivial of infractions, simply comments like 'I find it hard to believe someone would express that belief in good faith' get you banned, and as you don't get a ban without a report someone has to be reporting comments that no reasonable person would report.
I came to the conclusion - after a while - that it's a secretly walled garden, that basically 100% of the content is automated, probably by bot-runners aligned with the moderators, and that users that stray into it are intensely surveilled until a plausible reason exists to ban them. There is zero metacommentary allowed and making a post 'about the moderation of this subreddit' gets you banned for metacommentary. You now also aren't allowed to question user activity, so if someone's account history is 100% botlike you get a ban for pointing it out.
Wow, based on those prices I could make a killing selling my reddit account!
I haven't worked there in over a decade, but if I worked there now, I would definitely be scraping eBay and all the other sites and closely monitoring all the accounts that I see for sale. I'd probably even buy a couple of them to see if I could find some patterns in the sellers.
> I haven't worked there in over a decade, but if I worked there now, I would definitely be scraping eBay and all the other sites and closely monitoring all the accounts that I see for sale.
I think if you did that you would rapidly be asked to leave, because it would expose how few genuine users the site still has. I don't get the feeling - when I use reddit - that a majority of the comments or posts are genuine.
Even with the shilling, overall on average searching Reddit produces the most useful results for several topics. It’s really good for gauging the severity of a flaw/defect in a product for example, which you’d be hard pressed to find data on elsewhere.
Generally speaking I search Reddit for genuine looking negative reviews not the glowing ones. I'm looking at Reddit to steer me away from obviously bad choices.
Basically, shilling has become so pervasive that positive reviews are automatically untrustworthy regardless of source or apparent trustworthiness. It wouldn't even surprise me to find that companies are generating fake negative review content without ever endorsing their own products, but it's at least a tiny bit harder to fake genuine modes of failure to report on.
This entire thread dumps on Orbi Wifi devices and praises Eero. Maybe Orbi is inferior to Eeero, but the one-sidedness of the discussion is a bit unsettling.
The top comment was created by a poster who almost exclusively posts on the /r/eero subreddit over the span of one year. Many of their comments are specifically in praise of Eero devices.
It's one sided but that's a lot of shills if you think they're shills. Shills with 10-year-old accounts and active posting history that continues to this day. No, I'm pretty sure at least some of them must be authentic.
Of course, this only hilights that user reviews are hit and miss. It's not so uncommon for something to work well at start but then you discover problems later on.
On technology specifically I tend to view these types of threads more through the lens of tribalism than shilling. Like, I don't have reason to believe Lenovo or HP is paying people to whine about Apple online, I'm willing to believe people will do that all on their own.
I remember picking up on this when I was researching vpns. Compare the comments on Reddit and hacker news, and the difference is stark. It becomes fairly apparent that a lot of the Reddit comments were paid.
Is there any place on the web with authentic reviews? The only thing that comes to mind is something like steam, where you have to at least purchase a copy of a game before leaving a review.
Do you really trust CR? In the UK we have which.co.uk, which is similar and you have to subscribe to it. But I'm fairly sure they are shilling for some of their product categories after having subscribed a few times. Like, the mattress category is filled with mattress in a box companies, which are notorious for paying off websites/blogs to promote their products. I really don't trust any of these sites anymore.
Amazon reviews with a picture inside a dirty living room with, like, a toddler only in their diaper in the background or something have been pretty reliable.
>> Is there any place on the web with authentic reviews?
This is where a good social network is useful. I'd trust my friends more than any random web site. Unfortunately I don't have a huge network to call on, so it would be nice to trust my friends friends and so on, but the trust quickly drops. We need a way to improve a simple network with some user-controlled measure of authenticity and trust.
And Google really only has itself to blame for that, TBQH. They have had how many mediocre chat apps that they could have turned into Discord with a baked in user base over the past 15 years?
I mean, it's not hard to tell shilling, and truly authentic, quality shilling falls along a border of marginally still useful content.
Like, I'm hoping to find a specialized community discussing products relevant to that hobby or interest, often posting images or discussions of data gathered.
For example I am starting a container garden on my patio using fabric pots and have searched reddit for a wide variety of gardening products. Perhaps I fell for shills but I looked for people posting images of their gardens and discussing opinions and results, so if companies reselling chinese factory sourced fabric grow containers are hiring people to literally make home gardens and shill online about products, then so be it, thank you for the content? And even then, I don't think I saw anyone specifically pushing any brand at all, and even calling the top brand (SmartPot) overpriced . Everyone seems to use a different brand and they all seem to do fine.
I think that's a strength on reddit. It's harder to shill in a specialist community than it is in an Amazon review or personal blog.
Another reddit community I would look at often for opinions and brands is chef knives and knife sharpening tools. I really don't think there's a lot of shilling there that gets upvoted and promoted.
Depends on subreddits. I love shoes for example, and /r/goodyearwelt is a really decent resource in general. /r/malefashionadvice and /r/rawdenim for fashion are pretty great too.
Thursday Boots astroturfs a large number of the comment sections of posts about their products. Red Wing, Nicks and Truman have periodically had self-identified employees that frequented the site and their specific subs if they exist.
The small group of GYW brands seem to at least be generally aware that a community of enthusiasts exists on Reddit and that's probably enough to start being skeptical of positive reviews.
I am okay with self-identified employees rather than astroturfing. Nicks themselves have mentioned how Reddit attention helped them stay afloat, and how much of a boon it has been for PNW bootmakers. IMO this is a far better outcome than that industry dying, the manufacturing parcelled to China/SE Asia and a storied bootmaker ultimately resurrected as a Frankenstein fast fashion brand. Wesco, for example is doing great because of online attention.
However, even before Reddit, StyleForum was great, and their original darling was Viberg.
I wish it was Google search that was dying. I suspect the problem is a decline in good quality self published text.
There are adversaries gaming the algorithm and pushing low quality results as alsways and they seem to be thoroughly winning on Youtube. While search isn't working as well as it did for me I am not sure it is entirely a search problem. It used to be that a well selected query would almost magically bring up the desired answer as the first result. Even adding search params to exclude low quality sites like quora there is often nothing in pages of results now, if you even get more than a page or two. I remember when results sets used to be massive. But is it the search that is lacking or the content?
IMO Google deserves a large share of the blame. Killing Google Reader inflicted a huge blow on distributed self-published content and helped drive people towards a bunch of walled gardens and systems that promote low quality content.
Where once the blog reigned supreme now content is in the hands of companies like Facebook and Twitter where ephemeral, low effort writing is either behind a wall or drowned in noise. A lot of blog content is now dripping in blatant promotion of people, products and service.
I’ve posited repeatedly that when Reddit IPOs, I’ll be reallocating a significant chunk of my portfolio into their stock.
Their management has historically lacked focus, but if Reddit ever builds a half-competent search index, and positions itself as a search-first, discovery-second destination, they will be in the FANG tier of stocks.
They have the data. They have the dedicated, active user base. They have free moderation. The hard parts are solved. If only they get someone like Satya at the helm. (Also a big reason for me to believe that an acquisition may also be a good play for a AMZN/MSFT)
As the deceased child comment to this post mentioned, reddit's UI is horrible. I strongly prefer old.reddit.com, but I don't expect it to live too long. A good UX designer would make their site much, much nicer to read (and hopefully much more performant). It appears that they are trying to push people to use their app, which I don't particularly wish to do since I'm on a desktop.
As for search, maybe they should just make a deal with another search engine and have it run the query on their site with site:reddit.com or !r in the background and show the results on a branded page. You would think that having access to their own data would make searching easier, but apparently it doesn't.
I am a big fan of reddit, even in it's broken state. As long as you're street smart and know where to stay away from, browsing reddit is a positive experience, at least for the subreddits I hang out in.
I'm really really hoping that one of the big companies doesn't buy reddit. They wouldn't know what to do with it and it would die an ignominious death. In my opinion, of course.
Reddit userbase is fast deteriorating. The power users who were responsible for much of its highest-quality content have been fleeing the sinking ship for quite some time - once a fully credible alternative springs up (and some are in the works already, with superior tech underlying them) they'll be as toast as Digg unless they radically course-correct.
If reddit ever becomes a significant influencer of people's buying decision (like this article is claiming), it too will be gamed.
Think automatic gpt3 bots making up life stories of how he was hiking up mt everest and just so happen to be wearing brand XYZ which saved his life. Along with autogenerated selfies to submit to gonewild to farm upvotes and other stuff to create a realistic user history.
I can't see how reddit can defend against seospam of that type when Google can't handle the simpler problem of content farms. Reddit will die overnight from being replaced by 99% bot accounts.
They can't keep their website online and their search tech is so broken it regularly fails at returning my own sorted post history, let alone find anything.
You make good points:
>They have the data. They have the dedicated, active user base. They have free moderation.
...but overall I think they're likely to get wiped out mass exodus Digg style due to some black swan mismanaged incident before they enter FAANG tier
Have you ever used reddit? It's manifestly full of bots, with the second largest usergroup being schoolboys looking for porn and talking about video games.
There's a reason the ads you see on reddit are for absolute garbage products made by companies you've never heard of.
I add reddit to a lot of google search terms because I want to find discussion on the topic I am searching for. Most of the time I find an opinion, perspective, or more information on the topic I am looking for. Reddit is a lot of things, including hot garbage, but it's also a wealth of information.
Here's a billion dollar idea if anyone has the time and ability. Build a search interface that indexes tiktok videos and makes them searchable. To do it really well you might have to transcribe the videos.
Here's my VC pitch. Tiktok answers questions you never thought to ask, but if you can find a way for it to answer questions you do have, you have provided access to an obscene amount of interesting information.
Ive been experimenting with Kagi lately. It seems very promising. The results have been fairly reliable at all types of queries except the ones where I straight up ask google a question.
Yeah, I like Kagi - and I even talked to the founder about working there. It doesn't do well at questions, and doesn't have some of google's widgets (e.g. currency conversion) but it's better than duckduckgo.
But seriously, who actually needs widgets and all the other distractions?
I've used Kagi since December and I'm ready to pay $5, $10 or even $20 a month if they just continue to provide the same quality as they do today.
I mostly search at work and I come to a search engine to find things I search for, not to get suggestions for what I should search for instead, not to enjoy cute widgets and stuff.
A search engine can be as basic as it wants if it gets my results, but if results are equal obviously nice is better than ugly.
Ah! You are right. Google supports a level of indirection that Kagi does not. "100 pesos to dollars" or "100 pesos to usd". If you use the actual currency code, Kagi works.
I use kagi exclusively now and have since I was invited a few weeks ago. I've tried to think of about a usecase where I need Google like shopping but even then kagis result are better and Googles are irrelevant after the first few results.
When looking for an answer we don't want the average opinion of the whole world, we want the best opinion of the expert in that subject.
As the internet (and Google) reach out more and more, we get closer and closer to everyone and their opinion being online. And so the average answer online gets closer to the average opinion in the planet.
I know Google thought PageRanknwas the answer for that but they now rely as much (?) on people looking for X and moving on. Which means looking for "what is calculus" "most" people will hit a maths dense page and bounce for a less complex / demanding explanation.
All of which is a long winded way of saying if we want an Oracle to pick Truth from all the pages of the Web, we are not going to find that Oracle.
Humans and human science and curation can only do that.
you're really pushing that appeal to authority to its logical absurdity. how do you find the real 'best' expert and their 'best' opinion? everyone, even in their so-called field of expertise, is prone to error. limiting your information gathering to a single person exposes you to all of their errors directly, with zero error correction applied. taking input from many people, even non-experts (gasp! the horror!), gives you a much fuller expanse from which to make your own decisions. this expansive approach more fully covers the decision space, and provides in-built error correction.
and that's the basis of why many (uncorrelated/diverse) opinions are a better strategy than appealing to authority.
you might wonder why it's not better to take a consensus of experts only, but then you're back to the biased proposition of determining who's an expert in the first place. but more than that, "experts" (fashionistas) tend to be highly correlated, usually in a few well-worn directions (the fashions), even if sometimes opposing, because they don't come anywhere close to covering even a significant portion of the potential vector space.
this is the problem with google. their hubris leads them to erroneously believe they're better than the average bear at determining what people want in search (and ads). they're decidedly not.
I am going to go with expert as someone recognised by other experts. Yes that's horribly self referential but it is the basis of everyone who calls themselves doctors or scientist or engineer. In this case I can have expert plumbers and expert removal men.
And in this light an expert is someone whose opinions come from the foundations that all other experts in their industry recognise and agree with. Lysenkoism in other words is not going to fly - not without a shit ton of new evidence that stuns everyone.
So no I am not appealing to authority, I am appealing to science. There are views / opinions / statements that are falsifiable. And if the web is full of falsified statements (let's go with vaccines cause autism), then Google cannot say "51% of web pages that contain "vaccine" also contain "causes autism" therefore that is now true". (and no that's not how Google works, but unless google hardcodes what is truth, it can eventually only go with what is fed into it. And as we are seeing, what is fed into it is getting worse, partly through SEO and partly because more and more of the world is getting on line and we once had a web dominated by the pages of university professors and now it's dominated by drunk twitter rants. Sometimes by the same university professors ...
Humanity has great geniuses, evil scum, but the most of us muddle about in the middle. Science found a way to take the work of a genius and then keep it balanced there in mid air for the next genius to stand on.
We should not assume the best of us, the best of our work, the best of our actions, can be found by averaging the planet.
I do have hope - pop shows demonstrate that we can vote for great singers, so i do trust that mass voting will be part of the solution - I trust in democracy. I am just not convinced Google knows how to fix truth nor is set up for people to vote for truth.
you're making a statement of faith, not science. you want to believe that the experts you have faith in are consistent truth-seekers and truth-tellers, but that's a (political) belief, not a (scientific) fact. you have no way of confirming even a single sliver of veracity that way, especially not through so-called 'expert concensus'. rather than contriving a simplistic strawman, to find truth and facts in a social system, you still must use your own little brain to discern sociopolitical machinations (on top of merely technical observations), rather than naively trying to offload it to others. our brains have evolved over millions of years to do exactly that.
I have faith that a bunch of self interested scientists will happily gleefully point out the flaws in other scientists arguments and shoot them down. And that doing so repeatedly with just one agreed rule - the rule is that experimental results are the only, final arbiter in the dispute. Experiment says your idea is wrong it's wrong.
Of course scientists are not perfect truth seeking missiles - but the incentive structure i science and engineering (pace lysenko) is setup so that what works works abs everything else can take a hike.
An "expert consensus" in scientific terms is for example evolution - a theory that the most intelligent argumentative minds of ten generations has been unable to tear down. Not every scientific or engineering axiom is so robust but hells bells that sort of thing is worth putting my "faith" into.
Yes politics intrudes, but if the last two years has taught us anything it's that politicians standing against experimental outcomes are just like King Canute - the experimental results will simply overwhelm them.
And I honestly have no faith that our brains have evolved to spot other apes lying to us sufficently well that people can do much better than chance even face to face in the same room. Thinking we can spot lies across billboards, tv ads, trade routes seems a bit much.
We can however rationalise incentives - and I suspect you are thinking spotting other peoples incentives is spotting lies. it's usually correlated!
Although it's true that google results for subreddit in the specialty I'm looking for are (to me) top results (because I know I'll probably read from knowledged people on what I'm looking for), most of my queries are more about general trivia and other stuff that no platform like reddit can really encompass better than google it self.
I pretty much throw everything at google (like grammar, quotes, places, trivia in general, tech questions - reddit still isn't as good as stack-overflow for developers), Brave browser will take the ads out, and I get to choose my result. It's quite a nice experience.
(And no, I don't recommend Duck Duck Go either. It fails to show obvious results every now and then. I learned that the hard way.)
You can add !g to a DDG search and it will pull from Google. I'd say the quality of DDG is slightly less than that of Google, but not so often that sometimes having to !g the search is an appreciable problem. I've heard people say that DDG just uses Bing, but I'm not entirely sure how true that is.
The general problem with all search engines is that the moving target of the search algorithm often doesn't move fast enough anymore, and there's so much data that you're virtually guaranteed to have an overwhelming amount of wildly off-topic results. SEO farming has significantly damaged the utility of search engines, too. Finding obscure material is difficult because keywords are swamped, and it's exacerbated by the fact that the overwhelming use of search engines is for common URL lookup or to replace whatever invariably godawful embedded search a website might have. It's mostly DNS for people rather than a tool to actually search the web.
Speaking of, why is embedded search so invariably godawful? It's really quite impressive how useless it usually is.
Embedded search in arbitrary web sites sucks because it's a hard problem. The naive solution is to put all text in a database table keyed by "page" and do a sql "like" query. Don't ever do this. Some db's have full text search nowadays. I've implemented "embedded" search a few times in the past and I used Lucene, which actually works pretty well.
Maybe this anecdote illustrates the point "even quotes don't get you exact results any more."
When I search Google Maps for hotels or restaurants, it offers filters to apply to the results (price, quality, stars, etc).
If I apply the filters I want (4.5+ review, $$ price), the map continues to show other non-filter-passing businesses, cluttering the screen. The reason (given by the side panel list) is: "Here are some businesses that don't quite match your search".
*Well if I wanted to see those, Google, I wouldn't have applied the filters!*
All you've done is cluttered up the map which was the main thing I wanted to be able to see the location and distance of things exactly matching my criteria. If I wanted to get all the rest I would've removed my filters.
Makes me feel that Google is trying to apply too much suggestive content for reasons other than what users want, and that someone is causing Google to lose its way. (I know, it's just a small example.)
I think that Google's monopoly and their worldview that everything is data to be mined has created a feedback loop that killed the web of old.
The web was originally very social in nature. By linking to another page you spent some social currency to lift up others who were worthy. You, the other author and the audience all understood the social contract enshrined in a link. Google rejected that model and successfully changed the web to be a place where Google simply directed people to their final destination.
SEO became first possible and then necessary. The audience for hobbyists and anyone who can't pour money into optimising their Google results has dried up. The nature of Google search results undermines whatever opportunity the author had to build a relationship with their audience. Google reminds you that this is all an impersonal transaction: this page has the recipe you want, it lets you buy a product, or it answers your question. The value of this connection can by quantified as unique visitors, bounce rate and time on page.
Intertwined webs of relationships between pages have been replaced by a graph where the only edge that matters is the one that connects you to Google.
With the social dimension of the web undermined, and the audience siphoned off to the big, commercial, operations, it makes sense that creators migrate to platforms that will give them an audience. It is not that Google results are getting worse, it's that Google has accidentally killed the web as a source of valuable data for them to mine.
> Google reminds you that this is all an impersonal transaction: this page has the recipe you want, it lets you buy a product, or it answers your question. The value of this connection can by quantified as unique visitors, bounce rate and time on page.
> With the social dimension of the web undermined, ...
I agree. Whether Google did it by accident or not, we'll never know. But the consequence of removing personal transactions between pages is the impersonal, shallow, SEO-targetted web that we see today.
The thing that annoys me the most about Google's results is how they're intent on giving you any results, instead of actual useful results. Too often I'll type in something, and it'll give pages and pages of results that aren't what I'm looking for.
For example: searched something that "didn't have many results" - this is indicated (but somewhat hidden) at the top of the results, but isn't made visually obvious - well, I don't go looking for that to know if the results were actually what I searched for; if it gave me results, it's natural to assume the results were actually relevant.
But instead, Google decides to be absolutely less than useless by giving me back a bunch of irrelevant results, instead of simply returning NO results because there were none.
This is the main reason I've completely given up on Google.
Edit: another annoyance is Google altering your search terms - "searching for YXZ instead", or even worse, excluding some terms in the results (and having to later click "must include YXZ" which, again, is hidden within the results). This is particularly infuriating when looking up API terms.
I don't know that I agree with the thesis that it's dying, but I think the symptoms it describes are very real.
Searching for "good restaurants in <city I'm visiting>" is useless. Entire real companies exist to fill the top few slots on that search for any given city. My workaround is, as the article says, to search Reddit instead. Look for the subreddit dedicated to that city, then find their most recent thread on good places to eat - you find much better results.
That said, I strongly suspect this only works because it's not a widely-known strategy. As soon as a critical mass of people start going to reddit instead of google, the same enormous weight of effort people put into SEO will instead go to finding ways to subvert Reddit's authenticity. Sock puppet accounts, astroturfing, generating spammy subreddits, voting rings - there are plenty of strategies, and dedicated experts will have a huge incentive to invent more. Reddit will put up countermeasures, just like Google tries to prevent SEO spam. I don't have any reason to believe Reddit will be more successful in the long term than Google is.
> I strongly suspect this only works because it's not a widely-known strategy
I think this post in combination with HN users' comments indicates that this is a widely-known strategy. It's cool. I've been doing it too and didn't know that everyone else did it too. Reddit is already being secretly advertised on but I guess we'll see how things end up.
Sure, but "widely known on hacker news" is a very different bar. :) Really, the question is, "is it a widely known enough strategy that there's big money to be made subverting it?" And as soon as the answer is "yes," the battle will be on between spammers trying to fill Reddit with low-quality info and Reddit trying to keep them out without hurting legit users.
And the reason why these small communities are "more trustworthy" is moderation by actual humans. This is always the secret sauce. Google's biggest problem is that they think they can (and that they must) solve everything with automation, but any automated system can be defeated by a sufficiently motivated human.
"any automated system can be defeated by a sufficiently motivated human"
That is exactly the whole problem. SEO SPAM is a symptom. The problem, is that Google went all in on ML and AI. I've been exposed to ML and AI going back to the early 1990's. It always ends up the same, stale sterile outcomes.
Google can fix this by someone doing an intervention with Pichai and concincing him that we are several decades away, still, from any meaningful use of ML and AI, get back to making search good.
On a positive note, Google does work well today as a spell checker.
This is it. If you let the machines do their thing unsupervised, you're creating an optimisation game someone is going to win.
I honestly believe the only way for a search engine to be as valuable as Google was years ago is to let human knowledge drive more decisions. Maybe closer in spirit to what a web app store would look like, or closer to the categorisation Yahoo did. Not to browse by category (though even that would be refreshing at this point), but to better select the most relevant search sources.
This is an excellent point. There's the actual moderators who are, at minimum, filtering out spam. And then there are the human users voting, which I assume carries over in one way or another to Google's search ranking.
EE380 will meet online today, 16 February 2022 at 4PM Pacific
Speakers: Dimitri Kyle Brereton, Danny Sullivan
Title: Google Search Is Dying
Yesterday, February 15th, Dimitri Brereton blog link was posted to Hacker News (news.ycombinator.com). The comment, Google Search is Dying, garnered a large number of comments and responses.
http://dbr.io made #1 on Hacker News frontpage.
http://news.ycombinator.com as of 0:21 Pacific 2/16/2022
Google Search Is Dying (dkb.io)
3428 points by dbrereton 23 hours ago | flag | hide | past | favorite | 1489 comments
Today's EE380 is to discuss Dimitri's observations about Google Search. Danny Sullivan, Google's Public Liaison for Search will address Dimitri's concerns. Some additional panelists have been invited but are not yet confirmed.
Speaker Bios:
Dmitri Kyle Brereton is a software engineer at Gem, and the founder of BlogSurf – a directory of personal blogs. He graduated from UCLA in 2019 with a B.S. in Computer Science. He has been doing independent research on the question of how to organize information on the internet since 2020. He is currently working on a search engine for blogs.
Danny Sullivan is Google’s Public Liaison for Search. His role is to help the public better understand how Google Search works and to engage with the outside community to hear feedback on how search can be improved.
Contrarian opinion ahead: when everyone is skating in one direction, I try to look the other way to see what they might be missing.
With the rush to ML and AI in search and recommendation systems, I think there are huge opportunities in curation.
Curation is the opposite of where tech has been heading. Why? “Curation doesn’t scale!” People cry, but this ignores two important points: (1) it can scale via the crowd, and (2) it doesn’t necessarily need to scale to be valuable.
Regarding (1), others in this thread have suggested website up/down votes. Just one example of a scalable system. There are so many sites I’d tag as spam for my friends to avoid if I could. It’d be great if we could work together to eliminate the code spam sites, for example.
Regarding (2), I would happily take book recommendations from someone like PG even though he’s only read a tiny tiny tiny fraction of all books written. Critics don’t read everything, but they use their experience and social graph to find great culture. I’d value a domain expert’s opinion on great books over any number of Google queries.
I feel like there’s a massive opportunity here to “take a step backwards” towards curation and the high quality content it has traditionally provided. There are dozens of huge curation startup opportunities waiting for those willing to go against the wind.
Yup, search is terrible when looking up for very specific/ niche topics.
It only works as a fact machine now. For example search for "Who is the father of the president of USA?"
I think majority of this problem arises from academia/industry disconnect as well as greed.
1. Greed: Ad revenue.
2. Academic people + research = Let's create a general solution for all. They never bother to understand what problems users are facing.
You may wonder, why would any one care to ask stupid facts? Turns out people don't need internet for finding relevant information. They are already bubbled up, so they search for "facts" to verify or argue against their belief. Eg. "Kanye west and Kim Kardashian". And for these examples, google works best.
It's really HCI problem. It doesn't take for them to tune down on the ads, but why would they? If you search for niche, they just show ads because they get $$$. But if you search for facts they just give the highlight. And this small highlights create positive reinforcements among it's users. It manifest to common users that google works.
So google basically is fact machine to find clues for an argument or bubble up belief. It is utterly useless for anything else.
I have a slight contrarian viewpoint although there is some common ground. There is nothing technically wrong with Google's search, it's that the content online is not authentic anymore. And this is what people find on Reddit -- real opinions from real people. And the fact people append reddit to their google searches is a testament to the fact that Google will actually find it. And that Reddit's search won't, and I don't think it will be that easy for them to make a good search. If it's easy, I don't know why DDG gives me even worse results than Google. I tried, but for me it's borderline useless.
It’s definitely easier for people to froth at the mouth and blame Google, but I believe that the issue of pervasive SEO is a chicken-and-egg situation.
There’s no actual incentive to produce quality and authentic content. Quantity is the name of the game in the click economy.
On the other hand, one could make the case that Google is at fault for creating the conditions necessary for this perceived decline. (dominant search engine, ad platform)
I don't the same level of vitriol towards ads as some, particularly on something like search where there is a clear intent to find something and an ad may well be the most appropriate result. Like if I search for "Bosch vacujm" why isn't an ad for a retailer selling one the most appropriate?
Just so long as ads are clearly labelled as such I'm completely fine with it.
But the whole content farming thing is much worse and it explains the "reddit" thing. Searching for reviews is now impossible because of all the astroturfed affiliate link spam. Adding "site:reddit.com" is one of the few remaining ways to find real people talking about something. That woo will probably end at some point.
But this is a good example of how a metric that becomes a target ceases to be valuable as a metric. In this case, the links between pages became a goal so those links, the content on the pages and the SEO became a game and it doesn't matter if it's Google or someone else. If anything, affiliate links are a much bigger problem because they fund this "industry".
There will always be a need for search. Google search isn't going anywhere.
Agree with the premise, but seems to me that the article does not justify this. I can understand ads, but ads do not affect search results. If you move past ads now (which most users do as they habitually ignore the space where ads would be) even then you should expect good results.
SEO seems to be a big problem. Just saying Google is big and they should fix it ignores the nuance and the whole cat and mouse game that goes on. Eg: I am based in India, and am looking for which cable channel/streaming service is broadcasting a game of my favorite soccer team. The first 10 results would not have the answer, but as a user you would only know that after opening the link and reading through 500 or so words introducing teams, opposition, competition, form etc. but not what I am looking for. Most of these are news websites, who would make a loud noise if their results do not come on top. For a search engine relying on signals (even with AI), it's an incredibly hard problem to know if those 500 words would have the exact answer. [1]
Reddit is good for searches where things are in flux, or when it's a user centric thing. Because they have done the SEO well. Similarly the results leading to Stack Overflow for developers are equally important. Yet, when you want to research on some topic, or learn more, you would inevitably start with Google.
If I were to predict, Google would start identifying trends and slowly start ranking reddit higher for user centric queries. In my limited dev experience, that is already happening for Stack overflow. I love how the results are clubbed together under the first result.
[1] The result which surfaces often include the direct question: "How to watch team A v team B game in India?". How do you design algos to combat that and yet include legitimate results. Have a lot of text on the page is often the most given advice on SEO.
> If I were to predict, Google would start identifying trends and slowly start ranking reddit higher for user centric queries. In my limited dev experience, that is already happening for Stack overflow. I love how the results are clubbed together under the first result.
Weird, I'm having the opposite experience with stackoverflow pages. Often I get pages from random websites that copy and paste stackoverflow content with some jammed-in SEO ABOVE the actual stackoverflow results.
In principle, I can't contradict anything in this article with my own experience.
The coup de grâce for me came with a cell carrier switch, after which Google began constantly nagging me to complete captchas when I searched. That annoyance started amplifying all of the other piques I'd had with Google (including progressively less-relevant results as time went on).
I found myself constantly using quotes and minuses to get the results I wanted from Google, and appending keywords like "wiki" to prefer specific sources.
The day that the realization hit me- that I was really just using Google as an unnecessary entry point for other search engines, all of which had their own perfectly good search interfaces- was the day Google ceased to be my default across all of my platforms.
My default "generic" search engine is now Qwant, and if I specifically want Wikipedia or Amazon or eBay or Reddit or anything else, why, it's just a simple drop down away. Level of abstraction removed, and as a bonus, the blasted captchas are gone.
Although I don't use Reddit that much I must say Reddit is diamond in the rough; communities are super helpful and unlike Facebook it is not walled garden plus you don't have to use your real name. It still needs work and improvement but I really like Reddit.
reddit is absolutely a walled garden. They'll instaban you for wrongthink at the drop of a hat. Powermods will ban you from many subs for participating in one they think is guilty of wrongthink. And if you create multiple accounts ('throwaways' are encouraged in many subs) but then fail to track which subs have banned you which of your accounts for wrongthink reddit will ban all of your accounts for "ban evasion" even though their own help page tells you how to copy subs from one account to another: https://reddit.zendesk.com/hc/en-us/articles/205243365--How-...
And most subs won't let you talk on a new or low karma account.
Peak reddit was 2015. Giving moderators the power to lock posts was a sign of authoritarianism to come and it's only gotten much worse.
A bit of both, I think, they feed into each other.
Google being useless for something like product reviews means that any smaller sites that are any good are not getting the traffic they deserve, because they're being outcompeted by the ads and the fake review sites.
People still need the content, though. But how do you find that review without a search engine? You have to resort to that even though Google sucks for the general web, it's still very much useful if you restrict it to something very particular. You have to know where to start though, so you have to resort to one of the 4-5 huge sites like Reddit or Hacker News where you know people are going to be discussing whether a given wifi router is any good or not.
Which has the side effect of concentrating things even more on those sites. If the place you can trust for networking equipment reviews is Reddit, then probably you'll also comment somewhere on Reddit next time you get bitten by a bad one and want a second opinion or just to warn people. And reddit gets bigger still.
I don't know about dying, but definitely more hidden from view. Google owns a huge share of the search market and a lot of the remainder information discovery is on the large social portals.
Alt search engines do offer an alternative view to what's out there, but are not on the scale of Google and many of them are ultimately meta search engines relying on Bing for crawling and indexing.
I haven’t read any comments. That being said I think a lot of what this is saying is anecdotal. Anecdotal evidence is good in this case because it would probably be hard to get good data on what this is trying to prove. I think some of the problem is that most of the people who google something don’t care that they use google or even think about what they type in. People who are using quotes to google things I would already put in the advanced category of googling. I google things every day for programming matters and most of the time it’s good with the short answers it pulls from SO or bringing up documentation as a first result. But not once have I used the quotes function to google something. If the quotes not matching was a big argument of his and then later revises that with “oh the quotes do work… for now” I don’t see why this has so much traction. As for Reddit being so popular I think needs more of an explanation. I think one large factor is Wall Street bets. That probably brought a lot of people and kept them. And beyond that people who heard about WSB had never heard of Reddit and downloaded it and kept using. Reddit is nothing more than a forum. Is there a fine line between a forum and a search engine? You tell me.
In the mean time just learn how to google. I google obscure questions about Flutter and specific flutter packages and google does fine. If not then I switch to duck duck go.
As a trader, I can say that at least in Finance, the dead internet theory does apply. Type any stock ticker into google, especially a small one (random example: MRAM), not a single article for "mram stock" is written by a human, you have AI generated pages such as "stocknews.com", and dashboards like FinViz (which fwiw is a good and useful website). The articles are generally generated based on quantitative (and therefor easy to automate) aspects of stocks, for example price to earnings, or that the price has grown alot.
A second type of website exists, which I learnt of from searching tweets in google, and finding that algorithmic spinning (replacing words to avoid plagiarism detection) was clearly being employed. The grammar is often laughably bad, but clearly in terms of google is either the best content available (as their algos see it), or the only content available.
One more Note: Google Trends data is 'deflated' to the number of searches (you can verify this by using Google Ads which gives numbers of searches and comparing to trends data). I assume the method is similar to what is in use for the transparency report's "censorship / outage" feature (explained here: https://transparencyreport.google.com/traffic/overview?hl=en)
While the content on Reddit is probably more authentic than what you're likely to find on Google these days, there's still quite a lot of obviously (or not-so-obviously) corporate-sponsored stuff, and I imagine this is only going to get worse. For example, see https://www.reddit.com/r/HailCorporate/. I think this is unfortunately the fate of any platform that gets sufficiently popular.
There's disincentives for Google to do the right thing, for sure, e.g., ignoring quotation marks. I assume this is so that you never see a blank page (and so ads can be shown, which would be weird to see if there were no other results).
But, as the author mentioned, a lot of the problem is the inauthentic content on the internet that Google must sift through and filter. What makes Reddit still not half-bad (although this quality is under direct attack by brigading and troll farms) is that you have user-generated, user-curated content and a not-too-bad voting system.
In this context, I think a future iteration on search engines will be hand-curated results, under actual human-curated topics (rather than fuzzy machine-learning-inferred ones). Think of a huge directed acyclic graph of topics that goes down twelve levels or more in some cases. If you have enough people involved in this kind of crowd-sourcing, I think it can be made to work.
A challenge that arises in this context is how to prioritize content added to the wiki search engine by good contributors, and deprioritize content added by the content farms. I think this can be managed with a combination of well-conceived reputation management and providing users the ability to specify other users (people who seem trustworthy and whose tastes are solid) whose preferences will then be used to weight search results.
Search for "carbon monoxide" on Google, Bing, and DuckDuckGo.
Google serves you an entire page about carbon monoxide poisoning, and recent news stories about carbon monoxide poisoning. You have to scroll through a lot of junk to get to Wikipedia's entry on "carbon monoxide". Bing and DuckDuckGo do a serviceable job telling you about the substance CO.
You cannot search "carbon monoxide" to learn about carbon monoxide, and that is the issue.
Odd how much Google became so vastly different between users. For me DDG and Google do just about the same thing here. Wikipedia first link (well, 1st at Google, 2nd at DDG) with a card and then governmental websites with regards to health and hazards. Both have one row in their own card dedicated to news.
The only difference is that Google also gives me links to Canda and Québec govs links (where I live) while DDG is all American links.
DDG is absolutely not better at giving me a chemistry lesson than Google is in this instance, they're both all about poisoning, and to be honest it makes ample sense.
With the following text just bellow the search box.
> Carbon monoxide (chemical formula CO) is a colorless, odorless, tasteless, flammable gas that is slightly less dense than air. Carbon monoxide consists of one carbon atom and one oxygen atom. It is the simplest molecule of the oxocarbon family.
Odd I don't see that. I am told "People also ask" about five different questions that each of them amount to "what is carbon monoxide poisoning", which replicate the results of the search itself.
the fact other search engines get this right, even if not nearly as well as Google would have in 2008, tells me the problem is not advertising or a contentless web as much as Google specifically and deliberately preventing you from searching about what you mean.
There is no innate reason why a Google search on a subject should return high quality results. This is predicated on there being someone willing to write thoughtful, informative, well-researched content and post it on the open internet such that a search engine can monetize that content via user profiling and advertising.
There may have been a moment when enough people were willing to put up their writing/images/videos for free such that Google's search engine appeared helpful in "organizing the world's information". But that mission statement was is a smoke screen. Google didn't organize. The company, as a gatekeeper, profiteered off of the writing/images/videos of others.
The problem isn't that Google search algorithms are low-quality, nor that Google has been gamed by SEO. The problem is that Google has engaged in a scorched-earth policy of capitalizing on the work of others. Google created a secondary market in information, without funding the primary market -- which then withered. And now there is a tertiary market of SEO spammers capitalizing on the propensity people still have to think that a Google search will return the truth to them, gratis.
Reddit is a great source of people's discussions and quality posts on various topics. When I tried to find people's pain points to automate collecting problems to solve(and now it's an advanced Reddit search tool[1]), I found out that niche forums are great places to collect them. However, it's difficult to find more of them, plus scraping the data is time-consuming(custom parser for a new forum). And, the most important thing, there are not so many discussions that one may find on Reddit.
What I like about the website, it's you can find a huge amount of subreddits, every one of them dedicated to a niche topic that people there are willing to discuss. They share opinions, actively engage in discussions, and help in moderating good content. Is there any other place like this?
There are many situations when one still be preferring Google, but as for niche discussions I don't see any other good place to visit. Maybe it was Quora before, but now it's a spam place.
Reddit is incresibly heavy on censorship. It is only a great source of topics if they happen to align with a very specific worldview. I recommend to have zero trust in Reddit when it comes to politics, history, health, and even philosophy.
> I work for Google Search, passed your feedback along, thanks. You said in the post that quotes don't give exact matches. They really do. Honest. Put a word or phrase in quotes, that's what we'll match. If anyone has an example where they feel it doesn't, please let me know...
"cheese bacon bread" seems to match "cheesy bacon bread", "cheese & bacon bread", etc. I'd link some example results but don't want to feed the SEO beast. I visited the sites, I viewed sources, and did not find that string. I assumed what is happening in the 2nd case is that "and" is treated like "&" and both are "punctuation" in their model. And in the first, cheesy=cheese.
While it's a trivial example and might be what most people want, it doesn't help with more technical topics. And recipe blog spam is a particularly sore point for many searchers.
Is Google still a Search Engine? Or is it rather an Answer Engine?
Answers are more-and-more provided on what used to be SERPs, but now is too often dominated by answers on the page, ads, and big marketing budget SEO optimised landing pages.
We still believe in the value and power of discovery; call us old-fashioned but we focus on 10 blue links using an independent index. Your vanity search maydisappoint, and our ranking needs improving, but you will find often hidden gems and information rich sources. Plus we send you to those rather than demanding your eyeballs.
This is an important distinction, albeit semantic and somewhat contrived. If I believe the earth is flat, I may find much evidence to corroborate that theory via Google. Though I may ignore the overwhelmingly greater volume of evidence to the contrary, I will still find what I want. So is it now a search for "the facts" or "the facts that I want" ? The AdSense model is arguably tilted towards the latter.
It is. We also see this as an issue of "objective search". Two users we think should see exactly the same set of results, for a given query and settings they can control (eg location/language setting). That's another positive that comes from our stance on no-tracking; with no harvested personal data, we can't "personalise". So no steering users into content reinforcment whirlpools and/or ad-pulling filter bubbles.
This gets to the heart of of why we created Neeva.
A product that fundamentally breaks the misaligned incentives between serving results for you and serving advertisers. I have been at this for a long time and the freedom to innovate that comes out of removing ad-revenue is liberating.
Our team is constantly creating new and better features to push what can/should be expected in search such as FastTap that gets you results without sending you to a SERP, or enabling you to search across third party apps like Dropbox, or more useful info about the page you are on.
We recently launched a free and paid premium version with the goal of making it available for anyone to try it.
Are we at parity yet with Google? On some things I would say yes, on others no, but is anyone else? What I can say since joining is that the product and deeper query results get better everyday.
Barely six months out of beta and we are running fast, feeling like the old days when building Chrome...
Google search still works well for many things, but is SEO spam/bot infested for anything that requires a hive mind opinion about something from mostly genuine people. Like if I want to know what kind of a best in class/best bang for buck/newest [insert product] to buy and have no idea where to start, Google is often the worst place for it.
I'm not sure if that's because most opinions like this are shared on social media like reddit (and in rather unstructured form) instead of on blogs/websites that Google mostly indexes, or if it's just really difficult machine learning problem to formulate something resembling a non-spammy consensus opinion from experts by just crawling shitloads of websites that all try to SEO spam the crawler
>If you’ve tried to search for a recipe or product review recently, I don’t need to tell you that Google search results have gone to shit.
Not that I usually do but I tried for macbook M1 and apple pie and it gave me ok results - Tom's Guides and a BBC recipe
Competition - I don't really know anyone who uses Bing or DDG though I believe they are out there somewhere
Ok I sometimes stick reddit on the search which is fine because I like Reddit. I guess the ads are probably annoying but I don't see any due to uBlock.
I'm not sure why I'm the only one saying it's not dying when that seems to be what the facts suggest? Nostalgia for some mythical past when you got unbiased results for "best laptop" or something? Not sure really.
For what it's worth you're not the only one. These Google search is awful posts happen a few times a week and I always wonder what I'm doing wrong to not notice. Seems to get me the results i want which is usually wikipedia or SO but then I'm not expecting Google to perfectly review products. What are people not finding and what are they expecting their search engine to do for them? Seem to be a lot of general rants
Yeah I mean it's changed a bit in being more AI like which is better in some ways, worse in others. So I guess if you like to old way it could be annoying.
I am soft-launching unfluence.app in the coming weeks. It's live now, though not yet marketed.
It is a platform for finding and sharing recommendations within your own trusted network. I'd love to hear your feedback!
You can read more about it on the home page[0], from its inspiration, comparisons with existing solutions, to a down-the-road monetization model that aligns with the network.
It is being built by Kujo - a brand in the lawn care industry, and so is seeded with products and brands for that community. The initial launch will be within the lawn care community. However, the platform is community-agnostic and supports creating communities for any groups.
I did not know that many other people also append "reddit" in front of their search queries. Now that I think about it I have stopped using google search for things I really didn't know about. For all things I didn't know or need to learn about I append "reddit" in front of the search query on google or go to YouTube for video instructions. I use google search exclusively as a short cut to typing for a link, or stuff I know it has already indexed like a place of business. For example I want to go to an imdb parents guide for a specific movie, I just type the name of the movie and parents guide and google shows me the link -- this saves me a bunch of clicks and page loads.
There’s a strong possibility the author is totally wrong here. Reddit search is famously broken, so the only way people can reliably search Reddit is via Google. (Because Google search is…broken?)
The default subreddits are really horrible. Reddit is still okay if you unsubscribe from all of those and only subscribe to the narrow subreddits that you're interested in. Of course, it's hard to find new subreddits that way.
You might be lucky and there is a subreddit for your favorite topic that has very strict rules about memes or lazy content. I wish more subeditors would encourage lazy content or have a "fork" with such rules.
Something as simple as subsubreddits would solve this. I.e. a "funny" subfolder or something, like proper forums are organized in different sections. But Reddit want a eternal feed to show as much ads as possible. Low quality posts makes them money since you have to scroll by them and thus sees more ads.
I won't blame the company, is the community. I don't use reddist as much but as a Star Trek fan I see there are a lot of subeditors, one for each show, a generic one, one for the haters, ones from memes, a more technical one etc. Reddit the company won't care if you spend your 30 minutes free time on the no-jokes one or on that hates-everything one ... you just need a big enough community that would enjoy a more niche and strict subreddit.
What do you expect though?
To go on home page of Youtube or Reddit and find content exactly on your taste?
That is not reasonable, if I go on private mode on youtube homepage I am not surprised to find the most popular music that my countrymen are watching, stuff I dislike, so I bookmarked my youtube subscription page and subscribe to stuff I enjoy and use the search.
Although I've yet to evaluate Kagi (though I did get a beta invite the other day), the only search engine that seems to be not totally nerf'd today is Yandex, and even that one has problems (lots of foreign language content). I still primarily use DDG a lot, but often times I have to go to Yandex if I want a more exact match.
I do have to wonder exactly how Gen Z and Gen Alpha are using search engines, if at all? This isn't to say I think they're not bothering with search... but that I just don't actually know. Might it be that the use patterns of the youth are influencing how Google and the rest of the search engines are tailoring their algorithms?
I stopped using Google in 2021 because I found that I was getting better results and less ads from Brave Search and Duck Duck Go. Recently, I signed up for the Kagi Search beta and have really liked it, especially the "Programming" tab, which limits the search results to programming-related results.
My only concern with Kagi is it requires you to create an account. I don't like Google tracking me and the idea of Kagi knowing what all my search terms are isn't appealing. At least they aren't planning on selling it.
It seems like we're approaching what I call the "dismal equilibrium." This is the idea that any free site/service/app eventually will have to monetize itself in order to remain free, inevitably in a way that degrades the experience. Ads, typically, "pay to win" for games, or perhaps even calls for donations for public radio. An equilibrium is reached when further monetization isn't possible without driving away users; quality is just barely tolerable, hence, "dismal."
I think this is an interesting topic - for me it really highlights the problems inherent to making algorithms profitable. Often pushing them in one direction or another has really pronounced effects on their unbiased nature. I personally think PageRank is still the best algo around and there are not too many good copies. The other thing to consider is that allowing for selection of 'common searches' reduces server load and is computationally less expensive than processing the same search over and over. Also, the way people ask questions may have changed. I mean I know many people that use the 'omnibar' to go to a webpage they know the address of. Like searching google for 'Facebook' just so you don't have to type '.com'.
I remember the days of AskJeeves when your query literally had to be a question - that was very tedious. I am not anxious to go back to that if thats what a decentralized internet looks like.
But I do think we are on a precipice where the size of the company plays a huge role in getting noticed. If you want to stop this don't click on the 'ad' links in the search results. Scroll down until you see the page you want to go to.
@fxtentacle It occurred to me that Microsoft may have blocked the Google Crawler so that people have to switch to Bing. I am really not a fan of how much Microsoft is trying to force people into their ecosystem and are rapidly closing the doors. Took me two hours to figure out how to remove Windows Defender from a VM.
The article is right, but I think it's missing the main cause.
People who used to make high quality web content have moved to youtube instead because you can make more money there and it's probably easier.
Add to that, I think because of the move to smartphones, google tries to give you a direct answer to your question rather than directing you to sources where you could educate yourself to answer your own question which it did more in the past.
But yeah google search is noticeably worse and I don't know that google can do anything to fix it.
The author's thesis is a bit confused. Reddit hasn't replaced Google as a search engine. Reddit does have a search function, but have you used it? It's frankly terrible. Rather, Google has become the search engine front end for the huge database that is Reddit. Currently they have a symbiotic relationship, but if Reddit ever decides to take its data private and build its own competent front end, it could potentially splinter off for itself a good chunk of traffic.
For whatever reason, Quora has been hogging up my search results for the past month or two. It happened suddenly, and now I'll have to use site:URL or similar to get the desired results.
And agree on the reddit thing. Their search engine sucks, and you're stuck with using search engines like google to find anything decent.
Edit: Should be mentioned that google still yields decent results if you're using quotation marks and logic operators - but for free text, it took a nosedive.
It's annoying Quora makes us sign in, but at least the answers are written by humans. On average, answers seem better than affiliate link blogspam found elsewhere.
I agree with this article, but I can't entirely agree that Reddit is a good alternative for a search engine. As much as I appreciate the content on Reddit, though, it's a database and a forum, not a search engine. I often find myself searching Reddit by either 'site:Reddit.com' on Google or, as of recently, using you.com, which I am positively surprised with; I'd say you.com search of Reddit is probably the best right now.
The biggest issue I have with Google is that every search is performed in the "now" context. This makes looking back, especially on political issues, basically impossible; There's no way to explore how topics have evolved or progressed over time. I don't mind google search for resolving technical issues as it works pretty well in this context, but the second you start to get curious and look for anything older everything breaks down.
I've found a solution to this - you need to use the internet in any language other than English. You get much higher quality results that are not so full of ads and SEO and all of the other evil stuff which is going on. Its not perfect, but if you search for "pancake recipe" in French or German you have a much higher chance of being linked to some "old internet" style blog rather than some food.com bullshit.
All I want is a feature to black list certain domain names from search results. Similar to YouTube never show this channel again option. If google hosted such a feature then they would get a very strong signal on poor results and would go a long way to punishing bad behavior and cleaning up the net. It’s so easy to do that I have to assume they chose not to because they make a percentage of the revenue from the content farms.
There is a fundamental change in Google Search and its purpose.
Google Search no longer is a Search Engine. It's now aiming to be an answer engine.
That may sound like a distinction without a difference, but in practice the changes are profound.
With an answer engine, your goal is to find THE 1 correct answer. So your algorithm consistently refines and eliminate low "accuracy" results. If you reliably knew that only 5% of users found a certain response useful to their query, in an answer engine you get rid of that response because it's clearly "wrong".
In a search engine, however, you retain that response. In fact, you signal boost it, because it clearly shows that even though there is another answer that is chosen by say 90% of users, the 5% usage for the 2nd response indicates that even if it may not be the correct "answer" to the query, it's of value to folks, and is related to the query being searched.
An answer engine eliminates serendipitous connections, because almost by definition those connections are not the answer to the query being asked. A search engine not only doesn't punish a serendipitous connection, it seeks to surface it.
Ironically, Reddit search is terrible so you really have to use Google to do a thorough Reddit search. reddit + search term is a powerful combo on Google.
https://camas.github.io/reddit-search/ will sometimes find things that neither Google nor Reddit search can dig up (it queries pushshift and includes recent and deleted comments). It doesn't have any relevance ranking but it's still possible to get interesting results; also, it supports asterisk suffixes on words.
I do this also, especially for problems/questions related to real-life situations. Someone on Reddit has either already asked that question or someone provided an answer. Google should learn from this.
Also, I think that author should have mentioned the new crop of AI writing tools that have been coming out in troves. And, honestly, some of them do a pretty convincing job of writing things like blog post intros or specific paragraphs.
And, best of all, all this "progress" is driven solely by monetary interest. Google has made millions of people rich, and for a while will continue to do so.
Lastly, I'm bit of a digital marketer myself. I have been in the game for a loooong time, too long. And, I can say from personal experience - a lot of the top 1 results on Google are still being gamed. You can, technically, report blackhat spam[0], but who knows how proactive Google is to listen to those reports.
The weird, possibly beneficial, consequence of Google becoming increasingly awful is that I've begun aggressively building my technical book library again.
I've always been a fan of technical books, but would almost never buy classic reference texts because if I just needed to look up an idea or concept real quick I could usually find an adequate explanation online.
The problem is that content marketing in my domain (stats/data science) has gotten so bad that nearly all of the results are Towards Data Science and similar garbage articles, written by relative amateurs that were rushed out to get ranking for a longer tail of search terms. The number of times I've researched a topic I know well but want to understand some nuance of only to find results that are at best naive in their understanding and at worse outright wrong is astounding.
Now whenever I see any recommendations for good books I buy them, even if I don't have time or immediate interest in reading them right now because I know that if in 6 months I have some relevant question I'm likely to find the wrong answer online.
Last night while doing a search, I found myself pondering the fact that recently I've been using DDG more than before.. and it's not because DDG has become so good, it's because Google has become so trash.
Ironically, only a moment later I noticed on an IRC channel I've been on for nearly two thirds of my life that someone just complained about Google giving nothing but SEO trash.
I have been appending reddit to my queries for 1-2 years now. I agree with everything in the article, but I believe that the whole AI trend had a larger negative effect on Google than more ad optimization.
Just like the gmail effect, teams internally have been pushing to integrate AI into search results. Not necessarily because it's the best thing to do, but because someone needs to get promoted. They can't just leave search as it is.
Of course, "best thing to do" is meaningless. What are the metrics? Getting reliable metrics and running big A/B tests is really hard if you have to measure fuzzy things like user satisfaction instead of concrete metrics like CTR. But that's really what's going on here. Initially, users may have been clicking and interacting with results more, but after realizing that those results are not actually what they wanted, or are SEO spam (hello Medium), they become disillusioned and append reddit to their query.
> Why are people searching Reddit specifically? The short answer is that Google search results are clearly dying.
What’s the connection between Reddit being searched for and Google dying? Read the article, doesn’t make sense.
Might as well say that GitHub is dying because Discord is where many projects have community discussions.
People are always saying Google is dying or search results are getting worse. How many sites existed in 2010? How many in 2022? How prevalent was SEO and content marketing then vs now.
The fact of the matter is that the web itself is becoming more littered with spam. Literally on HN there was a thread on how to make 50K a year and one person proudly stated they did so by using GPT-3 to create spam related to content they were selling.
Inherently any search engine with programmatic results can be gamed programmatically.
The chart in the article is easily explained by the fact that it’s hard to search those platforms using Google and that the internal search is more useful.
> What’s the connection between Reddit being searched for and Google dying?
Google returns page after page of seo garbage. You often have more luck finding what you want on reddit.
> Might as well say that GitHub is dying because Discord is where many projects have community discussions.
The point is that a good search engine would find the result you want without requiring you to go out of your way to specify the site on which you're likely to find that result. It gets worse when you think that these reddit results often provide links to what you want on the web. Somehow google can't do that.
> The point is that a good search engine would find the result you want without requiring you to go out of your way to specify the site on which you're likely to find that result. It gets worse when you think that these reddit results often provide links to what you want on the web. Somehow google can't do that.
This seems too handwavey - what concrete metrics would you use to evaluate the quality of a search engine?
Reddit has even more garbage than Google. The only difference is that people can say so on there, unlike on Google.
Reddit search doesn’t even do basic spell correction.
> You’re missing the point - a popular sites own lack of internal search would explain the use of external search.
Sigh. The point is that 1) Google gives trash results by default 2) users know there are good results out there on the web, in particular on reddit 3) people append reddit to their google search query and suddenly good results start popping up 4) if google search wasn't trash by default, people would get good results without having to specifically direct the engine to reddit.
> People use Google to search Reddit because Reddit has terrible search.
Hey why don't you prove your assertion.
> Your other assertions would need to be proved more rigorously. Not just for Google, but for any search engine.
These are not my assertions. It's a claim the fine article is making. And they're not the only one making that claim. Anecdotally a lot of people are saying they do this to get useful results because Google results are trash.
I reckon the parent is correct about reddit having poor search capabilities. I often search for terms with site:stackoverflow.com as searching within stackoverflow consistently asks me to fill in a captcha challenge or gives poor results.
Honestly, I would blame Discord for Google's inability to return good results as much as anything else, a bunch of the new "authentic" discussion has moved there and other non-indexable platforms.
Google went downhill when they changed the "Google Instant" algorithm from what people actually are searching for, to something more aligned with some leftist ideology based around what they hope people are searching for. This happened around 2015 [0], and was in full effect by 2017 [1].
I have absolutely been adding ‘Reddit’ to my queries for 2+ years and waiting for this kind article to bring some discourse about shit google results.
“This [AI-created content being widespread] isn’t true (yet), but it reflects some general sense that the authentic web is gone.”
It isn’t gone, but it is different. Reddit is essentially a site of blogs turned inside out. Each post produces individual comments that are often really blog posts tied to commentary/chat discourse. Problem is, each post and it’s daughter discussion/blog posts isn’t useful for continuous coverage of a topic (e.g. cooking). Thus the subreddits exist with quality control through mods that curate content.
Yet, something is missing when there is a single umbrella organization with power over these fief post blog chats. I don’t want to read archives from 2005, but it is the last time it feels like the kind of personal blogs I find here on HN were prevalent and searchable through places like Google. Each article is presented in the context of the user/owners wider work and enriched and enriching for being presented that way.
The ‘authentic web’ of 15 years ago was better, more pluralistic, and more diverse in literary and artistic design when there were more ‘online magazines’ in this way.
This death of Google feels unlike the way Usenet died. I was less broken up about that death when it happened precisely because the web offered a broader, richer landscape. What I Think we are being taught, though, is that perhaps USENET and the web should’ve existed together and been supported, since Reddit is just Usenet, after all, in many ways.
Google is like a former ritzy neighborhood that has been corporatized, had the blood sucked from it, is falling into disrepair, and now is ghettoized and awaiting gentrification, which will probably mean a return to the walled gardens of yore when they start charging for improvements (as in Youtube Premium).
I agree with the article. It's getting harder and harder to get good quality results. Most of the time I use a site:reddit.com or site:news.ycombinator.com prefix, depending on the type of content I am currently looking for.
Lately I've noticed another breaking change. Typically I phrase my queries in English despite being from Poland, due to higher quality content. Over the past month I've been getting more and more Polish results despite the query language. Case in point for anyone who wants to test - "garmin fenix 6 vs 945 comparison".
Search seems a bit off on other Google services as well. Most notably YouTube, which interweaves results with ads and recommendations. Video discovery is becoming increasingly more difficult and it feels like I'm stuck in an information bubble. Which surprisingly works, as I use the website longer, despite it being less entertaining than beforehand.
Up until a month ago, when I searched Youtube using the "latest" filter, I could reliably get the latest videos uploaded that were relevant to the search terms. Now, it shows a couple of recently uploaded videos followed by many which are for weeks ago, while I know that many more had been uploaded in the recent days.
We're not far from a publisher revolt against google - essentially if you're a publisher with good data, doing the legwork of curating and moderating user generated content, and making it discoverable... google is just cherry-picking your content and laying it into the search results in the form of answers and snippets and plastering their ads on it (or AMP-serving your content).
I don't think it's sustainable in the long run and the barrier of entry to make a solid search engine is lowering every year - there are several solid alternatives where before it was a unfunny joke to assert there would be a proper google competitor 5 short years ago.
I don't think Google is going to fix this, the fact that it has evolved to where it is today is a result of concerted and persistent product focus in that direction.
High hopes that we'll see a better-than-google alternative break out in the near future.
What if it's not google results that suck, what if it's the internet? Reddit is (in theory) what we wish the internet still was: a bunch of loose communities with people sharing and discussing content, both original and not. The internet at large has become primarily different forms of ads. There was a time when the internet was littered with ads in popups, then they became banners on the side, now they are the content itself.
It's feels silly to wish there could be an open version of reddit because that's what the internet is. It's just that there's so much noise now that it's impossible to find the signal. At one time google was that filter to find the diamonds in the rough. But now they have no incentive to filter that stuff out, because 9/10 times, the rough is THEIR ads. We need a new filter that's not funded by advertising.
I’ve (finally) come to the realization that most websites are trying to sell me something. It’s usually affiliate link spam, or the articles provide just enough info and then ask you to sign up for their newsletter or buy their ebook or subscribe to their service or whatever other predatory monetization bullshit they’ve implemented.
I get it, websites cost money to run and providing useful information for free is a bad business model. My issue here is that Google search rewards this spammy behavior in order to maximize cash flow. And this type of thing works very well on normal non-tech-inclined people so it won’t ever go away.
I dislike Reddit’s current browsing experience, but the value of the platform has always been its smaller interest-focused communities and the ability to access the opinions of actual real humans instead of content marketers.
Google's home-grown recipe metadata format, and a single WordPress plugin to create recipe blogs, are the reason you can't find a decent recipe on Google.
Google's search engine is, without a doubt, superior to all alternatives. The fact that it's full of ads and junk is a conscious choice. Google could turn all that crap off tomorrow, and it would go back to being the best search results. Nobody has invested as much money in accurate results as Google has, and nobody will get close for years.
Search is, itself, dying. Search is probably one of the hardest things you can do with technology. We've gotten to the point that there's just too much shit to search through in too many ways. We need to stop relying on search, and start curating knowledge. "That's impossible", you say; I direct you to Wikipedia.org.
pretty much what we're up to at Breeze -- we currently curate internally, gradually opening up public curation -- dm if want to test some of our upcoming topic searches ahead of public release
The logged out views of reddit only show a couple comments from each thread, and then the pages are full of hidden comments from other unrelated threads.
So if I search for some exact text on reddit, google will often present an unrelated page that doesn't contain the queried text-- yet it does contain it: hidden. Actually finding the real thread with the text is a nightmare unless you know of some of the few reddit full text searches out there.
Sadly, even the broken logged out reddit interface is still often a better thing to search than google... but only in the sense that southpark's "IT" (spoof of the segway announcement) beat dealing with the airlines. ( https://www.youtube.com/watch?v=SK362RLHXGY )
Why oh why can’t the author make a point without resorting to a sweeping conclusion drawn from an anecdote?
Google search is simply an indexed representation of the indexable web. If you’re seeing SEO spam, that’s a reflection of how the web has evolved thanks to the most popular monetization mechanisms available today.
Reddit is simply a great site for user generated (mostly) textual content. It is not comparable to a search engine . . The popularity of “+Reddit” strings appended to the ends of search queries likely pales in comparison to the volume of overall search queries. One can investigate the differences through Google trends where one would see the string “tiktoks” beats Reddit this past year.
Articles like these full of self validating biases such as “My Opinion of Google search is everyone’s opinion and here are some selective quotes to show I’m right” are childish.
Google is absolutely terrible for anything even closely controversial. Their algo is too bias towards approved sources and recency.
Another crazy thing, I'm starting to use Microsoft Edge because the feature set and performance is actually really good! I've even convinced other devs at work to use edge, and we're all mac users. The read aloud feature has changed how I consume information completely - mostly because microsoft has text to speech voices that I can actually stand. It's an absolute game changer.
I looked into this, but you have to sign up for an invite, and they first demand you give them a bunch of information. I bailed at the second question. "what do you want in a search engine". How about not making me answer questions to use it?
I give them the benefit of the doubt that they are trying to create a good product and are truly interested in understanding user needs. afaik they are planning on making the search engine a paid product. Which I'm okay with. I seriously plan on paying for it if it's a good price. For some searches, I genuinely feel like I have a leg up among my peers because I'm able to find higher quality relevant information much faster using kagi.
How about uBlock Origin, but for search? A crowd-sourced + manually vetted ban list at the domain level. Google is too timid and can't just permaban low quality SEO spam domains for fear of getting sued, instead they can only tweak the algorithm. But a community solution has no such limits.
Copying Stack Overflow questions and answers to your site? Entire domain is banned, no appeal process, nobody using "uBlock Origin for Search" will ever see your site again. Boom, done.
Maybe it could be done as an extension? It puts a little ban button next to a search result on google.com, that instantly bans the domain locally for you, and also nominates the site for the global ban list.
I feel like there's a lot of low hanging fruit here, like completely banning just the top 1000 SEO sites would already dramatically improve the results.
- When they became ad obsessed where entire search results are filled with ads or YouTube recommendations
- Developed recency bias, aka if it's new it must be better, which has incentivized frequent content regurgitation rather than originality
- They became Woke, disappearing results that apparently aren't passing the Woke Filter and/or that don't align with political ideology/objectives
As for reddit, does anyone really trust reddit? Reddit is absolutely inundated with crap, spam, blatant marketing campaigns, political bias, and all sorts of screwy stuff. It's also highly manipulated, and no more trustworthy than Yelp, slightly more than Yahoo Answers.
Search is ripe for disruption but it's a tall order, and user/marketer generated crap like reddit certainly isn't it.
As a developer who use google (for convenience) the research results are getting worse. More and more aggregator sites appear in the search result without any advantage and google give a sh.t of it.
You need extra extension to block these sites from the search result.
Time to move on and use something else, but not google anymore.
Indeed, I came to the conclusion to search reddit for answers some time ago without any input by anyone (other than being a reddit user already). I want a frank discussion of the pros and cons of certain choices, not some obvious click-hungry promotional article with a bland rosy opinion.
- For that query, Bing's image results are much better, but the #1 site hit is still the exact same SEO-manipulated auto-generated e-ecommerce page, not any reputable reference source like we might expect. And that is a basic query.
- I tried the query on Reddit, the results are a disorganized jumble.
- So, the surprise winner on that query is... Bing. Or "none of the above". Back to atlases and encyclopedias.
There are a lot of good points here about why power users (i.e. HN types, technologists, scholars/researchers, etc) find Google Search frustrating, but it doesn't really provide a balanced perspective which would acknowledge that Google Search for the average, billions-scale user is an incredibly optimized, positive experience. For those users, Google Search is doing exactly what they want: providing instant answers to trillions of queries without making the user click or read anywhere else while making Google an absurd amount of money through ads. I'm not being facetious when I say that if you find Google Search frustrating, then you are no longer the target user of Google Search.
I've noticed that Google Search also provides too much weight on recently added/updated content than actual valuable content. A great example, while anecdotal, is Paul Graham noting that searching his quote on Google--`"Prestige is just fossilized inspiration."`--the first result is typically a third-party blog that is quoting him, not his own website where the quote was originally published. (Though he refuses to add SSL, which is why Google may be dinging his site.)
I, too, find Google Search frustrating for a lot of technical topics. The content ripping spam is overwhelming, even with technical topics. The past couple years, the proliferation of scraping sites that rip information from GitHub Issues/PRs/Discussions and StackOverflow information makes me incredibly angry and frustrated, and that is directly Google's fault for not identifying that spam and removing it. There is also nothing we as consumers can do because of Google's near monopoly on Search. We can switch to competitors, but it doesn't hurt Google's bottom line.
I have absolutely done the `${search query} "reddit"` 'hack' to find reading for my more niche queries--technical or non-technical. Reddit is a wealth of user-generated information, but it is typically a densely written answer and requires a user to comprehend that information. It can be easy to forget that the average reading level of a US adult is middle school level. That average user with a low reading level isn't going to spend their time trying to read paragraphs of text in order to both discover and understand an answer.
tl;dr Google Search is only dying for "us," not for the more profitable "everyone."
This is exactly right. Search for "Seven" on Google and duck duck go. For the average user Google hits it out of the park. 100% useful info with FAQs, where to watch and trailer. DDG is a mess and less than 20% of info is useful.
I agree with this sentiment. You can meaningfully append 'reddit' to only a fraction of actual searches.
But we should also note that HN users are the "spearhead" of adoption curve and if there is ever any meaningful alternative to Google, just by the virtue of HN users adopting it, could mean strong propagation in their social circle - the less tech savvy family members, friends and work.
Yes, power users (HN users) are early adopters, but I disagree that it would result in any meaningful change that requires social pressure. Fundamentally, power users and average users view search differently. Power users want "results," average users wants "answers." Power users accept a fact-of-life burden to skim through results and find the right resource that will help them with their query. Average users view skimming results as a waste of time and want immediate information. Results require more effort, answers are immediate and consumable.
You're essentially proposing that less tech savvy users switch to something that requires more effort from them. Even if the results are 10x better--hell! they could be hand picked--it won't convince the average user to take a path of more resistance.
That said, I would love to see a competent competitor enter the marketplace--there already are a few. But I have a feeling we'll be heading back to a system of more niche/focused search engines in the future.
> You're essentially proposing that less tech savvy users switch to something that requires more effort from them.
Not really. When I say meaningful alternative to Google, I mean something that is same or less effort for an average user, not more.
Ideally a perfect search engine is simple and easy to use in the default mode, but can uncover an advanced mode with a few clicks (for example ability to ban sites in results, just to give an example. This is currently lacking in Google and a solid source of advanced users' frustration).
I wonder if spammers have somehow gamed the Google political controversy filter.
I stopped using Google because their search results returned only things in agreement with mainstream talking points for anything remotely controversial.
If you search for "What countries are using ivermectin" on Google, you get the second link being a broken spam site (the kitchen sisters) and pages of results saying Ivermectin doesn't work. I wonder if the broken spam site figured something out to get ranked that high.
If you use duckduckgo or Yandex you get a whole page of relevant results that actually answer the question. The number of topics where Google refuses to return relevant results and instead focuses on talking points is very large at this point.
From a Digital Marketer who has worked with search engines since 2007, and can remember using Google as a primary student in 1998/1999, Google has made some incredible leaps....
Their search results however are getting worse, whilst also being exploitative. I'm a big believer in an open, free internet, but ripping off content to display on the home page isn't justifiable except for a monopoly, wanting to consolidate as many impressions onto their ad laden serps.
Google Flights imo was the start of Google's greed in unfairly competing against content platforms & aggregators. Pretty big conflict of interest there.
They're really stuck with Google Ads and dependent on it for a lot of their revenue. This will only get worse.
It used to be enough to append the word "reddit" but now Google tends to ignore it! It learned to route around people's desire to get useful results and learned to ignore it and show the garbage links instead. You can still get it with "site:reddit.com" though. I wonder when they will remove this option. Afaik a lot of search operators are already undocumented. And they removed the "+" for forcing inclusion of a term, so that only quotes worked as intended, but then also removed full support for quotes and it's now just a hint. Probably every step boosts some engineer's or manager's short term metrics and evaluation reports so it keeps happening.
Have you ever utterly failed to extract information from google and just given up entirey? This happened to me most recently when I tried to look up what model snowboards are used by certain athletes in the winter olympics.
You cannot bring up a relevant result. The minute you add the athletes name and snowboard to the query, no matter the surrounding terms, it just brings up the media dump of articles about the snowboarding event, not the equipment.
I ended up giving up, I couldn’t believe I couldn’t find anything relevant no matter how hard I racked my brain coming up with different terms for my query. What a frustrating experience when the tool you’ve relied on for 20 years has stopped proving itself to be reliable.
Oh all the time. There are a massive amount of searches that I'd like to do, and I KNOW there are useful hits out there for, but I don't bother because the odds of actually getting useful results is so low.
I don't know this, but would wager that a lot of people have scaled back what they search for on Google either consciously or subconsciously. The amount of topics you can expect quality results for has shrunk an awful lot, especially for subjects that are technical and/or non-current.
The Dead Internet Theory reminds me of a minor bit of flavor from a Neal Stephenson book, Anathem. In it it's mentioned that this far future civilization (which has seen civilization broadly collapse from technology zeniths a couple of times) had to abandon their set of planet wide communication networks because people intentionally set up computers that would put out huge amounts of information, but all of it was information to mislead, manipulate, obscure, deceive, or convince you to pay money for something. This was done so much that effectively all information, and all actors which might share information, became adversarial on the broad internet, so it stopped being usable.
Can somebody please recreate Google circa 2005-2012?
I'm pretty happy with the other search engines, but I do miss having a google profile that would feed me the correct kinds of search results. I refuse to believe that nobody knows how to do this (I don't) as Google was doing their indexing with commodity hardware on bread racks in the beginning. There have been scores and scores of swe in and out of that company.
I know that web crawling is hard, but we could use a few more options.
Is it inevitable that spam SEO and even legitimate applications like quora, stackoverflow, will dominate every search result?
Is it because of the "Deep Web" of content and information locked behind commercial, login required, and Web2.0 UIs?
>most of the web has become too inauthentic to trust
I hadn't really noticed that my own search habits had slowly changed until this article. Appending "reddit" is now a fairly regular habit for me, for exactly the trust issue mentioned.
Yeah, reddit also has issues with bots. But it's generally MUCH easier on reddit to gauge the quality of the content you're seeing than some top result on a google search.
When looking up "the best <product>" on Google, the results are utterly useless. It's always some site with a financial incentive to buy their particular product. Increasingly, that's low quality Chinese clones of products (and you'll find the same effect is true on amazon).
At least with reddit you can click on a user's post history and spot if something is obviously suspicious. On more popular threads you also get way more signal about whether something is sketchy.
Having come of age professionally in the Mozilla era, everything the Internet has become since then is the worst case scenario we all saw coming.
Once commercialization of personal information and site tracking became a norm, it has been downhill since then. I don't begrudge making money from the internet. It's the commoditization of user data that has corrupted everything.
However, it is encouraging to see the responses here. Perhaps the tide changes to something more equitable in terms of network value. Hopefully, something that rewards people and companies for their contributions to the network (content, inventions, knowledge, etc.) rather than mining their clicks.
I would love to have search engine similar to Google (I search in Polish and English, duckduckgo is no good for this, I tried) with ability to have favourite pages. If there are result on any of this pages from my search always show them in top 3. It should not matter how old are this favourite pages are or when last time they were updated or if they have low amount of reference links to them. They are my favourite so show me results from them on top.
And they should still show in my results even if Google or any search engine delist them for some boggus reasons from default results (they are my favourite so I veted they are good for me).
Google is dying but appending reddit to searches is not the solution.
Product recommendations on reddit usually boil down to a couple of products for each type, the hivemind keeps recommending them and the process kind of self sustains without any chance for other valuable products to be even considered/reviewed/recommended or pass the upvote threshold to be noticed.
Technical questions sometimes have an answer much more times get you to a dead thread that didn't lead anywhere because the attention span on reddit is way too short.
Also reddit users are mostly US based, local communities aren't usually big enough to lead to something useful on localized searches.
Reddit search can be hit or miss, but overall the content is higher quality than the numerous garbage quality articles that you will get for searching a broad term. You'll also often have access to a broad community and maybe wiki if you search a popular enough topic. Reddit really shines on niche topics though, like what capacitor do I need to replace in my 1996 CRT monitor. Often if there is any information to be had you'll get the best explanation through reddit, of course this doesn't apply for all things but it's good enough that I'm guilty of adding reddit to my google search terms more and more.
"Google's results are clearly getting worse". Can somebody quantify this in some objective way?
As in: I have this concrete metric (that anybody can inspect / replicate) and I saw it declining from 201X to 2022 etc.
I don't dispute that it is a true fact. The comments reveal both ways that this manifests, inventive workarounds and possible causes. But without having read through the 765 comments(!) (at time of posting) I don't see something that can be quoted as a measured reality.
NB: It would be really useful to have such an independent quality index, also for future reference when invariably somebody provides a "better" search engine.
This post is on point, but the bottom section highlights a nuance that may mean Google Search is not in fact dying: Google remains the best way to search Reddit, by a longshot. As someone who searches Reddit multiple times daily, I have tried a number of Reddit clients and always find myself falling back to Google.
Perhaps this is where the entry point opportunity is...build a search engine for power users that effectively filters results to "authentic" content from reputable UGC platforms.
As an aside: the advent of GPT-3 is going to make it really hard for reddit mods to keep doing as wonderful of a job as they do today.
There was a time I could find everything in google. Now results I would've gotten easily 10 years ago no longer appear, even worse, I get 0 results quite often, whereas even obscure keywords, number patterns or hex patterns would easily yield a blog or two about a specific thing, now not so much.
Even searching for a particular blog, having forgotten it's name, I tried every single keyword and couldn't find it.
I also find it funny that I am doing exactly what the author of the blog post argues about. Every single time I look up something about trading, ADHD or disabilities, I append reddit or even prepend it.
Anecdotally, I got some scam crypto ad called "Bitcoin Era" in my GMail app just last weekend. The whole thing is laughably scammy. I Googled the name out of curiosity, and surprise! Almost every single result in the first three pages is a fake review encouraging people to "invest" in this BS. Only 1 or 2 results are actual reviewers who fight such scams and point out how stupid the whole scheme is, with comments from actual people who unfortunately got scammed. The whole experience seems to corroborate the points made in the article quite well.
I think Google should just remove all pages with affiliate links from its index (which of course includes detecting all the ways to defeat that like URL shorteners, redirect pages, JavaScript hackery, giving a different version to the crawler, etc.)
Every time you search for "best X" you'll find a page with low-effort copied or write-for-hire content design to get you to click on Amazon affiliate links as opposed to what you are looking for, which is an actual review by someone who is an X enthusiast, personally bought and tested all the options and is eager to share their findings.
Google is dying because people are searching Reddit using Google and the data generated to conclude Google is dying is drawn from Google search volume stats for labels and keywords available on Google and trends...ok.
Am I the only one that has trouble with language preferences being ignored by Google? Accept-Language HTTP headers seem to be completely ignored, but even Google account language settings are ignored sometimes. IP address seems to be more important to them.
When I connect from a VPN exit point in Brazil it only shows me results in Portuguese (even when I'm logged in). When I connect from my hometown in Chile it's mostly fine but I think it's still not the same as if I connected from a US exit point, even though I have US English as my preferred language everywhere.
I'm annoyed each time I use google because it shows me an unreadable search page in arabic (because I live in an arabic speaking country). Fortunately I've been using ddg for years and I rarely use google
It's also categorically broken in somewhat basic ways recently.
1. I searched a term, and there were not many exact results for it, with the suggestion to try verbatim search - clicked it, quotes where added, and then the _same suggestion_ appeared with added quotes. I kept clicking until I got a few hundred quotes in a row and google thought I was a bot.
2. Just today I searched for a camera related term, any many results appeared from one website with the suggestion to search for more results only from the site. For some reason, that search returned only a single result.
I agree with the article points definitely, also quite surprised that this issue hasn't raised by google itself or the money outweighs the product usefulness in this area? Non english searches are even worse, usually some huge companies create their landing pages which get higher SEO/paid keyword scores then the actual useful pages.
Doing the reddit trick also for the reviews, but at some point it would also get broken as some marketing people will ruin it by buying reviews etc. Authentic reviews on products/services looks like unsolved problem :) (startup idea).
I've been doing the `site:reddit {my search query}` for years, and it's been great to be able to find authentic opinions.
In case anyone else does this and is tired of typing `site:reddit` all the time, checkout the Mycroft project for search engine plugins. I use one in particular[1] and alias it to `.r` in Firefox.
I don't agree with the thesis. People are doing that because they trust Reddit to be a source of relatively authentic opinions, yes. But you're not going to search Reddit to find official Web sites, established news sources, things for sale, and the like.
The complaints given in the article feel a little bit like observing that and steering it in the direction of the same old complaints about Google not being good for technical search, something that just doesn't matter to most users. I'm not going to Reddit to get answers to those questions either.
I remember being on a call with the Bebo people like 2 days after that absurd buyout and asking sarcastically: “So what color is the Ferrari?”
The answer: “Yellow”.
Now you have the Battery.
Reddit is so friggin user-hostile that I don’t read most of the comments anymore: because I literally can’t. You can do a little browsing but the minute you’re trying to pay actual attention you get slammed with the dark patterns about installing the app or linking your gmail or both.
I hope it’s the management team that gets weeded out of the gene pool and not the whole site, because it was fucking cool at one time and could be again.
I respectfully disagree. I love how culty and weird subs get, it’s always a forcing function to me on living in a complete stranger’s worldview. It stretches my brain.
Should ‘/r/all’ tilt as woke as it does? Eh, dunno. But that was/is the beauty of Reddit: you can dine at the buffet of subs as you choose.
One thing that's worrying me that isn't covered here: a lot of this low-quality SEO content is constantly regurgitated to produce... low-quality SEO content. Content farmers use content from Google results to write content for Google results. Google is getting devoured by loops like this.
I'm not a native English speaker, so at one point I was trying to find an authoritative source for an old idiom... and the entire first page were all different websites regurgitating the same inaccurate text! They did no independent verification of their own.
I wish wish wish Neeva was already good. I dig that they have a better aligned business model but their search quality still sucks. I was looking for a specific gun to buy and Neeva took me to a scam site as the first hit and I stupidly trusted it and completed the purchase through it (luckily was able to cancel it and get my money back). They also pushed an update which crashed their Beta iOS app and never fixed it. I’m rooting for them but I was not a happy customer so they will need to earn back my trust for me to subscribe.
I posted around 3 years ago how Google search results had become extremely unreliable. Searching for thugs like “Reddit best hand mixer” and setting the date filter to be for example “last year” would give me results from 8 years ago. This wasn’t exclusive to Reddit. Plus this used to work perfectly fine around 4 years ago. I remember when it stopped working.
Also programming related searches have now started giving me results of random shady websites which are copying results from stackoverflow and Google puts them at the top for some reason.
My own anecdote. Even though I'm not actively using reddit myself and mostly I just read-only participants on few subs that are interested to myself.
Anyway when I need to decide what to buy, or find an advice on something about certain country I'm gonna visit for the first time, or just want to learn near-scientific knowledge I do append said "site:reddit.com" more often than not.
I literally only use Google itself for some programming documentation or to find a picture of something. In all other cases it's results are awfully bad.
I don't want to jump on a bandwagon but I've been somehow reluctant to use google more and more. It's just a tiny feeling but it's telling. ddg.. even bing.. something snapped.
This problem will become more acute as all-day wearable AR devices become mainstream and the state of the world is recorded and parsed by a distributed network of them. You’ll be able to check the quasi-real-time and historical states of a particular restaurant, pull up the exact menu that was being served yesterday, as well as reviews. None of this will be indexable by Google. However, Google is particularly well positioned to leverage their technology and resources to be a leading player in the market for such devices.
It is only a matter of time until "Search term + Reddit" leads you to a thread with multiple, legitimate looking comments with varying degrees of upvotes and downvotes, and the entire thread has been ran by a marketing firm/search engine hit squad.
I too am guilty of trusting the Reddit concencus when searching, and if there were a few legitimate looking threads that had been planted I probably would have eaten them up.
Sure you get the occassional comment or link share, but I'm talking like 300+ comment thread carefully executed.
>Google increasingly does not give you the results for what you typed in. It tries to be “smart” and figure out what you “really meant”, in addition to personalizing things for you. If you really meant exactly what you typed, then all bets are off.
In simpler words: it behaves like an assumer, and as an assumer it's prone to vomit stupid shit. Should we start treating it as an assumer then?
A shame that the "Google + “site:reddit.com”" lazy hack doesn't work for me. I care about accuracy, not authenticity.
The article writer didnt' look carefully at the graph, which says in fine print at the bottom, "Y-axes are not comparable, charts show when* each had its own peak search interest. Data Source: Google Trends"
So, it is a mistake to assume that the quantity of the searches can be shared between the different graphs. Unless there is another data source that shows Reddit has the most searches, this is not meaningful. Actually, the graph is a Google graph, so everything on the graph is from a Google Search.
I actually use the term "forum" as an additive search term. It usually goes to domain specific forums which actually contain what you look for.
I also tried googling an answer to (an apparently common bug) in Windows relating to Bluetooth connections and I have not found any non generic answers anywhere in the search results no patter what I quoted or whichever term I added. Just generic crap, that same crap copy and pasted over and over and over again and non-specific bullshit answers from Microsoft itself.
It's interesting that on the Technology subreddit, the majority opinion seems to be that Google Search works well.
I'm not sure how anyone can come to that conclusion unless they were very new to it - people must b looking for something that's very different than what I look for.
Just wait until the people doing SEO now realize this and start astroturfing at the same scale on Reddit. It'll be ruined much faster and with less hope of getting fixed than Google.
Despite the assurances from the Google employee (Danny Sullivan) at the end of the article, I have not found quoting a phrase to work reliably, especially if trying to exclude the phrase. But what I have found to work is typing the quoted phrase in twice in the same search query. Weird, but so far, effective. And no, I don't remember what the searches were for. Wish I did. Not for a second am I accusing Danny Sullivan of dishonesty. I take him at his word. Quirks abound on the net.
The circularity of this meme is funny to me, even if the article is fundamentally true: This article is very very popular on hackernews, indicating that people agree that google search results are bad. What sources does the author draw from? He cites opinions from, among other places, Hackernews and Paul Graham.
"Find an opinion popular on hackernews, restate it in a blog post, refer to previous discussion on hackernews as evidence" may be a lucrative strategy for accruing internet points!
Oh man I'm glad I'm not the only one who adds "Reddit" to every search. If I want info about computer parts, software, games, cooking, fitness, etc. Then I don't think there's anywhere better at the moment. At least there's nothing better that Google serves up.
I'm fed up of Google returning blog spam, ads, and shamelessly rehosted content. I want real information by real people, not automated blog posts with titles covering every common search term.
I recently found myself looking for info on gardening potatoes. Each time I used google, I'd get the equivalent of "Top 10 things to plant this spring". It's AI generated drivel, void of any substance. Like reading book's table of contents, nothing else.
I added "site:reddit.com" and I had my answer on the first hit. This is sad.
Ten years ago I was following many search query by Facebook (looking for an event, a place, a business ecc). That was not implied that google was going to die but:
1 - I know before searching which content source was the best to answer my quest.
2 - Google was (and it still now) the best engine into insert my contestualized query and reach quikly the right content. Far better than the internal website of the content source (facebook yesterday, reddit today)
I’ve been doing this for a few years already. Not for all my searches, sometimes I append other domains, but generally I now tell google what domain I’m interested in, and Reddit is a popular one.
Unfortunately, only a matter of time until Reddit is gamed to hell unless they take steps to prevent moderator corruption (which is already happening and severe for many popular subreddits). And so the cycle continues. Avoiding people who want to sell you stuff is a sisyphean task…
I remember when searching for vacuum cleaner recently. Google 1st page is 100% SEOers gaming search to earn money on affiliate links. On reddit you can find comparisons like this: https://www.reddit.com/r/VacuumCleaners/wiki/recommendedvacu... . That's a clear example of what article is talking about.
That’s why I think reddit, facebook, stack overflow, and others will replace Google at some point. It makes more sense to search through user discussions than a register
Yesterday I wanted to find something on reddit (brooklyn vs chicago), in last 3 years. Despite setting search result to last 3 years google kept showing reddit posts from 8 years ago. Tried bing and still it sucked, then tried ddg and finally got something relevant, not as good as how google used to be but it pisses me off each time I search something up on Google. Google has become better at local search and deteriorating on global web search.
Danny Sullivan, the Search Engine Watch and Search Engine Land founder, now works for Google. He claimed on Twitter that exact search with quotes works as expected. Replies from users suggested he was wrong. I can remember the day Google announced they were removing/downgrading support for the quote operators. I remember because there was protest on HN about it. I cannot recall any announcement they were adding this functionality back.
This reminds me of Microsoft hiring Mark Russinovich, the Sysinternals founder. Big Tech companies neutralise informed critics by hiring them. The former critics defending become defenders of the company as it now seemingly represents their own work.
Yea. Reddit is a super useful content island. The idea that folks prefer certain such content island was one of the many reasons we started you.com (other than privacy, time saving, developer focus, etc)
We are the only search engine that allows you to set that Reddit preference once and then whenever relevant - the reddit search app will come up. Same goes for Stackoverflow and other apps.
This is interesting, as someone with an e-commerce site selling pretty niche ev conversion parts(www.bratindustries.net), I’ve kinda ignored seo optimization….
Pretty much all of my customers come from the isolated communities I'm active in.
this is enforcing that fact that it’s more worth my time to be active in more communities, rather than push for ads and seo.
Resulting more information rich communities. so is this just pushing for information silos or adding more?
Google started occasionally showing me Quora results where I would have expected Stack Overflow ones instead. (There is absolutely no way I'm touching Quora for anything technical. Or anything, for that matter. At most I might use a Quora post to find links to something actually useful).
So now I'm using Stack Overflow search directly for programming related questions, which has eliminated 75% of my search needs in general.
Wow, the response from the Google engineer really highlights that they're blind to the problem.
It's only a matter of time before a new Search engine comes along and takes over the lead.
I resort to Bing and Yandex more than Google these days due to all the reasons mentioned in the article. It's not just search though, Im consistently finding Bing Maps has newer satellite images and streets that Google doesnt know about ( thanks to OSM )
I won't name the specific search engine I am using as a default, as I'd like to prevent it being manipulated for as long as possible, but I will say this:
Google is my second to third search engine choice at this point - never my default. Google search has, in effect, become the 2nd or 3rd page of Google Search results; you only resort to it when you are truly desperate, and have very little hope of it doing any good.
There is a mode that google has which is basically the old mode; since I discovered it recently out of huge frustration with google search results.
After you search for something; select `Tools -> All Results -> Verbatim`
This will get google to actually search the way power users expect. I am surprised how little known this feature is. It should be default, but once known it completely removed my frustration with google search.
This is true, but I think the main concern is that Google appears to be asleep at the wheel.
Will Google combat the low quality internet? If not, search will continue to degrade, so what alternatives exist? That's the thrust of this surprisingly vehement discussion.
The idea of the whole-web search engine is dead. There's too much junk, and too much incentive to surface it in the name of engagement.
I got so sick of Google's useless results that I started out on a fool's errand. I'm using publicly available, curated or moderated link sources to build my own STEM focused search engine. It'll probably end in tears, but I intend to give it a shot.
Google "died for me" not when I first switched to DuckDuckGo but only after periodically switching back to Google to check "if I was missing anything" and finding only ads, irrelevant knowledge boxes, and garbage organic results.
The only thing Google still does better for me is provide "Stack Overflow" results.
DDG/Bing might not be perfect but it works for 90% of my web searches.
Google has integrated pretty much every useful tool that they can to keep users and create a monetized user profile. Between DNS and Google Fi, they have all of the data that they need. I've completely de-Googled my technology stack, with Neeva taking care of the search piece. Everything else is Nextcloud, NextDNS, and others. All of my de-Google substitutes are at de-google.xyz.
Reddit has peaked, at the time PG wrote that tweet you could see some volatility with still some isolated peaks, but if you look at it now it's more clear that the general trend is downwards https://trends.google.com/trends/explore?date=all&q=reddit
Google could restore relevance to their search results by severely demoting or eliminating pages that contain any advertising from an ad network, especially adsense. Of course, they’re not going to do that, because relevance of results is low down on their list of incentives. The article is correct in pointing out that Google’s founders predicted Google’s demise with pinpoint accuracy.
I think that Google Maps and GMail are also "dying". Google Maps has gotten very bad at picking an optimal route in the past year. GMail spam filtering is becoming worse. I think that Google and perhaps even Alphabet are putting resources into markets that they think will reap benefits in the future, and letting the areas where they have a dominant position run on autopilot.
I feel like something happened in the past few days that made Google significantly more infuriating to use. I switched my default search engine to DDG after the nth case of Google presenting search results that matched zero of my (fairly mundane) search terms.
The DDG results aren't superb, but they also don't invoke the feelings of communicating with a distracted child or poorly-trained pet.
> Large proportions of the supposedly human-produced content on the internet are actually generated by artificial intelligence networks in conjunction with paid secret media influencers in order to manufacture consumers for an increasing range of newly-normalised cultural products.
The total fog of war surrounding product reviews is a big reason I enjoy shopping for older, used things online. Nobody is being paid now to shill for 10 year old Dell servers, or a 40 year old analog oscilloscope, or the quality of 1997 made-in-Japan Stratocasters. There are some YouTubers or subreddits who I think I can trust, but the incentive is there for dishonesty.
Some time ago, I overheard a conversation between my kids. The younger one wanted to find out about something on the internet (don't recall/missed what) and told the older one "let's ask Google" .. to which the older one (early teen) replied "Google is useless. It will only show ads and random links. Let's search Wikipedia."
> [Reddit] can’t be bothered to build a decent search interface, so instead we resort to using Google, and appending the word “reddit” to the end of our queries
That gave me a good chuckle, it's a daily habbit for me.
Seriously though: the search used to be even worse. I remember when they re-implemented it and made a bit thing about it. Wasn't it in collaboration with some third party?
I had this exact conversation with someone yesterday, who wasn't aware of Google's "site:**" query functionality and expressed the same frustrations on not being able to find actually relevant content. It's simultaneously reassuring and disappointing that many others also rely on site-filtered Google to find information on the internet.
Site-filtered Google is basically the only way I search the web now. As many others have expressed on here, Google used to be such a good "gateway" to the informational web. Now the most relevant results are almost always auto-generated content.
I resort to searching specific sites like HN and Reddit as a safe place to get human content, but I feel this to be limiting in it's own way, almost like echo chambers. Are we past the Wild West days of the internet? It now feels like a dystopian reality where I'm constrained to certain pockets that seem relatively safe.
I believe Google used to allow a "discussions" filter on queries, which would limit your search to forums. I'm not sure why the functionality stopped being supported. The "Dead Internet Theory" is very real. Given the amount of bots and resulting distrust in information, there's an urgent need for some sort of conversational search.
A forum-only search filter is an easy place to start. This could also potentially be a good use case for some decentralized, blockchain-based trust network. If anyone knows of any ongoing projects in this arena, I'd be very interested in contributing.
On one hand I totally agree - Google is becoming unusable for any refined specific searches, if you use any SEO-enhanced keyword you mostly get nonsense.
But on the other hand I am thinking - Google is not stupid, they know what they doing. Maybe this kind of search is a good fit for the majority of the less-tech-savvy people, and only audience here on HN think it's bad.
Not for work though, if I search "3 way solenoid valve" or "food safe stainless steel" I get good results. Sure, I have to scroll past a few ads but the cost of running the thing doesn't come out of my pocket.
For other stuff, yeah, Google is in pretty sad shape. I remember how exciting Google was when it was first created, those days are long gone.
The algorithms haven't been over those specific subjects enough to "refine" them to modern "standards" and it's only a matter of time before enough people search 3 way solenoid valve to get it connected to the forever growing mountain of useless search terms. It's actively decaying.
oh man, on point. I literally just did a day of picking a new gas range and finally the best results were Reddit. Trying to search for information on through google and general sites was so infuriating. It sucks now.
I would seriously pay $10 a month for a search engine that worked really well and wasn't in the ad game. But I guess that's not a common stance.
I've certainly learned to appreciate higher-quality 'review' content more than I did in the past, and to be more willing to actually pay for it (e.g., Consumer Reports, Wirecutter).
Although I like the idea of DDG's bang operators, I rarely use them (mainly !g when I'm feeling desperate).
What I would find useful is to be able to whitelist a bunch of sites on DDG, so that it prioritises results from them first, when I search.. basically most of the sites with ! operators I guess.
That way I wouldn't get all the SO clone-sites returning their rubbish.
Alternative interpretation: Google so useful that it instantly searches whatever sub-corpus you desire if you simply mention it in your question.
Google says it shows zero ads on 80% of searches. So the whole "ads now take up entire screen" thing is based on the qualitative ramblings of twitter accounts who don't know what they are talking about.
I think part of this is the inverse of the base rate fallacy. As people's use of search has gone up, the absolute number of bad experience (many ads, or poor results) has gone up regardless of whether the actual quality had gone up or not. Combine that with the elevated expectation and the confirmation bias, some people's perception of search quality will get worse and the number of people with such opinion will increase.
100% agree. Google search is only useful for searching other sites that don’t have good search. Online communities have better results. If there was a search engine that curated results from various groups across social platforms, I think that would be useful. Especially for technical information or anything else with a small group around it.
I have never appended reddit in my queries. I don’t find majority of Reddit credible or complete. The author here is extraordinarily hyperbolic. Reddit is not “next” search engine by any possible stretch of imagination. I would think search driven by very large transformer based models is probably the next thing but it’s 5 to 10 years away.
> Even the exact match query operator (“ ”) doesn’t give exact matches anymore
I wonder if this isn't because most people don't think of quotes as being the "exact match" operator, and so expect fuzzy matches. The former exact match operator (plus) didn't have that issue, and was a better match for the exclude operator (minus).
I tried switching to DuckDuckGo years ago, but found the result quality just didn’t match google - it wasn’t getting me to what I was after. Now I feel that way about google even more strongly, so perhaps I’ll give DuckDuckGo another go.
PS I also do the kind of searching in the article with hacker news, e.g. ‘JavaScript testing site:news.ycombinator.com’
Reddit search is superior because the results are community curated and because, counterintuitively, the search algorithm is terrible.
It's doesn't suffer _as much_ from the deluge of garbage on there dead internet, and the search is good enough to discover what you're looking for while remaining bad enough to provide compelling surprises.
Google promotes advertisers. Reddit promotes shills. Popular platforms will be bought.
The problem is trust vs the appeal of corruption—that is, some people will always want to deceive the masses for profit.
At scale, reliable human trust only exists in democratically-policed communities, where authentic users control corruptible owners—something few platforms want.
To be honest, contrary to the popular hate, Reddit is actually one of the more useful social network I've used.
It contains a lot of memes/junk but it also contains a wealth of people's knowledge.
Reddit should steal 1-3 top search engineers from Google and build out a much better search. And might as well steal a few ads engineers from Google too.
I miss the old school yahoo directory, in part because it seemed to be curated by real people like a library would be. Or web rings where there would be humans curating content. In a way, Reddit is a crowd sourced content curation site with human curated topics. No wonder I find almost everything I need there, and Wikipedia.
> Reddit is currently the most popular search engine. The only people who don’t know that are the team at Reddit
Hahaha. This has legitimately made me laugh.
The article is sadly quite on point. I'd add that Google is increasingly deteriorating for me during the last several months. It was actually still little better than now, a year ago.
Used to use reddit long ago but left due to the inherent (if subtle) censorship.
However, I've been using site:reddit.com in google searches for years after leaving reddit as a user, mostly when I want to find more realistic opinions about certain products or solutions and I want to filter out marketing. It's served me very well.
I gave up on Google search around 2012 or 2013 (I occasionally use some of their specialized engines such as Google Scholar, or use a Google search as a last resort). So this feels sort of like a post marveling that blogger and blogspot are no longer as popular in the USA as medium or substack, its true but not news.
Both Google and Reddit are afraid of each other. Google doesn't want to show reddit results by default as it doesn't give them any revenue and doesn't want reddit to get too big and reddit is afraid Google might build a reddit alternative by including more and more reddit features within the search.
I'm almost happy to see this for the wrong reasons. For the past few months I don't know how many times I've complained about the dire state of Gsearch.
For every single search, I have to consistently scroll one page down to skip ads and product matches ala "google shopping" belt. It's just insane.
Just an anecdote but I was shocked when I went home over the holiday and nearly everyone told me they use duckduckgo now. These are not tech people either, I am from a small rural town in upstate NY. I couldn't believe it and although it is a small sample size there must be serious problems with google.
I'd go further than the article and say that Amazon marketplace is decaying for the same reason. Instead of SEO webpages we have cheap knockoff junk from questionable oversees sellers. If I want to buy anything these days, I search for best ___ reddit and then look for that exact item on Amazon.
Because Reddit's native search is completely broken.
If I'm looking for something specific I've seen before I use redditsearch.io if I'm looking for a comment by a specific user I use redditcommentsearch.com if I'm looking for de novo results I use Google site search on Reddit.
I was skeptical at first, but it really does seem to work for most queries, especially queries about products. I tried standing desks, streaming setup stuff, keyboards, linux desktop configurations, it's all there, all mostly ad free, definitely SEO free.
Does this mean that "traditional" advertising is becoming more powerful?
If the users can't trust Google to return relevant results, would they simply trust brand power and go directly to the websites they trust? (e.g. go directly to nike.com instead of searching for "running shoes"?)
I was skeptical when I started reading, but then I started thinking about it, and 99% of the time I use Google (actually Startpage, I don't use Google directly), I already know what websites have the content I'm looking for. Those sites just have piss-poor search tools.
Love it, same old paul graham propaganda, of course Reddit hasn’t peaked even if the graph resembles the one right next to it :) I think pg peaked some 5 years ago.
But the main topic, has google search peaked. Yes it has. The amounts of ads vs great relevant search results peaked some time ago.
I find it amusing that even though I find more "organic" results on reddit, reddit's own search isn't great IMO, so I find myself often googling site:reddit.com inurl:<subreddit> followed by the search query I'd have preferred to enter by its own.
Honestly, I append Reddit, Stackoverflow, or Stackexchange to probably 75% of my searches.
From my point of view, there's wayyyy too many blog sites out there full of crap content, meanwhile forum posts on these sites often yield results that are something I can actually do/use.
Agreed. I rarely search for anything without "... site:reddit.com" or "site:github.com". the sheer number of sites that scrape github and then pop up above github in the search results is a clear example of this. why isnt provenance weighted more heavily?
It's perhaps a little bit early in its creation to be sharing this but I am working on a new search that should help to fix the problems mentioned in the article, https://namusearch.com/
Adding money to the mix is always problematic. Money destroyed sports. And money destroyed the web. The amateurs are still out there, but the content served by SERP's are mostly from professionals. Just like when you look at sports on TV there are mostly professionals.
Just throwing my weight behind my agreement and belief that Google has gone way downhill in the last two years to where if there was a good alternative (and no, DuckDuckGo is not a good alternative), I'd use it in a heartbeat. Google as a search engine sucks now.
While I do not disagree that google search has degenerated because of ads it still finds what I am looking for better than any other search engine. I just skip ads.
As for reddit - it is the last place I look for things.
The whole article reads as someone advancing the agenda without any real substance
It seems like every critique around Google is immediately trending on HN. I wonder how long it will take to see significant market shifts towards competitors. DDG recently surpassed 100M search queries a day and I'm curious how their growth will accelerate.
Not sure if Google’s fault only, but searching for programming related content is much, much worse than it used to be. It is extremely difficult to get Google to show deeply technical content, presumably because it falls outside of the majority of search terms.
I've been using Google Search since it was a cool beta site announced on Slashdot. Over the years I've built a career based on my ability to effectively search. I remember about ten years ago, people thought I had some kind of insane gift because I could immediately find ANYTHING. Not really, I just had an instinctive skill for creating effective queries.
Good search has been a huge part of my ability to develop software. I don't mean StackOverflow either. I learned to use Google to search Microsoft APIs and forums, as well as to dig up long obscure posts on the almost-dead languages and technologies I found myself supporting. Day by day, this is less and less possible. I'm losing a critical tool that has helped me be productive.
As an Autistic one of my strengths is feeling patterns in systems, and in the past few years, I've definitely noticed the garbage results described in this article.
Yesterday, my wife asked me if we have any cold meds. I said, "we have several, but let's look up interactions with your new antidepressant." I know from experience that all kinds of unpleasant side effects can arise from mixing these.
On her phone search results, there were none of the quality sites I expected, such as Drugs.com. Instead I had to crawl through a bunch of SEO garbage and psuedo-health to find what I needed. If she was doing this on her own, she might have clicked on something dangerously erroneous. The web is becoming increasingly hostile. (And don't get me started about the infinitely scrollable boomer ads that come up below a local news story)
Hacker News is also becoming a replacement for google for some things for me, I'll just come here and search on some tech related thing which I think is highly likely been discussed on here to see what others think / first hand experiences.
Maybe this is a good thing. Maybe it was inevitable that the maturing of the winners from web 2 would lead to its death and that is the stick that drives dev of the next stage. It's certainly where my interests have turned the past few years
Perhaps true for some things, but technology related Google is still irreplaceable.
Haven't found a better search engine for my problem solving. Yes, a lot of answers point to Stack Exchange but it's not the only place where you can solve issues.
How much is this a broader Google issue? Ads are encroaching more on Gmail, Calendar randomly drops meetings, and Maps is now noticeably worse than Apple Maps.
It definitely feels like they’re being run by financial spreadsheets now.
I don't see why Google does not trial offering a paid service where advertisements are stripped away from the search results. A new revenue model in addition to the existing ad supported approach where you pay for your search.
I think what kills google searches is seo bullshit. Even the worst sites with seo are now on the first page. In the past, SEO really forced quality, but now it's enough to make your website compatible with bullshit like amp.
We need a Google Search Engine Filter Engine. A site that frontends Google, does a quick peek at the first 10 results and if they are infested with higher than X percentage of Google Ads, exclude them from the results.
Ironic that the web is being eaten by a glorified Usenet, leaving then, as the main use case for the web, the sort of remote commerce that was once handled by Sears catalogs and food delivery phone numbers.
I'd just be happy with a search engine where you can choose to remove an entire site from the results. Sick of getting towardsdatascience pages written by students rehashing chapter 1 of textbooks.
Speaking of inauthentic shills, how much did this guy get paid by reddit? If there was really a time where reddit was a good source of authentic information, it’s many years in the past at this point.
Great timing! Just today I did a Google search in an attempt to figure out why my skin surrounding some recent scar tissue had a yellow discoloration. Didn't find my answer until the third page!
I hardly ever use reddit, don't browse it, find its hints to install the app annoying. Its been so off my radar, posts like this keep making me think I must be really missing something special.
I found that Google sometimes returns very few results even though search word is common one. I suspected that so they can reduce their server resource. I'd like to pay better search result.
For me it seems that Google search basically has leverage to either make more money or show more useful results to their customers and they have chosen the first option to make more money.
I do site:news.ycombinator.com append to my searches very often. Higher than average quality of information is simply an emergent feature of any successful platform with social moderation.
This resonated with me: "The results keep getting 'refined' so as to suit the popular 80% of queries, while getting much worse for any technical or obscure queries."
Summarizes almost exactly how I feel. I also searched the title of the article, and one of the autocomplete suggestions was "why are google search results so bad 2021 reddit"
re: the Dead Internet Theory, anyone who browses the "news" sections of any stock trading app, Yahoo Finance / iOS Stocks app can see that the likes of Barron's, Zacks Investment Research, Motley Fool, Benzinga, etc have been autogenerating "analysis reports" for some time, where some basic fundamentals and options metrics are repackaged in some filler wording. I don't think it'll take much for lots of secondary content to reach this state.
I could live with all the nonsense SEO hacking results and the ads if it worked. But today it is like it is misinterpreting everything you put into it like some bad comedy movie.
This is all so true. I append reddit to most of my google searches because I don't trust google anymore, and I don't do it in reddit only because their search sucks.
Indeed. I add reddit to find authentic results by real people. Amazon stars are fake, and similar with other sites. Hackernews is also a trusted resource for real opinions.
Take this idea seriously: This is the "final form" of free search, and we will see subscription services providing relevant and useful links in the future.
Google is dying because people are searching Reddit using Google and the data generated to conclude Google is dying is drawn from Google search stats and trends.
This makes sense as platforms like reddit or this HackerNews are basically search results + social media engagement features sorting qualified opinions
Google is still the number one driver of users to my business — by farrrr — but non Google sources (mostly DDG) — have more than quadrupled in the last year.
I'm sure this will get burried, but as someone that uses google search extensively for finding solutions to basic coding syntax questions, SEO has effectively poisoned googles well.
Where when i would post a simple question previously, I would almost always get a SE/SO answer that was 80-90% correct, now I only get a bunch of copy cat 'learn coding' web pages that really aren't ever the question I'm asking.
I use duckduckgo as a browser and the !SO bang is effectively broken due to cookies so I don't know what to do.
Not only the Google Search is dying, the hyper-text web in general is dying, popups, huge banners, inline ads, autoplay videos, cookie consent footers, login-walls, pay-walls, clickbaits & content farms, geo-blocks, bloated JS rendered templates, hard subtitle inside videos inside iframes, non copy-pastable texts only available on exclusive mobile apps, etc.
It's no longer the same WWW I am familiar with anymore. Reddit is just one of the few sites still had higher text condensity (old UI, to be exact)
I want to share some experiences from within the particular niche that is wildlife photography and how it relates to species information. I've been querying such information for over 15 years.
Search for a plant species. You'll have the hardest time finding anything about the plant itself, results are dominated by parties selling the plant. Quite often not even the plant you searched for.
Search for a species of insect. Results are dominated by products on how to kill it or sensationalist misinformed articles about their danger.
Search for a species + location/country. Results include all kinds of stuff about the species (which is already poor, see above) but also simply forgets about the location parameter.
The above is a massive problem, because it leads people to believe that a species occurs in a country, whilst it may not. They then incorrectly identify the species on their photo. This error is then infinitely copied by others as it surfaces in search results, which many see as authoritative.
This still pales in comparison to the problem that is Google image search. You search for a species name but the photos returned more often than not are not the actual species. It's a photo from the same page as which the species name occurred on.
Not to mention the Pinterest problem, which absolutely grinds my gears. They get all the free search traffic for stealing people's original works. How the fuck can Pinterest rank so highly, it has no content and is never the original source.
It's hard to put my finger on the timeline, but all of the above has gotten dramatically worse during the last 3-4 years. It was definitely not always this bad. It used to kind of work.
You know what Google should do? They should directly feed from authoritative sources rather than scraping crappy content from bad actors. They do it with Wikipedia and I see no reason why they can't do it for other niches where high quality information is publicly available.
For the above niche, it's entirely solvable. Academics have open databases with species information that is trustworthy, not gamed, and authoritative. How about using it?
we're launching a bird identification search engine this month via breezethat.com ahead of the spring bird migration if you'd like early access to test it for this very reason. We can add others after that for plants etc., say one / quarter as part of what we're dubbing our planet saver topic searches. dm here or on twitter as DotDotJames if want to test the bird identification finder
They did that right when they chose the advertising model. It was never going to work in the long run and the founders knew it. They just thought they could build an AI system before that happened and it turned out they were wrong. Useful AI that could distinguish real knowledge from SEO optimized spam was much further away than they thought/imagined.
yeah i realized i've been doing this for a while now. For anything that i'm googling that requires some sort of querying for personal input beyond that of wikipedia/stackoverflow, this is what i use.
I will say as an academic, google scholar is still superior. I just search with the !scholar bang on in DDG.
Help me out: I have such a hard time understanding this line of argument. What search result do people want when they look for a recipe? A site without ads? You can pay for that, there are lots of premium great sites. News without ads? Buy the economist. But people don’t want results full of pay walls.
I’m struggling to follow. Can anyone give an example of a query, and then the ideal result that Google is not delivering?
Google has integrated pretty much every useful tool that they can to keep users and create a monetized user profile. Between DNS and Google Fi, they have all of the data that they need. I've completely de-Googled my technology stack, with Neeva taking care of the search piece. Everything else is Nextcloud, NextDNS, and others. All of my de-Google sources are at de-google.xyz
I use site:medium.com when searching for technical intro tutorials.
This is to exclude official verbose and robotic docs like the ones from AWS.
AWS docs have a LOT of content and words but they feel numb and almost meaningless, they say so much yet you read for hours and have no clue of how to solve the problem. It feels like reading a dictionary on *hammer* when you want to learn to use a hammer.
> TLDR: Large proportions of the supposedly human-produced content on the internet are actually generated by artificial intelligence networks in conjunction with paid secret media influencers in order to manufacture consumers for an increasing range of newly-normalised cultural products.
This strikes me as one of those explanations that gets very close to the truth, but then sharply veers off into fictional territory, which also makes is then trivial for the article to handwave it away with;
> This isn’t true (yet), but it reflects some general sense that the authentic web is gone.
What's true is that too many Google results are just aggregator bots reposting content from the largest news organizations. There are no "artificial intelligence networks" involved for any of that, that would probably even be an improvement by adding a bit of flavor to the samey content.
But it's very much just copy&paste, to such a degree that it feels like there's only a hand-full of news-outlets in existence, and everybody else just copies their headlines and articles.
In practice this leads to quite the extreme mono-culture when looking up certain hot topics, as the first page will be dominated by the same few articles, with slightly different headlines.
i think once google made the decision to go from returning results you want to see to returning results you want to see ( minus what google and their political friends don't want you to see ) created the incentive to look for alternatives.
I've lost all faith in most search results. SO MUCH CONTENT is designed and tailored to rank and drive ad traffic, not to inform. This isn't the case with Reddit, since the capitalistic factor isn't there.
Yeah, there's a risk of misinformation from results from bigger subs, but posts from smaller subs almost always produce factual, high-quality content from actual people with no hidden agendas.
Example. I was looking for a french toast recipe some time ago. Searching "french toast recipe" on Google on mobile (in Houston, TX) yields four pinned results, one of which is from the Food Network (which might be okay, but they're also a huge content aggregator), amongst a bunch of recipes, each of which has a marathon of words before the actual recipe because ads gotta ad.
Instead, searching for "french toast recipe site:reddit.com" gives you a bunch of posts from people who asked the same question along with answers from several people, some of which contain links to recipes that didn't rank before (or are ONLY on Reddit).
We're broaching on misinformation from the HN community where people say something is true simply because they want it to be true. This article isn't adding anything, but it will do well on HN, because it agrees with the community.
When you start your post with "Reddit is currently the most popular search engine" you are already well outside the realm of fact.
I come from the 2040s. Don't come here. In the parlance of your time, it fucking blows.
Google is still around. It is a third-rate search engine but a first-rate reputation engine. Boomers (we still call them that, even though they haven't been actual Baby Boomers for a long time) still use it to vet people before making hiring decisions.
For $159 per month (everything is on a subscription model) you can get the "personalized reputation" treatment by Google, so that when said Boomers are deciding whether to hire you, they see the unreliable material you paid for them to see--rather than, as under the old system, the unreliable material that emerged organically. It's a steep fee (these are the deflationary "new dollars") but it's a small price to be able to get a job and not be picked up by one of the "sweepers" and put into one of the performance improvement camps.
I wish I had overthrown capitalism in the 2020s when it was still possible.
When you search for something with presently political punch. And then do the same search on a different search engine. And see the difference in the results. See how Google controls what you see, to control what you think.
I'm not going to vouch for Google's search results - we all know they are declining. Even so, if everyone is using Google to search reddit, that doesn't tell me that google is losing its search dominance. It tells me that even as people try to get away from it, they still use it as much as ever and by that usage are likely helping Google figure out which use cases they need to develop to improve their products.
Also, the idea that reddit will replace it seems unlikely to me. As much as there is decent content on reddit, it exists side-by-side with junk, jokes, trolling, memes, shills, and straight up misinformation. This doesn't stop people from using and enjoying reddit - much of the silliness is all in good fun, but it will become a serious barrier to trying to become the search engine for online content.
Google maps results are on their way to death. I was recently doing a search having gone down to a three block area, did not show the result - Google pulled out a few blocks and showed a category result there. There was a higher ranking category result in the block i remembered, i had to remove my search term and zoom in!
The problem is not ads, it is not even capitalism, is the requirement of our western capitalism to require constant growth. Doing what Google did 5 years ago, with the profits of 5 of years ago, should have been fine - but the markets demand growth, so companies have to pull into unsustainable territory and that wrecks the company.
Boeing is a great modern example.
No one ever really expected much of reddit. It could just do its thing. But now, spun off, it will have to relentlessly seek growth, and the counter is ticking for its destruction.
unfortunately Reddit is dying alongside Google Search. of course by all means I don't mean economically. but quality-wise - yes, especially since it was acquired.
odd how no one mentioned you.com
- They solve all the issues mentioned in the link and offer features mentioned in the comments like surface content from reddit first
I think SEO was the big mistake. As soon people understand what makes a result show up first in the search it became a almost meaningless metric. There should really only be one metric that counts. Relevency.
They sift thru that and give about what you want. I just tried it - 'modelling clay'. Google's 3rd page (after many paid ads) has what DDG has on top after a disambiguating a/b/c box to help you know what you're looking at.
Google search has been next to useless for me at least in the past few years. Results use to be on the spot, and now I get wildly various results that have nothing to do with what I looked for. The biggest issue is it trying to substitute words which renders my terms useless. Then there’s also the quora answers showing first and most answers there are being paywalls now.
A major problem with search degradation is that lots of content is behind walled gardens now: apps, instant messaging/chat and video platforms that aren't as indexable like social video platforms, YouTube is pretty good about metadata to index. More content is behind paywalls.
Less and less is being written in blogs, sites and publicly indexable content.
i think it's less the case that "Google is dying" and more the case that the open and decentralized internet is dying. all the good content is moving into miniature walled gardens, behind paywalls, behind authwalls and deep inside apps where you can't change the font size on your smartphone. increasingly all that's left out in the public are these SEO'd craptastic advertorials.
Upon seeing this in my threads page, I feel the need to elaborate. Since SEO stands for Search Engine Optimization, this is an example of RAS Syndrome - Redundant Acronym Syndrome Syndrome.
A few years ago (2015), curious about where meaningful conversation might be hiding out online, I did a little experiment, making use of Google Web Search as it happens.
The process involved finding a set of search terms which might be expected to appear in more substantive discussions, or at least, the sort of discussion I'd tend to be interested in, and then see how many such occurrences there were across various sites, domains, TLDs, and the like.
The result was "Tracking the Conversation: FP Global 100 Thinkers on the Web".
The title comes from the list of terms I'd used, the Foreign Policy Global 100 Thinkers list, contributed by readers of that magazine (and I suspect curated by editors). That is, it's generated by a third party, reflects a largely refined audience, reflects a range of political and ideological viewpoints, and are mostly reasonably distinctive.
I approximated total page hits on a site (in English at least) with a search for the word "this".
And to proxy for more mundane comment, I chose to search for the arbitrarily selected string "Kim Kardashian".
This of course gave rise to the now-world-famouse FP:KK ratio. That is, the ratio of hits for the FP 100 Global Thinkers vs. "Kim Kardashian" on a given web property.
Another metric was FP/1000, which is mentions of the FP 100 names per 1,000 web pages (based on the "this" search results).
I chose roughly 100 websites and/or domains to search. This meant performing 30,000 cumulative web searches, a practice Google apparently take a dim view of, though performing one query roughly every 45 seconds or so seemed to work at the time. (Google's anti-bot defences have since become far more rigorous.)
The results were interesting and occasionally surprising.
Facebook had by far the most detected pages, 2.6 million at the time. Again, this isn't a precise count but a relative proxy.
Wordpress had the 2nd most FP100 results, and a density 10x greater than Facebook. This was when I realised that Wordpress in fact ran the sites behind a great many other organisations and publications, many of which are fairly high quality.
Metafilter had by far the highest FP:KK ratio at 32.75. (Compare Facebook at 2.10, and Twitter at 0.96.)
Google+, supposedly where smart people tended to hang out, rated only an FP:KK of 0.39.
I also looked at a number of mainstream and alternative media sites (the New York Times scored abnormally high, but that was largely through having one of the FP100 members as a columnist, mentioned not only on his own articles but in many others, Paul Krugman). Fox News scored predictably low (and many instances referenced the then Pope), but still higher than the BBC and Reuters.
Alternative media tended to rate higher than mainstream, but often focusing on a relatively small number of liberal thinkers, Noam Chomsky standing out in particular, also Krugman and Lawrence Lessig.
In education, what struck me was how much more content results appeared for leading private universities (Harvard, MIT, Stanford) than flagship public schools, with UC Berkeley especially paltry page count, though a higher FP/1000 ratio. University of Michigan represents better. I included a few European universities as well, which had modest results.
I don't recall why I threw Federal Reserve domains into the search, but this was when I realised that St. Louis is effectively the research arm of the system.
And I threw in generic and cc TLDs for good measure.
As mentioned, the reseach as conducted would be virtually impossible today, though there are now several quantitative searchable archives which report on the number of results across hosts and/or domains for various terms. I'd really like to be able to make use of those.
In the context of the past few years, refining searches to terms of more recent interest and relevance to information quality would also be fascinating.
Google is still a high-quality search engine, it just has ridiculously poor product and consumer focus. This could easily be turned around by better product management.
Some search results are poor, but they can be very easily changed by Google. Recipes, for example, are only poor because of Google's own published recipe metadata HTML format. Their results prioritize matches that include that format, and 99.999% of matches are pages from a single WordPress plugin that uses the format.
I moved to DDG a few years back and don't miss Google. While it is possible that Google might have provided similar results to what DDG did for the same query, I have noticed that when DDG fails to provide good results, Google fails with it.
And the author is right about appending the site name to the query (reddit etc). Sometimes, it is the only way to avoid the crap that the search engine would otherwise provide.
Sure, but just lately DDG started deteriorating for me as well.
Maybe that coincides with another big update they did and didn't tell anyone about it -- looking for certain phrases that describe sex no longer works. A lesbian friend made me aware of that; she recently complained that she can no longer find porn through DDG queries so she started bookmarking various such websites and is going to them directly.
...What is weird is that I tried a few phrases several weeks ago and they didn't work but I just tried a few right now again and they did work this time. Strange. But there are still a few that absolutely don't work.
> Sure, but just lately DDG started deteriorating for me as well.
Might be an issue with one or more of their backends censoring certain phrases in a sporadic fashion. While they do have their own crawler,[1] I don't think it has a significant effect on the breadth or accuracy of their results.
yes, DDG & most non-Google engines are Bing under hood - easy way to check this is pairwise image searches on say Bing and DDG in anonymous browser sessions
Startpage one of few that's actually Google under the hood; we do mix of Bing, Google, and our own scraping at breezethat.com (founder here)
True, it steps on Bing is what I think people said.
And yeah I too am not hyped for it but it still does serve a real need -- to be less ad-oriented than Google. But I guess it's time to start looking around.
- "Why are people searching Reddit specifically? The short answer is that Google search results are clearly dying. The long answer is that most of the web has become too inauthentic to trust."
This is it for me exactly. I search for the following kinds of things on Reddit exactly because results on other sites aren't trustworthy: Reviews are secretly paid ads. The "best" recipe for pancakes is only what's trending on instagram right now. The latest conditions on mountain bike and hiking trails are being shared inside communities like Reddit but not on the web. The same for trending programmer tools.
- "It is obvious that serving ads creates misaligned incentives for search engines..."
What I'm shocked by is that Google somehow maintained a balance on this for so long. Well, at least a good enough balance that people still use it primarily.
- "Google increasingly does not give you the results for what you typed in. It tries to be “smart” and figure out what you “really meant" ..."
This is the most annoying behavior because I really mean what I write.
- "There’s a fun conspiracy theory that popped up recently called the Dead Internet Theory..."
I hadn't heard of this. Now that's some sci-fi level of conspiracy but in today's world it seems totally plausible.