For instance:

"the actor that plays the news guy in spiderman"

ddg:

Tom Holland (side bar)

Spider-Man Homecoming (imdb)

Tom Holland (wiki)

Spider-Man: Into the Spider-Verse (imdb)

google:

Jonathan Kimble Simmons (splash, link to wiki)

J. Jonah Jameson (wiki)

J. K. Simmons (wiki)

The google result is exactly what I want. And the results were made in incognito mode so Google wasn't able to cheat with privileged information about me as a user.

At the end of the day, most people care about the product. I'm only willing to sacrifice so much to satisfy the ideal that there should be less concentration. Make a better search engine but trying to pull at the heart-strings of users about how Google is an empire and too powerful just won't work and it undermines your product and mission.

In the end, I think most people could use either Google, Bing or DuckDuckGo and be happy with the results. It's just that we're stuck on "Google is the best search engine", and while that may be true in some technical sense, many of the other search engines are just as good for most of us.

I don't really notice any infrastructure problems for ddg in Europe though, so Finland may be atypical?

that's not what Incognito mode does. It prevents your search from being included in the browsing history and doesn't send cookies from active sessions, but that's about it. Google still knows this is you being unauthenticated. You don't need to be logged into google to be reliably targeted with ads that fit your profile.

Another example is a query for "elm dict". DDG has little idea what you're looking for while Google links you directly to the docs of Elm's Dict data-structure.

You know you're looking for a data structure, right? Add structure to your query, and DDG will do fine. Easy fix.

The opposite case, when google thinks he knows what you're asking but it's wrong, is impossible to fix.

I don't want them to be smart, I want them to be predictable and search what I say, not trying to reinterpret the query.

Most of the time I see comments like this, people don't provide hard examples of queries they found unsatisfying with one search engine compared to another.

There are advertising companies that use fingerprinting for ad targeting, but Google doesn't.

(Disclosure: I work at Google on ads, speaking only for myself)

Seems like a pretty good reason to think it still knows who you are.

As to how that works, you tell us.

One way you could see something similar to this would be if you opened a clean session, logged into Google, logged out of Google, thought you closed the last incognito window but didn't, and then opened a new incognito window? Then the user cookie would still be in client-side storage

https://news.ycombinator.com/item?id=21463266

https://meta.stackexchange.com/questions/331960/why-is-stack...

Try visiting from tor

It’s in their best interests to also use it for ad targeting (in a plausibly deniable way so they don’t get in trouble).

We’ve seen them using dark patterns to coerce users into opting into more data collection, and another advertising company got caught using phone numbers for ad purposes even if they originally promised to only use them for 2FA, so why should we trust them this time?

If it was being used for targeting it would be practical to run an external study demonstrating that.

An external study to evaluate whether Google is using fingerprinting would be some work, but pretty doable. Targeted advertising is generally very blunt: if someone thinks you're especially interested in a valuable category they'll often pay a lot to advertise to you. So you could set something up where test browsers visit pages related to high-value categories (mattresses, asbestos cancer, credit cards, ...), clear client-side data, and then visit a site that loads ad scripts only from Google (to make sure you're not getting someone else's fingerprinting) and see whether the ads differ from a control group that never visited those pages.

And surprisingly for most of HN readers, Google has been pretty transparent on the policy of its ads business. In fact, Google has pretty strong incentives for transparency in this area due to advertisers, who give all the money anyway.

IMO, ads are probably the least worrisome way the data could be used. A boring but scary example is that aol search history leak (which is still searchable today):

https://searchids.com/

This person is identified by name for example: https://searchids.com/user/19431784-joann_whitman

That's literally the opposite of what s/he just said. The person you're responding asked "How would it know?", implying that they (while being on the Google ads team) think there is no way to know without fingerprinting (or cookies from non-incognito mode).

Personally, Google ads give me mixed feelings. I see how personalization is useful for everyone involved and, so long as only machines look at my data, I don't have any personal issues with it. But at the same time, Google collects everything on everyone worldwide to the point where I feel like the USA would have an easy time conquering any country they please (if a nation already has live data on pretty much all its enemy's subjects, war would be exceedingly efficient for them to start and quickly win), so that kind of threatens our freedom if you see what I mean; and secondly the data is not necessarily 100% secure, so in the event of a breach it might be seen by humans, specifically people that I would not want to know what I searched for (or pages I visited that have Analytics or an embedded YouTube video or ads or a map on their contact page or ...). So it's a mixed bag of feelings and your position (job) seems like the kind that would make one think about before accepting. I'm curious to hear your thoughts on it.

I've written some about this: https://www.jefftk.com/p/value-of-working-in-ads

"Many people would put ad tracking on this list of downsides: sites pass information to data brokers that build custom profiles for each user and allow personalizing ads. From my perspective, however, while having this information collected seems a bit creepy, it allows showing ads I'm more likely to be interested in. This makes publishers more money than showing untargeted ads, and I'd much rather fund them through better ad targeting (invisibly intrusive) than through more obnoxious ads (visibly intrusive)."

I chose this team because I thought the work would be interesting and I liked the people on it, and they were interested in me because of my prior work on mod_pagespeed rewriting websites so they would load faster.

> if a nation already has live data on pretty much all its enemy's subjects, war would be exceedingly efficient for them to start and quickly win

Lots of thoughts:

* I think you're dramatically overestimating how much data Google has and how well that is mapped to the kind of identity the military would care about.

* I don't think Google would share this information unless legally required to, and I don't think such a request would be constitutional.

* Many other countries are in similar positions; for example Criteo is based in France and has a similar ad tracking reach to Google.

* I'm still not sure how this is especially useful militarily. Military targets are mostly not in the data one of these companies would have, and none of these countries would go to war targeting civilians.

> I chose this team because I thought the work would be interesting and I liked the people on it

That is fair! I guess most people would make that decision if you already know people there and you think you'll enjoy the work as well.

> none of these countries would go to war targeting civilians

Not as if people in the army are somehow exempt from tracking though?

As for whether Google would share it in the first place: I don't think the government cares much what Google thinks if they're willing to kill (us) over something. Laws can be made by the same people that decide on this. I don't mean to pose it as a simple matter, but I'm pretty sure that's how it works in principle.

Now that I think of it: aren't "national security letters" exactly this? "It has something to do with the safety of the country, just give us that data [e.g. Lavabit private key]"?

Of course, the chance is remote in the first place. Much more likely, if it is ever used for this kind of purpose in the first place, it'll just be posturing and threats, and people will protect themselves better before it ever gets to armed conflict. Just imagine, though, if you're not in the USA, China, or Russia, and one of the three (the most democratic one of the tree, it is fair to add) has the rest of the world's data. That's kind of uncomfortable when I pause to consider it.

> how well that is mapped to the kind of identity the military would care about.

While not readily available, I expect that it's not hard to find a few datapoints to filter them out. Following someone for 10 minutes as they go through traffic and matching the coordinates against location history data is probably enough to find a subset of 1-5 possible accounts. But I doubt physical following is even necessary to find enough datapoints to find them in the data.

For the record, I'm not saying that I expect Google not to track me when they detect some privacy mode. It'll sure try to set cookies, and it may use my IP address and connect whatever that IP accesses as a weak indicator of interest for anyone else with that IP address (for a limited amount of time, since IPs change in many countries). What I don't think is that, when they say they don't do fingerprinting, they're lying. This person may not be privileged to know and say "I don't know", but that's different from saying "Google doesn't".

Also for the record, I didn't downvote you (and when you reply to me, I can't; I don't have an alt account with 1k rep or whatever it is one needs to downvote).

Anyhow, based on what I've seen, when you go into incognito mode it definitely doesn't use your profile when you go to Google services. Search results, suggestions, etc... are different. In fact, one of the use cases for incognito is using a different Google profile on a someone else's or a public computer. Keeping contextual search results from the 'main' profile logged into the browser would be counterproductive...

Google also censors some results in controversial categories, where Bing/Yahoo return what I am looking for. I don't trust a search engine that censors entire categories of results.

I am not an adwords user but anecdotes in another HN thread seems to affirm that Google is due to be disrupted: https://news.ycombinator.com/item?id=21667484

DDG's first page was only fake sites and wrong answers.

Google gave me the right answer right away (in the positions 2 and 3)

(for those who don't know legacy electronics: I was looking for the box with labeled pins, like on first page of https://www.turus.com.tr/class/INNOVAEditor/assets/PDF/MM584... )

The way they "cheated" was to look at the tens, hundreds, thousands, or tens of thousands of previous queries that matched yours in substance.

If the competing websites for "web search" had the same volume of submitted queries that are substantially similar to yours, then they too would be able to give you "exactly" what you are looking for "9 times out of 10".

If all people submitting queries on the web are somehow convinced to visit only one website and submit a majority, (hypothetically let's say 93%) of their queries there, then it should be no surprise when that website "magically" starts becoming more "predictive" than other websites in returning "exactly" what searchers are looking for and becomes "by far the best at search".[1]

The question raised by the blog post is whether enabling such "magic" is worth the trade-off of also creating a "panopticon" in that single website. Giving this level of visitation and query traffic to a single website makes any competing website (working off less than 7% of people's queries) seem irrelevant, maybe even pathetic.

Without the enormous traffic, I would surmise the glory of the "empire" (to use the author's chosen term), and its appeal to "followers" (e.g., those who marvel at its "magic"), might dissipate rather quickly.

1. Especially the part of "search" that involves dealing with repeated, similar queries for popular information.

Google's advantage is simply billions of dollars and 20 years of R&D into NLP tech.

Not directly, but it does mean that, say, the second 50% of people that ask the same question will get a better answer than the first 50%.

Google has been able to build fairly accurate instant results based on which sites users were clicking on before. I'd say that a majority of simple general knowledge queries are solved by quoting the first 3 sentences of the wikipedia page that match the search query.

But let's say that there is no easy match to show a quick result for. But after 1000 queries, 98% of users clicked on one particular site on the first page and never went back to the search results. Google then A/B trials how many clicks result from showing that website at the top of the results page in an instant result window. If clicks drastically drop, that's a sign most users are satisfied with that result. They were only able to do that, because of the 1000s of times people typed that query and interacted with the site.

So I'd say that yes, having common questions asked over and over again does help you find the answer that users are looking for.

The funny / scary part here, is that this may not be the correct answer. But it's the answer that satisfies the most users, and is therefore the one that will keep the most people coming back to the Google search engine.

I doubt that users have a strict expectation of finding the correct (interpretation of "corresponding") answer. Even if the most clicked answer[s] are still not correct, they may still be the ones who suck less, which can still be acceptable/desirable.

Keep in mind that in this perspective, the concept is similar to Google Translate - Google built a translator that, at least originally, doesn't understand language, instead, it applies (applied) a static model on a large amount of documents. While they certainly poured a large amount of money on it, it's success can't be centered purely on the economical factor.

There are at least three kinds of queries that a search engine has to handle. Requests for websites e.g. "Facebook". Traditional keyword searches across the web e.g. "Twitter ban political ads", and questions "Who was the guy who voiced bender?".

Plus, you're underestimating the percentage of queries that google has never seen before. Google processes trillions of searches every year, and still, 15% of those queries have never been seen by Google before. [0]

[0] https://searchengineland.com/google-reaffirms-15-searches-ne...

No. I am highlighting that 75% are queries that they have seen before. Note also that the 15% is a figure that is declining, based on what it was in 2007.

When you see google giving you a one/two word answer as a card, that’s very likely coming from it’s knowledge graph.

What we need is more of this open knowledge graphs. Google and Wolfram Alpha are both closed sources but have deep understanding in niche domains.

For example, say the film credits for Spiderman is a "primary source", and a cast list for Spiderman derived from the credits at IMDb is a "secondary source". Google extracts the information from IMDb and substitutes itself as the secondary source.

Does this raise an issue in that ideally users should sometimes be retrieving information from (i.e., accessing) primary and secondary sources directly, whereas a third party always acting as a universal secondary source, e.g., a third party funded by advertising, might introduce (more) bias into the information retrieval process, e.g., in competition for "eyeballs".

From the OP references: https://sparktoro.com/blog/less-than-half-of-google-searches...

It's also another matter that so much of the internet is now filled with copy-pasted crap and artificially inflated content that it's actually hard to find what you are looking for on the so-called primary source webpages. If I want to find George Clooney's age, I don't want to sift through a 2000 page ad-ridden Page3-esque gossip blog about him.

At the same time, when I want detailed info about something, I will go in and try to read through the primary material.

Even when I entered the actor's name plus the term "age" and was redirected to the Wikipedia search results, I could still see the actor's page as the third result and his date of birth in the summary text.

As for why anyone would want to search some things using Wikipedia versus Google, I can think of a few reasons. I cannot speak for other users however.

I have a script I wrote to search Wikipedia from the command line. For those who care, one can strip out "X-Client-IP" from the returned page html before opening the page in a browser.

If you want DDG to understand natural language queries, I think their privacy policy may need to be adjusted so that our queries can actually be used to develop that, and then they need a boatload of funding to catch up with Google's semi-ethical money-generating practices.

!bangs are the main reason why it's always going to be difficult for me to switch from DDG to anything else.

By contrast, the new privacy friendly search engine from the article, with a lot less money and users, can answer a simple question like "news guy actor in spiderman" with

Cliqz:

- And the new actor playing Spider-Man is... this guy (www.foxnews.com)

- J. Jonah Jameson (J.K. Simmons) | Spider-Man Films Wiki (spiderman-films.fandom.com)

Not bad for a small German company uh...

And even if it didn't, when I know it's Jonah Jameson, I can search for the actor that played Jonah Jameson. Search engines are not about having "the answer to life the universe and everything" but helping users refine their searches until they find what they were looking for.

But Google fails as well.

Most of the world population is not native english speaker.

Try "attore che recita la parte del giornalista in spiderman" (the actor that plays the journalist in spiderman) in Italian

- Spider-Man: Homecoming - Wikipedia (https://it.wikipedia.org)

- James Franco - Wikipedia (https://it.wikipedia.org)

- Spider-Man film: tutti gli attori | Popcorn Tv (https://popcorntv.it)

- Martin Sheen, dieci ruoli per scoprire un grande attore ... (https://www.consigli.it)

Or the simplified version "attore che fa il giornalista in spiderman" (same meaning as before, just more down to earth)

- J. Jonah Jameson - Wikipedia (https://it.wikipedia.org)

- J. Jonah Jameson - Wikipedia (https://en.wikipedia.org)

- Spider-Man: Far From Home, nel cast anche l'attore ... (https://tg24.sky.it)

Refining is still a very present need, it's simply that for common searches on common topics in english it's less so...

It's more or less the same in German, French, Spanish, Portuguese... I can't even imagine the results in languages like Arabic, Indonesian or Balinese.

Local, culture aware, search engines are the future, despite Google efforts, the generalist "one size fits all" search engine is not.

"Google's dominance is almost entirely due to the fact that its by far the best at search."

is, and I'm sorry to say, a make-believe

Google's quality is better than anyone else, that is a fact. Let's go to the point. [[I work on Cliqz, and in search]]

Do you know how much Google pays apple to be the default search engine?

According to you, nothing, because people will go to Google because it's the best.

Well, it turns out that last year was more than 9 Billion (with a B). Quite a lot of money poorly spend, someone should really get fired :-)

The other option is that Google does not pay to get the Apple users, which might come otherwise, but to prevent Apple to try something funny, either directly or indirectly.

In any case, the "build a better product and people will come" mantra is flawed when companies in the space pay each other billions for distribution.

Instead, Apple has a thing to sell -- "position of default search engine" -- and it is selling it to highest bidder.

I bet if Bing would offer 10 Billion, they'd make Bing default instead. Hey, if DDG could offer them 10 Billion, I am sure they would set it as default, easily ignoring the fact that many people say they do not like its results.

But please focus on Google, which is the one putting the money!

Why Google pays so much money if they already have the best product and what the users want? Are they just giving B10$ as charity?

There are many reasons for doing that, but none of them is aligned with the make-believe statement that "build a better product and people will come"

I would agree that is a necessary condition, but by no means sufficient, which is kind of sad.

Google pays billions to get those 90% of don’t care users. They already have the users who care, but more users = more money, so presumably it is worth it.

Cliqz, DDG, and others have no chance for those 90%, but they fight for the rest. It’s a hard fight because google is so good. If one spends a hour searching for stuff on alternative engine, then goes to google and finds the results in a few minutes, they will be unliketo visit other engines again.

Yeah, and due to economies of scale and network effects it will stay that way, unless some people are willing to suffer the minor inconvenience of using a slightly inferior competitor.

Yes, when what you want happens to be the lowest common denominator - in this case, the most recent Spiderman movie. When the next movie in the franchise comes out in a few years and you don't like it so much because the news guy is now played by someone else, you'll be complaining that your preferences have stayed the same but Google no longer handles them as well.

But is it? I find myself often having to fight it to search for what I entered into the box, rather than what it thinks I really meant. If you know what you want just not where it is, DDG and Bing are both superior.

Maybe it’s denial or weirdness on my side... after all I am also a vegan because I don’t want to hurt animals or other people just for a little bit of taste and convenience and most people seem to think that’s stupid on my side.

I know, Bing.. But I have used Windows for about 30 minutes in the last year and that was just to help my mom fix her printer. I don't think I have ever had a Microsoft related account since Hotmail. I know they track me but I don't use MicrosoftDrive or WintowsTube so I can live with it.

That said, it does't prevent the situation where searching for certain gifs or images gives you a page full of softcore porn, even with the moderate safe search enabled.

You can almost always, by phrasing more carefully, get exactly what you need with DDG.

I also think that it's better to learn to search properly than finding the best search engine.

For me their mission is pretty clear: Google ate Burda's ad cookies and now they are trying to get their hands into the cookie jar again. Given that today we have widespread TLS adoption the war about the endpoint has begun. Cliqz, alias Burda Media, is just another combatant - the one who controls the browser controls the ads.

Previous thread with more info:

https://news.ycombinator.com/item?id=18626790

And just one more thing: In Google Chrome at least I can turn off auto-suggest. This was not possible in the version of the Cliqz browser I tested.

Disclaimer: I work at Cliqz.

GET /api/v2/results?nrh=1&q=pythuxs=UTlnJzAh4ULkORaiZgLPO6LW9LOWZZoO&n=5&qc=0&lang=en&locale=en-US&platform=0&o=%5B%5B%22custom-search%22%5D%5D&country=de&adult=0&loc_pref=ask&count=10&suggest=0

HTTP/1.1

Host: api.cliqz.com

1. I saw UTlnJzAh4ULkORaiZgLPO6LW9LOWZZoO in other requests too, but not all of them. Can you explain what this parameter is good for and which information it encodes?

2. Can you tell if it is possible to disable auto-suggest (or Quick Search how it seems to be called in your terms)? If it is possible then how do I do it? I couldn't find it in the UI.

3. Can you specify what the exact legal relationship between Cliqz GmbH and FoxyProxy LLC is and if there is any shared ownership or if there are any common parent companies?

4. As you can see above the requests go directly to api.cliqz.com. While the terms go into great length to explain that information is routed via a third party owned proxy the information we are talking about here is exempt from that. I quote the relevant passages here:

> This channel collects signals about WHAT you search and where you land. That is why we do not collect any personal identifier here, which makes it impossible to associate searches with users. Moreover, all query entries and clicks on website suggestions are evaluated only as a single event, disentangling these signals from everything else. Thus, we are neither able to combine data from multiple entries or multiple clicks on website suggestions, nor to link this information with personal information like your email address or an IP address, either.

> Query logging data is used to further improve the Cliqz backend. More specifically:

> To be able to suggest websites in real-time while you are typing into Cliqz’ combined browser-and-search-bar, Cliqz sends your keystrokes to our servers. With every new keystroke, our backend scans our index and predicts the most relevant results for your search query.

> “Relevant” to that regard is (very simplified) defined by the frequency a given website is clicked on for a given query. In other words, Cliqz predicts the most probable site you will navigate to, based on the (partial) query that you type. In order to further improve this mechanism of relevancy, Cliqz logs the clicks in its drop down menu and the respective queries.

I wish the terms were more clear about the fact that crucial information is indeed sent directly to Cliqz and is not sent via the FoxyProxy route.

Thank you the questions, we are always looking for constructive feedback on and off HN.

1. These random values are used for grouping partial queries together, and they reset when you press enter or start a new query. Source code on how it's generated: https://github.com/cliqz-oss/browser-core/blob/master/module... We actually take one additional precaution of using crypto random and not plain Math.random(), which could potentially be used to link multiple sessions together.

2. There is no feature to disable auto-suggest. But I will pass your feedback to the team.

3. No there is no shared ownership, we don't have access to their servers. We also do additional encryption with bucketing on the payload sizes that we route through foxyProxy, so that the proxy provider cannot learn anything about the content of the message. We will have a blogpost explaining this on Wednesday - 4th December. Also, we are looking to add an option, where user can choose their own proxy provider too.

4. There are two parts: a. You can select the option from Control Center (Q menu) icon in the toolbar -> search -> search via proxy. (Now the calls should go through FoxyProxy) b. All calls to api.cliqz.com go through proxy when in private mode. The only reason it's not default is latency. As to what goes through FoxyProxy by default is: all Human Web data.

Once again, we appreciate you looking into details, and please keep digging, we would be happy to answer, improve our documentation and if there are bugs specially related to privacy and security they are on our uttermost priority.

Thanks for taking the time to dig into things. Comments like these are why I continue to regularly read Hacker News.

This "feature" should be off by default on any software that claims to be a privacy respecting alternative to Chrome.

Obviously, if you can get the content of what they're typing, it gets much easier still. I think I've seen papers where they ID programmers based on the code they've produced. This applies to other types of writing too.

You can build a model from keystroke timings and figure out people's SSH passwords too. https://www.usenix.org/legacy/events/sec01/full_papers/song/...

The main problem here is that even if you could do it, why would you? There are over 3.4 billion Internet users in the world. Given that people share IP addresses and even browsers, what's the actually gain identifying someone through typeahead search keystroke jitter? This would spend a lot of effort, and and then tell you what that a cookie doesn't?

I can't imagine that it's actually worth the effort.

That's called typeahead search, and that's how it has always worked since the invention asynchronous HTTP requests.

If you know of someone that sends a trie down to the client containing every possible completion a priori, I'd like know about it.

Well, that's how you implement an autocomplete feature in the 1st place. If they'd sent every keystroke you typed outside their URL bar, now that's something to be aware off. So, did they?

We will also be sharing more details on the process on data collection in the blog posts scheduled in the next 3 days. Of course, you don't need to trust what we say, all the code used for collecting data is open-source for transparency and auditing: https://github.com/cliqz-oss/browser-core

Lastly, privacy policies are legally binding documents, and we take the law very seriously. We are located in Germany, where privacy laws are as tough as they get (e.g. GDPR was loosely just a re-wording of the existing data protection directive in Germany).

I see two problems with their approach:

1. The product is not built with the 'grandma test' mindset.

More sliders and widgets is not what your grandma wants in a search engine. This is why building a search engine is hard. You have to guess with very little information what the user wants and get it in front of them at first try, without the user having to tweak anything.

2. Google must not fall because it is a monopoly. If it was to fall it should be because someone built a better product.

Similar to how ICE cars had "monopoly" over transportation and the time for change has thankfully come. Not because monopolies are bad, but because electric cars are so freaking awesome.

Google perfected what the 'best search engine' is to 99% of population. This comes at an expense of really annoying 1% of users but it is the price they are willing to pay. To de-throne Google you really need to cater to broad population with a product that will be better both at capturing intent and delivering and presenting relevant results. This may or not come with a different business model.

One of the reasons for breaking up monopolies is that they make it harder for newcomers to build something better.

So You get Google Search Inc. Adsense Inc. AdWords Inc, Gmail Inc, I imagine.

This would be just restructuring, Googlers are too smart to get hold back by this and they probably already have an emergency plan if it remotely comes as a possibility.

When Bell Telephone was split it was cut along geographic lines. That would also not work on a tech company for obvious reasons --- software knows no borders. I don't know of a sensible way to break up tech giants because the efficiencies of scale create a natural winner take all market.

Bell was broken up because they owned Western Electric and used it to vertically integrate the telco stack for the entire country.

A hypothetical breakup of Alphabet could mirror the breakup of the Bell system, as Alphabet controls the data collection -> advertisement stack. Spin off Search + AdWords as an independent business, while services targeting data collection services that feed it (Chrome/Chromium, Android, GSuite, Google Home, etc).

Personally I could see a lot of consumer benefits.

Search, inc is still going to want ads, presumably, so they'll contract with someone. Setup rules for the contract. Maybe require at least N ad providers with each getting a minimum of Y% of pageviews, and contract terms have to be FRAND.

Strongly restrict personal information passing between the companies.

Or, i guess you could go all Bell on them and divide the US into different territories and have Pacific Google and Southewestern Google and what not. Would be kind of weird to geofence search and ads though.

My point is that if your company sells information rather physical products then breaking it up is pointless.

Is Google somehow suppressing the creation of a good search engine? No, on the contrary it has created the best web search engine we've ever seen. I would personally be sad if you took it away from me, and I suspect that 99% or more of its users would be in the same boat (don't judge the zeitgeist by web forum echo chambers). You break up monopolies when they are harming users, not in order to cause harm to users.

With the world being what it is today, Google Search is in an extremely powerful position. We depend a lot on information that we find on the web. And these free form search queries are the best UI we have at the moment to retrieve it.

It is a good thing that Google Search is good and is getting better, but that doesn't take away from the fact that they wield a lot of power that needs to be checked somehow.

In the case of petroleum fueled cars, they are significantly contributing to a massive climate emergency and should be banned regardless of the existence of a superior product. In the case of the Google search engine... Well in the case of the Google company, it should be forced to split up because it is a massive monopoly that hinders competitors from entering any of their dominated markets, and they use their domination of one market to increase their dominance of another.

And this is should be done regardless of the existence of superior products. Goggle as a single company is making the world a worse place, and the right thing to do in that scenario is to split it up.

Would smaller farms with smaller fields and a larger variety of crops be better for the environment? Would those farmers grow more organic food?

I don't think that this is black and white. But smaller companies would likely be a whiter shade of gray in some cases.

But highly engineered products are often ideal for:

1. Having a longer shelf life. (Ideal for countries without the same standards for food preservation) 2. Having yields large enough to feed populations. 3. Increased ability for transport.

I agree that there are gray areas, but it's these qualities that help feed a world.

> One could rightfully counter that Google has a good product. They even offer it for free.

But Google Ads are not free. If Search is the product, then advertisers are the customers, not users.

I second that. It's somewhat reassuring to know that others realize that as well. Most people just have no idea how marketing and now 'deep learned' algorithms (applied psychology) make puppets out of human beings.

I consider "user feed control" (that users choose what, how, where, when and why things are presented to them) of equal importance to e.g. democracy as far as human freedom is concerned. But the current manipulation is so insiduous that from now to mainstream awareness to generally implementing solutions is a long, long ways away.

> at least $50, in addition to whatever the services I use already cost me

I think this tends close to an upper bound — not many people willing to pay, and even fewer at that level — but certainly one more anecdotal proof that premium services are totally viable for a certain group. The question is 'how big' that group, what's the market for that, but I'd wager it's enough to sustain a few 'premium' businesses (or alternative plans) for most common services (some are harder; search notably).

When a Wikipedia/encyclopedia article is what I'm looking for, why is Google showing articles?

Wikipedia used to be the top result.

I've noticed things on the spectrum of articles <-> blogspam crowding out the top results in many queries, and that it's substantially more difficult to find forum content discussing related topics to the query. It's a shame, as I think there is often more interesting content on such forums.

Google's algo changes since about panda have been burying good web sites and content while bringing quicker answers and 'sanitized' aka semi-censored results to the top.

Some of this is over reaction to SEO and trying to out do the spammers - but the collateral damage to the results and thus the end users who believe that google brings the truth is hard to calculate.

Combine that with regulation like dmca, right to be forgotten and others.. results are even more censored, and the general population does not know what they are not seeing, as they still trust google to be bringing the truth.

worry about bad PR from various factions - tweak the algorithms.. and you can say for sure that the results the more adolescent google brought a decade ago were often more of what people were looking for.. and the results today are often like cheap irradiated / sanitized snacks, not the full enchilada that was once a G search away.

Much of this started happening when whats his name became that adult in the room and started putting the finger on the scale to change what millions could find, it's gotten more and more censored every update since then, and less transparent about that.

in my biased opinion.. your searches and the results will vary. I still use other engines for different things, and I feel strongly that we need more search engines. Anyone who wants to create a better adult engine, let me know.

You should at least acknowledge your bias by being transparent enough to show you have a vested interest. It's really disingenuous otherwise, and makes it hard to take what you have to say at face value.

Though it's important to remember that not everyone on here has English as their first language, and we should generally be considerate of that when reading people's comments.

They've clearly optimized for the mass user base, since the average person is probably fine with a result like that. But I'm willing to bet no one on HN would ever click a link like that, just because of how "spammy" it feels.

When I search for something like "best headphones", my ideal results would be forum/discussion posts (eg. like "Ask HN") where I can read what other programmers/hackers are using, their experience with it, etc. And that's the problem with trying to make a one-size-fits-all search engine - it isn't possible to make everyone happy.

I've actually been working on a product to solve this exact problem. The best comparison would be the old "discussions" filter that Google used to have until they removed it. I'd love to show more people and get feedback. If this is something you'd be interested in trying, drop your email here ( https://degoogle.typeform.com/to/QzVy7c ) and I'll send you the beta

I feel the force of small sites like HN is precisely to be in the shadow and too small to interest spammers.

What bothers me is searching for a specific product and having Google promote a competing product to the top search result (or as a "featured" listing that looks like a normal search result).

Reddit is a great option, but there's a TON of hidden forums out there that are gold mines for information, which is something I've been also keeping in mind while building this product.

I made some typical searches I do in my job. They returned reasonable results. Impossible to know if they are better or worse than Google. I need some days of usage. I'll do my best to keep using it this week.

Maybe autodecting the language would help. It gave me a German page without any obvious way to change to English.

I went to the settings, changed the language and discovered that it wants to know my country. I left Germany because it looks like profiling and I don't want to help them at it.

Then I disabled every feature (news, weather, etc). I'm interested into a search engine, not into a portal from the 90s.

However I'm afraid that Cookies Autodelete and other privacy extensions will delete those settings and I'll have to do it again. I'll probably hide them with uBlock Origin. For the language a URL ending with something like ?lang=en would be great for bookmarks.

And finally, do we need more independent search engines? Yes, definitely.

This is still beta, so please keep on using it and we'd love to hear more feedback [beta@cliqz.com].

If not that, then at least IP, considering I’m sitting in the US.

You should create a contact form for the search engines. Everything reachable from the Contacts link at the bottom of the page is about other products.

* search engine is a cloud service, which is not controlled by user of that engine

* search gives final results in milliseconds (why? because google cannot spend many seconds/minutes for you, that's why)

* search creates information bubble (that user cannot control) because it tires to satisfy user's expectations

Most likely this new 'google-killer' will be:

* open source, because user should trust the code

* self-hosted (easily deployed in a click), not a cloud SaaS. Our search preferences have ultimate value, no one should have an access to it

* more useful than google because of accumulated data about you that are processed by computational knowledge engine (something like WolframAlpha)

* background reasoning - this engine can work continuously and utilize your own computational resources on notebook/PC and bring you brand new search insights that google never will be able to deliver (because they cannot dedicate a lot of computational resources for each google user).

Sounds good, isn't it?.. Maybe this kind of software already exists, could someone point me out?

>search engine is a cloud service, which is not controlled by user of that engine

So is hacker news.

>search gives final results in milliseconds (why? because google cannot spend many seconds/minutes for you, that's why)

Do you want latency to be larger??

>open source, because user should trust the code

That would make it easier for SEOs to game the system. Or perhaps it would put everyone on a level playing field. I'm not sure.

>self-hosted (easily deployed in a click), not a cloud SaaS.

Do you mean users should keep their own index of the whole web by themselves?

>more useful than google because of accumulated data about you that are processed by computational knowledge engine

First I think you greatly overestimate how useful information about you can be. Second if that worked then it would make they problem you mentioned before (information bubble) worse...

Instant results are good for sure, however very often few more seconds - in addition to instant results! - is not a problem if late, more carefully processed results can save minutes of my time - for now I spend it for opening the links and scanning the content with my eyes.

The same is about not very often but important searches that may be described as 'research about something', in this case I'm ready to make complex, well detailed query and wait even hours - then back and get well organized and intelligent results.

> Do you mean users should keep their own index of the whole web by themselves?

oh no! Users should keep only their personal data - in wide meaning, this includes all history of searches, search results, refinements, anything that ML currently uses to bring personal search experience. In addition to that, relatively small index of important content may be saved. For internet search this 'personal search' will use API of anything that can be used manually for now - google, bing, consume direct API of Twitter/FB/Medium/WolframAlpha and hundreds of connectors to other cloud services. It is important to say, that this 'delegated search calls' may be anonymized.

It will be important that search results are not limited only by what google decided to be 'top results for this user'. At this moment I can do all this manually - open N tabs, query many services, compare results, open most 'relevant' (from my human point of view, not google) links and scan them for most interesting information. I believe that all this can be automated.

> First I think you greatly overestimate how useful information about you can be. Second if that worked then it would make they problem you mentioned before (information bubble) worse...

As for now, all this just thoughts. I'm a programmer with almost 20YOE; I have understanding about how google works in general, how lucene works, how WolframAlpha works, modern approaches to NLP and search-driven queries processing, and I think - without a MVP that works, this is more belief, of course - that value of this 'personal computation engine' combined with modern ML approaches might be ultimate. Challenge, but nothing impossible!

I will grant that Google has a more intelligent indexing and ranking of Stack Overflow. However, DDG is making major progress there, and I rarely need to add !g

Pro tip: with the bang shortcuts, you can add them anywhere in your query, it does not need to be in the beginning of your query string.

“This result isn’t that great. I’m going to !g just this once”

Then, one week later I’m adding !g to literally every singe search so I switch back to google because what’s the point?

I really do hope to one day get off of google though.

An example would be something like "Rick and Morty episodes." I know google will give me a list with recent/upcoming episode names and air dates for pretty much any show. DDG will link me wikipedia and fan wikis. I make a "<show> episodes" query anytime I want to know when the next episode of something is released.

DDG's knowledge graph (and/or query parsing) is just so limited I skip it anytime I think google will be able to produce the answer directly. Similarly, there are things I'm confident asking a voice assistant, and there are things I won't even bother trying. If it's something I'd ask an assistant, I'm skipping DDG.

We're better served by breaking them up and enforcing and strengthening the pro-competition laws on the books. The government has a pretty mixed record with regulating industry, and a far more impressive record of investigation, enforcement, and prosecution - corporate break-ups are far more in the wheelhouse. Regulation requires constant vigilance, whereas a breakup is self-executing once the case is won.

It's not anti-success rhetoric, so much as an acknowledgement that tech has become increasingly concentrated and anti-democratic. The Sherman Act was passed in 1890 - these aren't "knee jerk" solutions - they've worked effectively in the past to curb corporate abuses.

But the tech giants run a single system: Amazon in the delivery, FB in the social graph (DB), Google in the knowledge graph. Those would be incredibly hard technically to break up. The less technically complicated split would mean that one new company has all the revenue and the other has all the costs.

I haven’t heard any credible proposal on how this “breakup” would actually physically work. If anyone has links to good sources I’d be interested.

Fair point - there are certain cases where software platforms are too valuable and complex to break up, and should be regulated like utilities with rules for fair play. Amazon shouldn't be able to use it's marketplace data to monopolize entire categories, Google Search needs to be a public platform, with regulated rates for API access (which Google itself will have to abide by).

You don't have to break up many software systems to prevent the worst abuses. Facebook can keep the knowledge graph, but they can't keep WhatsApp or Instagram. Amazon can keep their store, but they can't own a FedEx competitor or AWS - those need to be broken off. Google can keep Search, but Ads, Doubleclick, Analytics, Waze, GCP, Google Home - all of that needs to be broken up.

Some of the resulting businesses may have to update their model or may not remain profitable, but historically break-ups have resulted in an increase in value for the new businesses.

>Some of the resulting businesses may have to update their model or may not remain profitable, but historically break-ups have resulted in an increase in value for the new businesses.

All of this teaches one lesson: be more like Apple. Treat your customers poorly and massively overcharge them. Make sure you are never the biggest in your field. Essentially, don't offer products that are too good for the price.

I'd also like to point out that if those companies have to be spun off on their own, then the only way for some of them to make money is to sell your data to third parties. Right now Google doesn't seem to do that, but how else would Analytics monetize itself, if it cannot offer ads?

Charge money for the product. Plenty of other companies in the space do so successfully. There's ample business value provided by good analytics.

Google Analytics was originally the product of Google's acquisition of Urchin Software. At the time, Urchin's self-hosted version cost $895 with additional optional "modules" that cost up to $3,995 extra. Their "on demand" version cost $199 per month.

You’re wrong on both counts: Apple has its own share of potentially monopolistic behavior, and there are billions of people worldwide who disagree with you on how the company treats them.

But I rarely see Apple being included in these calls for breaking up the tech companies. Also, I'd be surprised if Apple even has a billion customers, let alone a billion who are happy with Apple.

Does it also have a monopoly on apps that can be run on the “HomePods”? Does every smart TV manufacturer with their own OS also have a “monopoly” on their ecosystem?

Let’s go further down the rabbit hole. Does Tesla have a “monopoly” on software that can run on its cars?

If non insignificant switching costs defines a “monopoly” every software as a service app would have a “monopoly”.

Unfortunately, neither the EU or the US has ever defined “monopoly” like HN posters...

I just explained one possible scenario how this could be defined as a relevant market and you're now throwing a bunch of random assumptions on my comment. I would read the link thoroughly (it explicitly mentions DoJ?) and study more on the history of the antitrust law and its applications before doing such.

If that were the case, every single console maker since the mid 80s would be declared a monopoly. Why hasn’t that happened?

I already told you that the market defining process is a very case-specific one and big techs now are unprecedented. Why are you trying to find applicable prior arts on such cases? You asked how Apple can be monopoly on a market and I gave you one possibility which may or may not materialize due to its uncertain and complex nature. It's pretty hard to understand why you're being so defensive on this issue?

> If that were the case, every single console maker since the mid 80s would be declared a monopoly. Why hasn’t that happened?

There has been multiple antitrust lawsuits and investigations on Sony, Nintendo and Microsoft for their gaming consoles while their exercise of monopolistic power was nowhere comparable to Apple's nowadays. Why don't you google just 2~3 words before making such a false claim?

Maybe because I don’t believe that people can randomly make up definitions instead of citing precedent?

There has been multiple antitrust lawsuits and investigations on Sony, Nintendo and Microsoft for their gaming consoles while their exercise of monopolistic power was nowhere comparable to Apple's nowadays. Why don't you google just 2~3 words before making such a false claim?

Well seeing that Nintendo specifically has been very strict about what was allowed on its platform, forced third parties to use its manufacturing facilities since the 80s, forced all software whether distributed physically or virtually to be licensed and to pay a fee and had a much larger marketshare, where was the government intervention? Where were the consent decrees? Lawsuits?

Please provide one citation where any of the console manufacturers were ever forced to change their business practices?

If my claim is “false”, you should easily be able to find a citation.

While you're trying to frame my argument as "make up definitions", the reality is not; this is a standard practice since 1982. Spend your time on searching and studying the topic, not mine. This is an area of vast complexities and I don't think it's effective to spend my time to enlighten you.

https://www.justice.gov/atr/operationalizing-hypothetical-mo...

> where was the government intervention? Where were the consent decrees? Lawsuits?

Your ignorant in the topic doesn't necessarily mean an actual lack of a prior. In Nintendo v. Atari case, there was not much arguments on the market definition, but its practice was illegal or not. Yes, I've been talking only about the market definition and you're intentionally conflating the concept of the relevant market definition and antitrust violation. Don't do that. And please don't even try to say "come up with evidence". You can spend your time on studying this.

And your citation were about hypothetical arguments, and went to show more of my point. All throughout the article it speaks about the government’s arguments being “defective” and still doesn’t show a single example where a vertically integrated minority player was called a “monopolist” nor where government imposed remedies or sanctions were implied.

In fact, Atari vs. Nintendo affirmed that Atari did in fact violate Nintendo’s copyright when it tried to circumvent Nintendo’s control over its platform.

So in fact, you still haven’t come up with a single precedent where Apple could be considered a “monopolist” on its own platform or where they are violating “antitrust”.

If you really want to change the landscape and make it actually feasible, force companies to expose APIs or impose standards on them. Both of these are very hard things to do in practice but at least they give a hope of a better future - increasing competition.

Breaking up big tech, depending on the specifics, will cause either worse productivity or (in the best case) change nothing.

The facebook conglomerate could similarly be broken into Instagram, Facebook, etc. Each would get a portion of their current ad business unit.

In terms of software, each baby Facebook could get a flexible royalty-free license to all the software currently owned by Facebook, and they would be free to derive from it to differentiate over time.

If there is a will, there is a way.

Turn the relevant parts of it (the ones which are affected by network effects, or other barriers to entry) into an open platform that can be accessed on an equal basis. That's quite a bit easier than breaking up a mining company. For example, Android is already "broken up" from this POV, since the likes of Amazon can use AOSP to create their own Google-free platform, and applications written to run on AOSP will work on either version.

Is it the index itself? There's already a fairly decent open-source equivalent to that in the form of Common Crawl, which has been out since 2011. People (including me) have tried to build search engines off of it, but it never seems to work quite as well as Google.

Is it the computing infrastructure? That's already been commoditized and offered as a service by multiple providers - AWS, Azure, GCP, SoftLayer, etc.

Is it the serving infrastructure? Commoditized by ElasticSearch, which uses many of the same techniques as Mustang and in many ways does it better.

Is it the ranking algorithm? It used to be that Google would agree with you. However, the ranking algorithm changes basically continuously - the one in use now is very different from the one that was used when I left in 2014, which was different from the one where I learned how it worked in 2011. I've heard the new one is heavily machine-learning based: given a suitable training set and some learning-to-rank papers you could construct something similar.

Is it the log & clickthrough data? Imagine the privacy advocate conniptions if that were open-sourced. AOL got in huge trouble when they open-sourced their click-logs circa 2002.

Is it the evaluation system? Mechanical Turk exists, and Google's rater guidelines are public.

I'd argue that the real competitive advantage of Google now is the brand and associated consumer habits, and it's really hard to break up a brand. Same reason Coca-Cola remains dominant 150 years after they started selling cocaine-laced sugar water. This is a recurrent problem in the economy today - brands fuel not just Google, but also other giant monopolies like Coke, Nike, J&J, P&G, DeBeers, McDonalds, Wells Fargo, and so on, and in many cases the companies that own them get to practice some exceptionally bad behavior. But short of reaching into each consumer's head and getting them to consider each purchase on purely rational factors, I don't see how to fix this.

It's a huge advantage and critical ingredient for ranking in all major websearch engines as far as I know. Google gets billions of queries and clicks every day. Your startup? Pretty much none of that. How are you going to train your ranking algorithm with no data?

> Imagine the privacy advocate conniptions if that were open-sourced. AOL got in huge trouble when they open-sourced their click-logs circa 2002.

Right, logs with such level of detail as individual user sessions would never get public for this reason, too much legal risk. But even simple aggregated datasets of the form "how many clicks did this query-url pair get" would be very useful to bootstrap a competitor and can be anonymized much more effectively.

Doesn't this run counter to private property though? Or what would you have happen when Google says "no"?

Counterargument: you can't copy the people who know how it works.

Google has:

- Android OS

- Nest

- Search

- Advertising

- Chromebooks

- Pixel

- YouTube

- Google Cloud

There are clear lines where they could be broken up.

I’m not saying they should be broken up.

>My biggest problem is: How do you break up a software system?

There are people who have created entire careers about exactly that: antitrust remedies. I defer to experts where I can.

We entrepreneurial wannabes on HN are only going to provide unrealistic answers that become battles of will for the rest of this Sunday before all is forgotten overnight.

All of the virtuals you mentioned have different stories. Cloud is profitable, and is not the top company anyhow. News is not a product it's a grouping of news stories. Youtube makes money through ads and better access and could be considered a loss-leader but shutting it down will not make the field more competitive. Android is open source.. and perhaps could be seen as dumping to prevent others. Gmail is a mail service, others exist.. and starting a new company will not cost you billions unless you plan on serving billions of people. Drive is one of many companies that didn't cost a billion to start but might be worth it now.. try dropbox or box.com or rapidgator.

A democratic country has an interest in ensuring that our labor and goods markets also remain open, competitive, and democratic in nature. We've made this choice as a country repeatedly throughout our history, and it's time to do it again.

“Shady data practices” is hand-wavey and non-specific. What do you mean? Many companies today (again, across All industries) have had issues with keeping user data secure, selling it to untrustworthy third parties, not giving users transparency or controls in what information is shared, etc. Google’s track record in these areas is far better than most. How does this necessitate a breakup?

Concentrating industry through M&A is a legitimate issue, in my opinion, but it’s also probably the easiest to regulate.

Our tech companies are the most competitive in the world, bar none. I’m not sure that cutting them off at the knees will help with that. ”We did it before,” isn’t a convincing argument that we should do it now, under much different circumstances.

I used a cover-all term, because there are too many to really name. Facebook is a co-conspirator in defrauding American elections, Amazon uses their data to kill small businesses, Google has been repeatedly fined in the EU for using search anti-competitively. Those are just 3 tech companies, and doesn't get into how data is used abusively by credit agencies, banks, and other financial institutions.

Regulating industry is a fools errand, and part of how we wound up here. Breakups are the only self-executing, corruption-resistant solution.

Are our tech companies the most competitive at in the world? The largest companies have been cutting off our startups at the knees for a decade, so we have no idea how competitive we could actually be. It doesn't really seem like our big tech companies have to compete much at all these days, actually.

So, suppose Google Search is broken away from alphabet. What changed?

I can understand that argument about Youtube (which is losing money and would probably fail), but google search will keep being dominant and their ad revenue won't change significantly.

Every week we have posts about new products, often self-posted. If they're upvoted, it's that people find it interesting. Quite ironic writing the blog is an emotional rant (that I would disagree with, I find it quite argumented), while your whole comment is whining that people upvote things that you don't agree with.

The problem with monopolies is when the company decides to e.g. price gouge or gets lazy by not innovating. Companies can only really get away with this and survive when they have a monopoly.

Monopolies also kill competing products and deter other companies from even attempting to enter the market - so you might love the product of a monopoly right now but if the monopoly had never existed an even better product could be available today.

I think people generally love Google's products but it would be good to see more competition. The upfront investment you would need to create a competing search engine is prohibitively high so most companies aren't going to attempt to make their own.