Personally I'm hoping DDG will at some point pick up the slack from Google dropping the Discussions tab (forum search). Evidently it wasn't so popular, but I found it hugely useful for researching product decisions and haven't found a good alternative. Seems like it could be a good niche to cover.
Agreed on this. I think the quality of Google searches has gone down dramatically for this type of search, because forum posts are often where people have questions answered. It's not just product research; programming, homework questions, general subject matter, all are often answered best in forum posts.
Could be. I'm not really sure if the quality of search went down, it seems to be a lot better imo after Panda+Penguin purged a lot of ridiculously spammy pages; but I still want a way to just search discussions.
Nice, but they should open up everything more and let people install any search engine. Apple did open up quite a bit today (in terms of letting others access their platform in places where Apple previously was the sole decider), so it’s not unlikely to eventually happen.
Apple is not about giving the user control. it is about damaging competitors.
My guess is that they were testing a new search engine, and since they gave that up/wasn't done in time/turned too expensive, they just put duck duck there. or maybe they didn't even tried... anway, that does not matter. Since the point was not to have a search engine, it was to not have google by default until google pay it up to be there.
...the fact that you hope apple will relinquish control of any aspect of your mac/ios usage is cute.
FWIW you can install alternative browsers with alternative search engines in iOS as well. I regularly use a Tor browser, for instance. The only thing that's fixed is the Webkit engine, but that's immaterial to this discussion.
It's not quite immaterial. You can only use an alternate search engine on iOS if you install a slower browser due to Apple withholding the JIT capabilities of Mobile Safari from 3rd parties, thus artificially propping up Mobile Safari as the fastest iOS browser. Although Chrome on Android limits you to only the 3 bundled search engines, alternative browsers expose additional choices without being artificially limited by Google.
Now the web view runs in a separate process with a separate sandbox, so it can run the same JIT of Safari without security concerns (mainly, writable memory with execution permissions).
This makes perfect sense in light of the growing relationship status of Apple and Google as major competitors in devices space (Android and iOS), and as such, it's a brilliant strategic move on Apple's part. DDG have made the first real inroads in search in over a decade, and they've done so quietly.
I can think of a number of widely touted alternatives which utterly failed to do so: Cuil, Bing, Blekko, A9, Teoma, and more (a DDG search shows a bunch of skeletons: https://duckduckgo.com/?q=%22new+search+engine%22+challenge+...). None has had the traction DDG seems to have garnered.
I see a confluence of a number of things:
• Snowden. People are now privacy conscious, and aware of the tremendous amount of information disclosed in Web searches. DDG's huge traffic spike following the Snowden disclosures is testimony to this: https://duckduckgo.com/traffic.html
• Google is the one to beat. Microsoft, Apple, Amazon, Facebook, Rackspace, and others see Google as their primary competition. This creates an alignment of interests among them.
• Google has shown vulnerabilities. Failures to execute on a string of social efforts, most recently G+, as well as an increasing sense of distraction, as well as possible signs of weakness in its core search business, suggest a vulnerable underside to Google. DDG isn't big enough to cause real damage yet, but it can certainly get Google's attention.
• Democratization of search. Was a time when massive datacenter investments were necessary for search. That's both no longer the case, and DC infrastructure's getting cheaper, both of which cut away at Google's core competency and advantage.
• Google's lost its favored status among the technorati. While it's not clear who's won that crown, there's an increasing strong sense among many that Google have failed at their "don't be evil" pledge, have disappointed users, and simply don't have the chops they once demonstrated.
• Specialized search is making inroads. OpenStreetMap is taking on geosearch, Wolfram+Alpha and Knoema specialized data search, Wikipedia is a basic more-or-less-trusted repository of actual information (as opposed to random Web sites), Amazon is a product and bibliographic research library. There are places to go for information which, if you've got a specific interest, are better than Google, and they're carving off bits of the search market.
So, yes, for the first time in 15 years, search looks like it may be ripe for a bit of disruption.
Don't get me wrong: Google does some things amazingly well. Date-bounded Web searches still draw me back (I did some here to turn up a few of the more obscure search contenders from the early 2000s), the Google Books Ngram viewer is fucking awesome, Google Trends isn't bad, and a few other elements. Reliability of Google services is amazing. But there are chinks in the armor.
"I can think of a number of widely touted alternatives which utterly failed to do so: Cuil, Bing, Blekko, A9, Teoma, and more"
Fanchildren, stop. You know that DDG is an untenable alternative without Bing, right? Sure, they do a little bit of their own result integration, but when you actually compare results side by side, there are very few differences:
My point is not that DDG shouldn't exist, or that it won't diverge, but that to claim that DDG has traction that Bing does not is silly. Bing is larger (and more popular: it's a top-50 global website!) than DDG, and it is clearly providing much of the technical infrastructure for DDG.
I'm aware that DDG relies heavily on Bing. But that only highlights the dynamic at play: the issue hasn't been technical elements of delivering search results, but providing a compelling case to switch.
There's the additional situation that I'm absolutely no fan of Microsoft and trust them to my search data even less than I do Google. Handing either my queries via anonymizing proxies is rather more palatable, however.
DDG's share of search is still small, but its growth rate is on fire. It also seems to be largely organic (today's announcement by Apple would be an exception to this), which strikes me as generally more persuasive than growth driven by various sorts of gimmickery (see the extensive manipulations of web server stats Microsoft attempted through the years largely through changing parked domain hosting status).
Do you have a reference for search engine ranking? I'm not turning up any clear stats, though from Fool.com I get a reference to 4.7 million queries/day vs. 3 billion for Google. That puts DDG at 1/640th Google's traffic, though as I said: growing quickly.
"the issue hasn't been technical elements of delivering search results, but providing a compelling case to switch."
The problem is that DDG's compelling case to switch (anonymous searching), undermines the technical reason that Bing can provide results that are competitive with Google. DDG would almost certainly suck if it wasn't built using a search engine that uses the collected behavior of millions of people to produce relevant results.
The "we don't track" thing is a great bit of propaganda for the insurgency, but it's a terrible burden if you happen to win the war.
"DuckDuckGo gets its results from over one hundred sources, including DuckDuckBot (our own crawler), crowd-sourced sites (like Wikipedia, which are stored in our own index), Yahoo! (through BOSS), Yandex, WolframAlpha, Yelp, and Bing."
I also don't see "we don't track" as a liability, given that Google themselves get relatively little by way of relevance from profiling (though it may contribute to ad sales). Rather (and this is straight from a Google engineer): "It's really hard to do much better in search advertising than current query + location."
Isn't Microsoft just supporting them to counter Google? I read somewhere (I have tried to find it again but I have failed so far) that Bing's cost of operation per search query was more than ten times that of Google.
I understand that DDG does not have to fulfill every Google use case. However, I don't see it likely that they're going to be number one any time soon. There are multiple discussions on HN where people say that if you can't be number one, you must shut down and walk away (I think we said something similar about Spring pad). Here we are encouraging DDG even though we know it will likely not be number one.
Isn't Microsoft just supporting them to counter Google?
As is Apple now.
Pretty much, though if DDG can make a go of it on their own, so much the better.
Sometimes balancing things against one another is useful. Actually, more than sometimes.
And if all it does is convince Google to stop ingesting so fucking much personal data I'll consider it mission fucking accomplished.
Though what I'd really like to see is a true distributed search engine capability emerge. There's a tool for this (YaCy) though it's still fairly rudimentary (and is Java based, which I'm increasingly allergic to).
> if you can't be number one, you must shut down and walk away
What a terrible and insane advice! Apple doesn't sell as many phones as Samsung and therefor they should shut down and walk? Or am I missing something here?
Don't quote me on this but this seemed to be the consensus in the spring pad shutdown thread right here on HN. How is this any different? Well, Apple doesn't sell nearly as many phones as Samsung but they are probably an anomaly as they have high margins and are sitting on a huge pile of cash. They can afford to tough it out. I see your point now actually. I doubt if Apple ever needs to be in "first place". They are doing perfectly well where they are and I guess holding a fourth to fifth of all app phones is a pretty good spot when the bill of materials is less than $200 and the sale price is $600.
Anybody who thinks that an enterprise should be stopped if you are not number one is a misguided twit.
1. It would lead to toxic consumer environments with no competition.
2. It would stifle innovation as market incumbents could coast without further optimisation.
3. It fails to address the fact that everyone was in second place at some point. No industry or product is a spontaneous invention which has no competitors. The automobile had the horse; even Gutenberg had competition.
Stop if you are not number one. Craziest business mantra ever.
To me, this screams that there's an interface problem for other search engines. Google gained the majority of market share when they were providing clean and simple search results while competitors littered pages with tons of ads, banners, "curated" (often shoddy, paid) content, and oh-so-many portals.
Bing's search results are OK, but when you compare the search result pages, DuckDuckGo is much cleaner and easier to navigate even with the same results. Google is almost as bad as Bing now, where the number of ads outnumber actual search results.
Bing, to me, tries too hard to be a destination, rather than a highly tuned tool to help me find websites.
> If I recall history correctly, Google was the first to monetize SERPs with ads.
You don't. All major SE that predated Google did so, from AltaVista to Yahoo... I used to maintain a CGI-based wrapper for a bunch of SE (these 2 plus "USE IT!", Lycos, Infoseek, WWWW, anyone remember them?) in the mid-/late 90's that removed the annoying ads from search results.
Google was the first SE with text based ads though, from what I remember.
More to the point: where the early search engines were placing paid results directly into SERPs, Google has pretty much always distinguished its search results from advertising. Not always with crystal clarity, but reasonably well.
Early SE used graphical ads that were clearly distinguishable from search results (although yes, some search results were probably unmarked paid placements, which would be illegal in some countries nowdays).
The current situation is worse for users both on Google and competing SEs, paid text ads are put in front of and made to look like search results. On mobile browsers, you typically have to scroll past several ads to see a single real search result.
I seem to recall that results placement was being directly influenced by advertising / promotional consideration. Granted, this is ~15 year old memories, but I was generally using Alta Vista or Yahoo at the time. I could be wrong.
Google's benefits were relevance of results (you'd find what you were looking for on the first page, very nearly always as the first result), as well as speed. Not polluting results with adverts would help with relevance (advertising is often distressingly irrelevant).
Interesting - so you perceive non-anonymised results as better.
DDG's main drawcard is anonymity, and Google/Bing are hoping the masses will prefer customised results. So if your perception is common, DDG is in trouble.
Or perhaps they can find a way to provide location data in their anonymised searches.
Many of my friends, technical and non-technical, do prefer tracking precisely for this reason. They find it more convenient that the search engine knows their preferences, location, etc.
Bing (and Yahoo), etc are all competing against google at pretty much Google's game. We already know Google will win at Google's game. They make the rules, they hold the keys.
DDG played a different game. It attacked the hacker sphere. It then gave people something they want, privacy. And the quality was good.
Note: Android still does not allow DDG as the search engine, and makes every damn effort to ensure Google is the search engine used. And of course note that DDG, being the only real competitor is magically excluded from relevancy. But no matter, soon, soon.
> So, yes, for the first time in 15 years, search looks like it may be ripe for a bit of disruption.
If Mozilla switched Firefox's default search engine to Bing (or even Yahoo!), Google's dominance of everything internet might get some healthy competition. Google fans would howl for a day (then simply switch Firefox's search setting back to Google), but hundreds of millions of other Firefox users wouldn't care.
And the risk of Mozilla switching to Bing by default is the reason that Google is so eager to pony up that $300 million every year. Thanks to Microsoft, Mozilla is the one with leverage in this relationship.
Microsoft is probably eager to pay more than Google to be Firefox's default search engine. I believe Microsoft and Yahoo already pay Mozilla to be included as non-default search engines. So even if Google fans switched from default Bing to secondary Google, Mozilla would probably still make money from Google.
Personally it's not really my thing, but a lot of users really seem to love this, especially when having DuckDuckGo as default search bar.
---
And I agree OpenStreetMap will probably be a really big factor in the future. Especially considering how objects can be directly linked to Wikidata and other services.
It really shines for me, since I have pentadactyl installed in my firefox. From the blank tab page, I can be searching github (or the python docs, or <insert one of the multitude of !searches here) with the following keystrokes:
o !github language:python
'o' tells pentadactyl to go to a url. When I don't give it a URL it instead sends what I type to my default search engine.
The funny thing is that unless I am actually searching DDG, I almost never see it.
Sure you could do this by defining a whole bunch of search aliases, but it is more fun to think, "I wonder if I can search the Arch Wiki from here," type `o !arch some search`, and all of a sudden find yourself on the arch wiki search page.
Pentadactyl or vimperator. I can never remember which one is being currently developed. I mostly use vimperator on Firefox. Installed it as a total lark and found almost instantly that I loved it.
You mean that DDG offers the illusion of privacy, right? It's more than a little naive to think that the NSA won't be able to pull data without needing the cooperation of DDG (and they're obviously not dissuaded from doing so by any laws). We also don't know about any secret court orders that may or may not have been issued to DDG compelling them to turn over data. And how are we supposed to know that DDG haven't done a deal with Microsoft to pass on data under threat of being closed down, or even that they follow their stated privacy policy?
I'm not suggesting that any of these scenarios have necessarily taken place, but we'd have no way of knowing if they have. Also, the more popular DDG becomes, the more tempting a target it becomes to the NSA et al.
So by all means use DDG if you like its other features, but try to be realistic about privacy.
When I think of privacy, I don't think too much of the NSA. Rather, I am much more concerned about Google storing every search I do to show me more relevant ads, because, hey, I'm not comfortable with being sold to advertisers.
Now, if I was really paranoid about NSA or wanted to hide stuff, I would use much more radical measures - say: using linux Tails, only browsing with Tor, and possibly from a public network whenever possible (in order to use a different IP each time).
For me, DDG is a decent middle ground. I suppose the NSA could track my search history if they really wanted to, but meanwhile it's a bit harder for them and I am less targeted by advertisement.
With Google, you've got tracking cookies, logged-in sessions, and IP-based tracking to deal with.
For DDG, with a surveillance proxy installed, you're limited to IP + browser fingerprinting. You can largely defeat both with the addition of privoxy and TOR. I agree that, were DDG compelled to install monitoring equipment, their assurances are fairly thin defense. But they are an improvement over the status quo.
Meanwhile, you'll also escape the sort of snooping and across-the-net following side-effects of advertising and marketing:
"It's more than a little naive to think that the NSA won't be able to pull data without needing the cooperation of DDG"
How would they do this?
It's possible there are SSL flaws the NSA knows about that the public does not, but I can't think of many other ways the NSA could achieve this through technological means.
There could be a secret court order forcing DDG to place monitoring hardware on their internal network, but given that they're basing a business on privacy, this would be corporate suicide. DDG aren't behaving in a manner that suggests they believe they're living on borrowed time.
"Backdoors in SSL certificates"? Are you talking about flaws inserted in SSL software, such as OpenSSL? Or do you mean that they likely have root certificates they can use to perform MITM attacks?
The problem with a MITM attack is that it involves changing the message stream. It's not a passive attack. A MITM attack is unsuitable for wide-scale surveillance, as it would almost certainly be noticed.
MITM attacks aren't suitable for mass surveillance. All it would take is for a DDG employee to check the certificate from outside their network and the game would be up. Not only would you know the NSA was using a MITM attack on a global scale, you'd have absolute proof of it.
The NSA might be using a zero-day flaw to passively listen in on connections, but the possibility of this will diminish once the OCAP gets around to finishing its audit of OpenSSL.
And they do it with a better looking interface too. Let's hope DDG's land a default spot on Firefox or Safari soon. They both lack there own search engine and are competing with the Google and Microsoft on some level.
Mostly Moore's Law: costs of hardware keep falling. The initial Google search clusters were pretty small, a half dozen or so systems, though the Web was also a small fraction of its size.
The aspects of this I see being possible game-changers across the hosted SAAS space are of 1) autoconfiguring cheap systems, 2) better residential broadband, and 3) distributed tools which work very effectively even with tremendous decentralization.
On a per-user basis, the amount of compute resources allocated by even very large sites is relatively small. You can provision as much on your own. What providers do offer are engineering, robustness, reliability, and data redundancy.
Engineering can be provided through self-configuring systems -- a Debian base, set of packages, autoconfiguration, and a minimal set of things the user needs to worry about, and you're up and running. With SSD / flash drives, you buy yourself around much of the seek latency of spinning rust.
Robustness and reliability through a grid-provisioned service would ensure that the system as a whole is up even if individual nodes are offline. You'd also need some smarts against attacks, but if the system could communicate among nodes indicating what traffic is considered harmful it should be reasonably self-healing.
Data redundancy can be provided through sharding and replication schemes. Figuring out how to account for balance of data of a given node (if you want to load a lot you've got to offer a lot, and/or obtain credits somehow, perhaps through payment).
The one additional benefit DCs provide is for very rapid communications between nodes -- < 0.001s, sometimes better, latencies, rather than 0.01s - 0.1s between Internet sites (sometimes worse). This matters for multi-tier applications where web proxies, application engines, and databases need to talk to one another.
And if you do need some DC infrastructure, costs for that are falling (and capabilities increasing) such that much of it can be provided on a light budget (traffic costs are likely your biggest concern).
What I see is an erosion of the technological limitations which have made the present commercial model viable and/or necessary. Which suggests that it may be increasingly supplanted, especially for mature spaces.
One thing that worries me about DDG is their recent UI redesign.
It felt, and still feels, to me very much like the failed UI redesigns we've seen from Google as of late.
Even after giving it some time, I still don't think I'm better off with DDG's redesigned site than I was with its old design back at the beginning of May.
It may not be a complete UI disaster like the Google Maps redesign, for example, but it's still unfortunately a step in that direction. I sincerely hope this isn't a continuing trend with DDG.
> Has the DDG user experience suffered? If so, how?
Perhaps not for the most common case (just typing in search terms and viewing results). But it does break some important things, the most annoying one for me is that you cannot link to the JavaScript-infested result pages like the expanded image search results. How do you send someone the result of an image search ("images foo" does not even work all the time and it's not the expanded view)?
Visually, the contrast is worse now than it was before. While this may be "trendy", it does harm usability.
The default coloring of the old results page was much easier to look at, as well. It made it easier to differentiate between the different types of content.
The fixed red bar at the top is distracting and pointless.
The auto-suggest is also wasteful and unnecessary. I find it contrary to the simplicity that DDG used to offer.
Yes, they're relatively minor changes, but they add up to result in an experience that just isn't as good as the previous experience. These types of minor, but harmful, changes can build upon one another over time, amplifying their negative effect.
If you click the "hamburger" icon in the upper right, you can choose alternative themes. I'm using contrast with a larger font, but you can also switch back to the old design (although I'm not certain how exactly it mirrors the old design).
Right, that is an option, but it unfortunately involves some effort from the user. This is effort that wasn't needed before. In my opinion, having to expend this extra effort even just once ends up harming the user experience relative to what it was before.
I actually find that the changes in contrast have emphasized important elements of the page. As an example, the buttons and text on the front page will probably only need to be read once unless you're actively looking for them.
The auto-suggestions have helped improve my results a few times, particularly when I can't think of a bang.
The red bar and default coloring are probably a little more subjective, so I can understand that. I do hope changes don't accumulate to become more negative, but I find these changes to be largely pleasant.
If your concern is privacy, then DDG's "no tracking" pledge offers an additional value-add over bing.
DDG's use of additional search engines other than bing (mostly for ancillary responses AFAICT -- the primary Web results seem to rely heavily on bing) is another differentiator, and can add value.
For months i have been in transition to DuckDuckGo. The only missing bit was native search in safari for iOS and OSX. This definitely seal the deal. And is good for all the industry that a viable second choice exist to bring balance to the force , and bring humility to google monopoly in search.
Just set it as my default on iOS and searched for osx 10.10 Top part of the screen shows a Wikipedia result and the remainder shows an ad. Switching back to google.
If they can make that mobile UI much better I'll make it my default but I've essentially done a search and saw no results without having to scroll.
I don't see why their mobile page should be different when iOS 8 is reelased. It's the same problem I have with their desktop site actually. The quick answer box at the top of search results that thinks it knows what I want takes up way too much space.
One reason I still not use DDG is no support for language specific search (settings not work). It mixed many similar language in the search result, for example, english and spanish.
I really really want to like ddg, but I can't seem to get away from a need to !g. Further, though its obviously a loss in privacy, the quality in search results returning from a "context aware" google is just leaps better. Its a trade-off I guess. I know HN is very much in support of the primacy of privacy ... but there is no doubt search and ad relevancy goes down in anonymity.
I've been using DDG for quite a while now, and I've noticed in the last few weeks or months that their search results have gotten much better. Before I'd have to use !g on most of my queries, but now it's much more of a rarity.
I've switched over to it for 90% of requests. The only thing I find really fails are very precise technical questions.
Edit: Let me rephrase, it's my default search engine. If the result isn't great, I may rerun it with !g which will then forward my request on to google. Makes it really easy to migrated while still being able to fall back when needed.
For technical questions, my first go-to is generally StackExchange (!so) or a site-specific search (w3.org is great for specific CSS spec queries which I seem to have been doing a lot lately) via DDG bang syntax. Google's my third choice.
I agree with this. For programming issues I normally use !g to search Google. I suspect Google is better for programming queries because they did a lot of indexing of newsgroups, blogs, and StackOverflow. But for everyday queries, DDG works fine (and I prefer it to Google as I don't usually want to see Google Reviews/Maps/Plus links in the top 50% of the screen area... and I don't need to see results for alternate queries on the bottom 50%).
Meaning, if I just need to RTFM, DDG finds it for me. If I search for "Pymongo find function" it will return the documentation I want. If I'm having a problem or debugging something not explained in the manual, there's a slight chance I'll have to use !g and use Google. If I search for "Pymongo find function not returning all results while sorting" I might need Google.
The number of searches where I've had to use Google has decreased significantly in the last year.
I'm in about the same boat as mey. It's my default search engine, and I rarely fall back to google.
On the occasion when I do see oddball results, I report them and it gets straightened out promptly. (It's kind of awesome to be able to report a spam site, get a human response, and have the site removed within a day or two. I'm sure it won't be that way forever, but their results also keep getting better and better.)
Yes, what dredmorbius said. There's also https://duckduckgo.com/feedback which lets you report/request all kinds of stuff. (It's listed as "Feedback" in the menu at the top-right of the results page.)
Initially it was just "send Gabriel an email", which he still graciously responds to, but I've been trying to do things the "right" way more recently because I'm sure he gets a lot of email :)
I've been using it exclusively as my primary search for a year now, and it's as good as or better than Google for nearly all purposes.
You can re-run any search on google with the "!g" bang notation (or "!sp" for a proxied StartPage search, which is nearly the same).
The recent UI changes have been nice (my biggest observation was that my local CSS changes no longer actually change anything about the site, which is to say, my annoyances have all been addressed).
The primary lacking features are date-bounded search (I still use Google for this), and certain specialized searches (e.g., Google Books, Google Scholar, Google News), though DDG have been expanding their tools, and I'm increasingly using site-specific searches (e.g., reddit, HN, StackExchange, and certain blogs) rather than general Web searches.
It's my default search engine... but I find myself going to Google for about 75% of searches -- at least for my searches, they just aren't good enough yet.
From the article:
"Private Browsing mode — which doesn’t save your browsing history"
I wonder whether iCloud tabs still work in private browsing mode. In other words, do Apple and others still get to see all your supposedly "private" URLs?
This is fantastic news for DuckDuckGo and for Apple. A nice solid search engine coupled with a nice new redesign. If I were Google I would be a little bit worried DuckDuckGo are steadily stealing users. They converted me.
I wonder if they had to pay or if Apple decided to include of free in our post Snowden era.
A couple of years ago, I was at a ddg hackathon when inclusion came up. I believe they mentioned that it was on the order of a 7 digit number to be include in the list of search engine. Regardless of how much it was, they couldn't justify spending that much money on being include in Apple devices.
Meanwhile, 10.10 will track more keystrokes and gather more personal data than ever. Doesn't have anything to do with privacy. More to do with being at war with Google.
Typing in the search field on iOS 8 will fetch web results as well, which means some web server is being hit with what you're typing. Spotlight on Yosemite is getting a similar feature as well.
I'd suspect that at least on Yosemite you'll be able to disable the web search. You can already choose to chop out huge sections of Spotlight's optional functionality on Mavericks, so I don't see why that'd change come Yosemite.
https://twitter.com/yegg/status/473545653667102721
Personally I'm hoping DDG will at some point pick up the slack from Google dropping the Discussions tab (forum search). Evidently it wasn't so popular, but I found it hugely useful for researching product decisions and haven't found a good alternative. Seems like it could be a good niche to cover.