Want the "cheapest car insurance"? Google is zero help. It sends you to Geico or a bunch of lead-gen sites, and no matter what, you will end up at an insurance company that makes the biggest margins.

If you really want the cheapest car insurance, you need to find a company that doesn't advertise, and is non-profit, that way your premium is spent buying insurance, not TV ads and Berkshire Hathaway's stock appreciation.

Whenever I see a SERP full of ads, I search for something different. When there is nothing but ads, Google is sending you to high-margin crap.

The categories where google is pushing a lot of ads or supplementary features like knowledge graph are the categories where organic results are not a good experience. 9 times out of 10, when i search for something on google i get a traditional SERP with little to no advertising. If you're searching for something that doesn't necessarily lend itself to a web search, google should be trying to find some way to make that useful to somebody, whether that is a feature that gives you better results or an admission that their organic results for that query are useless, so here have some ads instead.

I dont if it would be possible for a subscription based search service to exist, but as long as they are selling to advertisers and not to the people there is very little incentive for this to change imo.

What advertisers can afford to pay is margin X conversion rate. If people search for cheap socks and you have high margin socks, your conversion rate will be lower. In super competitive markets, the conversion rate is usually what makes one win. For something like insurance quotes, that probably means the one with a known brand, supporting advertising in tv & radio, etc.

The quality of the sites and the margins will probably be pretty similar among most players.

Where the dynamic you're describing happens is between product categories where you have a high margin per unit in one product category. Eg. If you want to sell a 'diy divorce kit with a $5 margin' you will have to outbid divorce lawyers who expect to earn thousands per client. The lawyers will drown everyone else out.

A better example would be someone selling "cheap" socks but with expensive shipping. Or, aggressively upselling additional sock products in the checkout, possibly with FTC-unfriendly opt-in vs opt-out methods. Or, hammering their users with advertising after the sale by email and phone.

When a particular vertical market gets aggressive, the item being advertised usually brings little, no, or negative revenue to the advertiser, and all of the money is made on the back.

The worst lawyers will drown everyone else out, including the good lawyers. The lawyer that has a high conversion rate by promising it will cost $5000, but ends up costing $30k drowns out the lawyer who legitimately quotes $10k.

I agree, for certain searches, the ads are helpful, but I stand by the notion of avoiding SERPs with tons of high value ads. The person paying for those ads will likely be you.

AdWords ad placement is determined by a mix of quality score and bid size. It's good that they're using quality of links to influence ad ranking, but here's what troubles me:

We're allowing money to influence information salience.

You could say this is how the world works, and I can't deny it. Money = power = eyeballs. But is this a good thing? Is this a dynamic we should be actively supporting? I honestly don't know.

Possible counterargument (because I can't help but play Devil's Advocate against myself):

Maybe the auction model is about determining the true "market price" of an ad position, in which case maybe this is a good thing?

Ironically, considering the original article starts off by complaining about its effect on search results, the most-free-from-financial-influence way of ranking information salience is probably the Zagat review model (selling aggregated secret shopper info)

Your argument seems to be that if we get rid of sponsored search results, this will push advertisers to invest more heavily in less visible forms of opinion manipulation. This is actually a very interesting viewpoint and possibly a valid argument, but it also feels like coercion in some way ("give us ad space or we'll reach you via less ethical methods").

While I agree with the concept, as I work for a non-profit and belong to a credit union instead of a bank, I don't think your hypothesis holds without some data to back it up.

First, Geicko isn't ranked number 1, affiliates who mask themselves as Geicko are. In high rewards markets like insurance (where 30% of the overall price of insurance goes to affiliates) it's actually about which affiliates convert the best and have the best payout. Those SERPs are dominated by black hat SEO. None of that is organic.

The larger issue (percentage of the page dedicated to SERPs vs ads aside) is that Google's formula worked best in a world where people weren't trying to game it. In the end it breaks down to relevance and links.

Relevance is based on tags and keywords, which everyone can get a perfect score on. So in the end it comes down to links.

Is a link there because someone just loves the site, or because some SEO schmuck built it? There are correlations, but there's really no way of knowing. The key to spam is mimicking a human process closely enough that a computer (or algorithm) can't tell the difference, then making that scale.

Google does manually review the SERPs for the highest grossing terms basically daily to try and find foul play, but those can be beat, too.

So why doesn't Google just manually find and serve up the best pages of content? Because you just described Yahoo.

So it's a very complex game that evolves every day as both parties advance and become more sophisticated, often in response to each other. If you think you can do better you either have to out-engineer Google or find a way to rank sites based on something other than links.

It is a great example of how constraints build value. By constraints I mean, they have had to deal with military customers stationed in far flung places for years. This built their capability to conduct business virtually.

Privilege Underwriters Reciprocal Exchange (PURE) is not "non-profit" but theoretically "policy holder owned" with surpluses to the benefit of subscribers.

To your point about credit unions, insurance 'unions' are interesting:

Overview: http://www.pureinsurance.com/about/index

That said, there is a marketing arm that extracts value as fees for marketing and management, and a financial stakeholder umbrella seeking return on capital. It's an interesting structure:

Structure: http://www.pureinsurance.com/about/pure_group

In any case, my experience with PURE has been superior to my experience with other insurers (except Met-Life who was excellent but unable to competitively insure my property), and my costs have been 30% less than the most competitively priced usual players.

Aren't mutual's non-profit? They aren't trying to extract a profit for investors or other owners, but just need to charge enough to pay business expenses (including salaries, as well as the main expense of an insurance company, paying out claims and funding reserves to pay out claims, etc.)

USAA mentioned in some other followups is... even weirder than a mutual insurance company, but it's legal structure is effectively the same as a mutual insurance company.

No, they distribute the profit to policyholders, who are effectively shareholders of the corporation. I belong to one, for auto and home. I get a dividend each year of 20 - 25% of the yearly premium, which is competitive even without considering the dividend. It's not open to the public.

AAA who do my insurance: http://en.wikipedia.org/wiki/American_Automobile_Association

If you're getting information for free your information likely has a cost you're just paying for it in an obfuscated way in the form of higher margins when you shop at a place that advertises heavily.

If you ever look at Hulbert's Financial digest, which tracks the total return of investment letters you'll see that the ones that do well are $120/year. Oh yeah, you have to pay for the digest which will tell you which newsletter performs the best and that isn't cheap either.

Of course free investment advice is available everywhere online and usually makes a good contra-indicator. If I see a hype filled story show up in the freebie investment news it usually means the stock is about to take a short term down turn.

Good time to short it.

I have an entirely different perspective of Consumer Reports (the online variant which has external site ads on it).

Understand that they are entirely biased (as we all are): They take home the lucre by keeping you thinking that you need them, and that they're doing an important service. Hence why the big noise about things like not recommending the iPhone -- all just noise to keep the subscription dollars coming and to create a sense of utility. They're also prone to taking complex products and then trying to distill it into easily quantified values that they can put in a chart and categorize, leading to a detachment from real-world value and "need to fill this issue". I was a subscriber for several years and eventually came to realize that their top picks were usually not really top picks, and they were often in areas where they were entirely out of their depth (I discovered that particularly after buying a set of their top pick speakers).

After that print subscription ended, years later I was looking at getting a new furnace and saw that the online site listed furnace reviews, though they were hidden behind a paywall. Okay, bought a subscription to find that there were no reviews at all, but some useless summary about basic efficiency levels. Well that's useless. Oh well, a year subscription wasted. A year later I discovered that they auto-renewed (never signed up for that, but whatever), and put it on my todo list to jump through hoops to cancel at some point, wasting my time (as they give NO clear way to cancel, despite subscribing being a trivial painless step). The next year they sent me notice that my credit card had expired -- awesome, easy way to finally unsubscribe from an auto-renew that I didn't opt into without sitting on hold for half an hour! Imagine how funny I thought it was when they guessed my next expiry and just used that, continuing the subscription.

I have an extremely poor opinion of CU, and I think the belief that they're more pure or their opinion more useful is a bit naive.

They are not guessing the expiry date - I believe it is a service provided by some credit card companies, possibly to large merchants. It happened to me for several years with my Yahoo Email account - that would renew for $20 / year even though my card expired twice during that time. It stopped after I reported my card as lost/stolen - and got a replacement card, so the actual card number changed. Fastest - albeit a bit drastic - way to get out of a recurring billing if you are having trouble cancelling.

That said, I'm not sure he can complain that a recurring subscription continued to, well, recur.

If their was fraud, you could get a chargeback, and I don't see that happening very often.

Guessing the new expiry date in order to exact a payment is taking an unauthorised payment. It can't be authorised because the card-holder can't authorise payments beyond the expiry date.

Taking a payment that isn't authorised by the card-holder by guessing some of the details of a card against the expectations of the card-holder appears to be fraudulent to me.

Even if you cancelled the card outright, you're generally technically (and legally) still on the hook for a subscription if you don't go and cancel. Read the ToS for just about any recurring subscription. It's your obligation to cancel, not their obligation to guess when you want to cancel. Honestly, I'm not sure how that's unfair; it's not a service provider's business to interpret your payment methods and how they might map to your desire to cancel or retain a service.

The only reason most services don't bother to pursue payment for expired cards is because it's not economic to do so. So when a card expires, they eventually mark the account as overdue and disable it. That's not the same as a clean cancellation; technically you still owe them for the services rendered between the last invoice and the final cancellation.

Maybe, that would depend on the legal terms under which you signed up. Mostly you sign up and pay for a fixed term. So contractually the expectation is that you don't continue payment.

If legally the payment is due, then that's still doesn't mean you can have authorised it on a card beyond the expiry date. Moreover you haven't authorised the payment with the new card (in the circumstances we're considering).

So the payment may be due, but it's been fraudulently acquired. There are procedures for collecting overdue amounts. Of course the companies don't want to follow those procedures and air their unfair [attempted] retention practices in court.

Look at the Amazon Prime TOS, in all-caps: "UNLESS YOU NOTIFY US BEFORE A CHARGE THAT YOU WANT TO CANCEL OR DO NOT WANT TO AUTO RENEW, YOU UNDERSTAND YOUR PRIME MEMBERSHIP WILL AUTOMATICALLY CONTINUE AND YOU AUTHORIZE US (WITHOUT NOTICE TO YOU, UNLESS REQUIRED BY APPLICABLE LAW) TO COLLECT THE THEN-APPLICABLE MEMBERSHIP FEE AND ANY TAXES, USING ANY CREDIT CARD WE HAVE ON RECORD FOR YOU."

I'm guessing that their definition of "CREDIT CARD" is wide enough that they can roll the expiration date; when your card expiresd at Amazon you can just update the date without entering a "new card" - they consider the new physical card to be the same "payment method."

You have a remarkably apologetic attitude towards a scumbag industry.

Just to be clear, here's the modus operendi for the retention industry-

1) Trick or outright scam people into recurring subscriptions. I never told Consumer Reports I wanted my subscription to recur, and it provides absolutely zero value to me that they did so. If they hid text somewhere for this, it is a clear and obvious example that the industry desperately needs onerous rules and restrictions prohibiting this, such as a customer yearly opt in and double-positive recurrence opt-in (rather than some hidden subtext that oh by the way we have your credit card so have fun cancelling sucker).

2) Hide methods of cancelling said subscription behind as many layers as possibly: It is shocking how easy it is to sign up for many things versus what is needed to cancel the same (as if charges on my credit card are less of an issue for me than the possibility that online access might be temporarily delayed). In Consumer Reports case, you could not email or fill out an online form -- the single and only option was a limited hours telephone number that required lengthy wait times. They know that many people will avoid the time sucking tax. Xbox Live had a particularly hilarious system where not only did you have to phone a call center and sit on hold, your cancellation had to be immediate (despite the fact that all payments were for future services, when you disabled recurrence they immediately downgraded your account) which of course is meant to delay people from jumping through the hoops until the last day when they might forget or give up and boom, another month.

While much noise was raised about Apple shaking down the subscription industry on the iOS platform, they will have to simply eat that cost because people like me have absolutely zero trust in the industry, and I greatly respect how Apple not only constantly reminds me of subscriptions, they make cancelling any of them the easiest thing in the world, so those services that want subscribers need to actually work

Recurrence is a tax on the world. It is inefficient and obnoxious companies that turn their business model towards unsavoury tactics rather than actually providing value that keeps people from wanting to be their customer.

1) By reading this post you agree to subscribe to future posts at the economical cost of $14.49 per month, billed bi-annually.

To your points:

1. While there are dishonest sites, I'm sure you also have numerous recurring subscriptions with a whole bunch of services where that's exactly what you agreed to, and what you want. Your internet provider, your mobile phone provider, your cable company, etc. I realize you may have had an issue with one company, but that doesn't mean the subscription model can't be used in an honest, customer-focused way. I have subscriptions with utilities, online content services (Spotify), and even a podcast network and for 90% of situations, it works fine and there's no trick or scam.

2. I totally agree that companies should make it easy to cancel, and it's despicable if they don't. It's also generally bad business practice because it hurts your relationship with them. But again, there's lots of companies with recurring subscription models that handle this right: just look at Netflix. Not only will they cancel easily, they'll also put your account on hold if you're going on vacation, etc.

You don't need to play their game. Send them a letter cancelling and revoking the authorization to charge your credit card. When they charge you anyway, send a letter to your bank informing them of the unauthorized charge. You'll get the chargeback. Send another letter to the AG of your state, too, complaining about the credit card fraud.

There are whole industries built around billing people for services they don't actually want, and this is just an example of it. I completely understand that it is normal across the industry, but it is incredibly scummy and puts deception and evasion ahead of providing actual value.

You're saying this is a service you don't want, but it's unclear how the provider is supposed to guess that. You signed up for it, you haven't cancelled it, and now they're going to provide it and then seek payment for it. The failure to cancel is on you, not the provider. Even if you cancelled the card outright, in most good ToS, they'd still have a right to bill you directly for the service if you don't cancel.

Let's say I run a gym and you're a signed up member. If a new month starts, and you've got an active account, even if your card expires I'm still going to bill you for the month unless you come and tell me you want to cancel your membership. The two things are unrelated.

Yes, I will complain. Intrinsic in the notion that I'm not bothering to update billing information after numerous updates telling me that it will cancel if I don't is the conclusion that the service, and billings, will cease. Credit cards have expiry dates for a reason, and ultimately it is fraud that they effectively manufactured my info for me.

You're hoping that the natural expiration cycle of your physical card - not the account - will impact a transaction that doesn't actually involve that physical implement. I can see where you're coming from, but I don't know that you really have a right to expect the expiration of your physical card to impact an existing business relationship with a third party that involves recurring charges (that you have likely agreed to pay regardless of your card's validity, or credit limit.)

I've worked on these kinds of renewal systems for large sites and generally, users don't mind them because the reverse case - where 4% of the users lose access each month because their physical card expired - sucks for both the site and the end users who don't expect access to be cut off.

If you want to use credit card changes as a shortcut for properly cancelling your subscription, you probably need to actually change the card number.

I will refine your hypothesis a little.

a) It isn't companies with the highest margin who win, but the ones which are able and willing to devote the greatest percentage of revenue to performance driven advertising. If the company has a slim margin to begin with, they will never even be able to start in the "race."

b) Education is a better example than car insurance. While there are geographic variances to what ads are shown, nearly all of the ads will be for for-profit schools (many of whom spend more on acquiring a single enrollment than on educating them.) Additionally, the organic results will either be for performance marketers driving leads right back to these schools, or it will be objective journalism plastered with banner ads and contextual text ads for for-profit schools (Forbes, Washington Post (who owns and receives half of their revenue from Kaplan but thats another story..))

c) Some markets get so abusive Google puts very heavy handed and aggressive rules in place. For example, if you read the FTC's news releases regularly a common scam is someone operating as a payday loan operator simply emptying bank accounts (not sure why this isn't an FBI/DOJ issue.) Its easy to bid high when your end result is the entire content's of your customer bank account. If you are operating in a market and your competitors start doing this stuff -- your fucked. Google can take years to catch up.

d) The "quality score" means little. It largely has to do with how a site looks, not what it actually does. Misleading ads tend promise more and thus get better click through rate which means they are even more likely to show up higher, not less. At the end of the day, Google will let low quality site run, they just charge them a little more.

e) The most disturbing thing to me is how Google is trying to wash away organic altogether. The original article points this out. Aaron Wall (SEOBook) has been pointing this out for year. Ben Edelman has been doing an excellent job of busting Google's bullshit PR (even though he may be paid by Microsoft for consulting its academic level work of the highest quality.) Google wants to keep serving that organic content to users, but block them from flowing through to the original source. Google Images' recent changes reflect this exactly.

f) All of this makes me think Google's original business model is in serious crisis. Ironically, Google is doing the same revenue escalation to their own users as their advertisers are forced to employ. If desktop revenue is dropping fast, Google has to crank up the "aggressive" meter to milk out more revenue from a diminishing ad clicker user base.

g) This is great news for the next search companies. Google is dependent on their ad revenue to fuel their behemoth enterprise. They are sacrificing their own product in an escalating fashion. (If you use AdBlockPlus you probably never even noticed..)

Unfortunately all these attempts were killed, because it is so much more valuable to sell an ad to a shady comparison site, than to run a transparent one.

I also don't think their business model is in any sort of crisis. Google gives people great search results consistently. It takes crazy logic to see good Google search results consistently everyday, and then not trust Google's results for a high value query.

What makes Google's marketplace uniquely challenging is the following: In the physical world, you can go to a store and make a pretty good judgement that the product being sold is real, it is high quality, the cashiers are happy, the (real) customers tell you they have shopped there for years.

Online, everything becomes about human perception of data. It is easier to represent quality when there is none. It is easier to sell a customer one thing and extract a great deal of additional value from them. When that happens, the winner is the one with the most visibility (I think Americans can greatly relate this to US elections.)

Are user reviews and testimonials the solution? Amazon is loaded with fake ones. Many companies exist already to help small business owners "tweak" Yelp reviews. Even when you put a real face behind a review, it can be gamed. A static review will always be quite different than judging something with all of your senses in person.

Load your own site or app up with ads and see what happens to user retention. Prediction based on my own experience: it tanks.

Google is in crises because their behavior tells me their revenue would drop immensely if the result pages had as many ads on them as they did 6 years ago. Google is in crises because the ad revenue is their life blood. They have a massive organization which must have this revenue stream to remain in business.

There is a segment of the market that will click on and respond to bad ads over and over and over again. There is another segment that is more perceptive. There is a whole other segment that doesn't even know any of this has happened because they have used ad blockers for years!

Google might be really good at avoiding the tipping point. I assume that right now they are balancing the flight to mobile with the drop in user retention from "too many ads." Or perhaps I am overestimating the impact of ads, and the real game for Google is making customers think they are just clicking on content.

Some verticals pillage their customers. Many verticals do not, the arms race has not happened for them yet. That is a good thing for Google, and an even better thing for Google's users.

Google can control what they do, however with every step they take in the "show more ads" direction, the opportunity for someone else gets that much bigger.

As pointed out in the original article, in some result pages 90% of above the fold is ads. Would you watch Game of Thrones if it was 54 minutes of ads and 6 minutes of content? I wouldn't.

With the new trend of usage of laptops increasing, a little shift of certain LCD screen on laptops changes the contrast of the advert background (very light pink) so much that it is impossible to determine that the first few links are ads.

I think people will continue clicking on ad links without realizing that they are doing so.

at my zombie startup, "Vadlue" (http://www.vadlue.com/node/en/contact), we tried to strip ads out of search results and them searchable, browseable etc. maybe future lies in not making ads look more annoying by mixing them inside content, but making 'some content' behave like ads.

And so online advertising re-invents traditional media's "product placement".

To paraphrase a comment upthread "Would you watch Game of Thrones, if The Freys had used Smith & Wesson branded crossbows to massacre the Starks? Or if The Lanisters only used AT&T brand carrier pigeons?"

Also many users do not know they are seeing ads, imagine if FTC cracked down on shady labeling? People do not like seeing that many ads and Google's revenue and reputation would go down the drain.

>> It takes crazy logic to see good Google search results consistently everyday, and then not trust Google's results for a high value query.

Why, maybe Google manipulates only the high earning keywords? There is no money on "Snowden leaks"

Business owners should band and push another search engine. Google has some guy name Mat Cutts offering "do this and that and your ranking will improve from [update a-z]" but apparently these are lies by Google, Google is actively taking away traffic to send it to ads. Some site that paid $50 for a click, will have to get that money back, and you, the user, will pay for it.

http://www.ftc.gov/opa/2013/06/searchengine.shtm

According to the DoE, 4-year for-profit colleges spend 2.9x more on administrative expenses (albeit including student services and academic support) than instruction. Unfortunately I can't find a better breakdown of this category, but I'd assume this is mostly leadgen.

Table A-43-2

Based upon personal experience (worked at a company that spent 8 digits on Google Ads on an annualized basis, profitably and successfully over a long period of time) I can assure you that Quality Score does NOT mean very little.

It is also a falsehood to state that Quality Score is entirely about "how a site looks". Google actually even measures how quickly users come back to Google after clicking on an Ad - and if users come back too often in too short a period of time (signal for poor quality ad landing page), the advertiser's Quality Score can be penalized (to what extent, exactly, is of course unclear).

This. I sometimes feel like I'm using a different internet than most users.

Does economy of scale not apply to insurance?

Do you have any ideas on how to better compete without the same type of margins?

That's a rather terrible example: a restaurant that had over priced wine and cheap ingredients would fail miserably in short order, regardless of how many Google ad auctions they won.

If you really want the cheapest car insurance, you need to find a company that doesn't advertise, and is non-profit

I doubt those two traits have any measurable correlation with providing the cheapest car insurance, or any other service or business, just as it's possible that a company with rampant ads on every medium and high margins provides the best value.

unless if there's so many restraurants, and the quality of wine is so subjective that most customers cannot really differentiate it. Then exposure is really the only thing a (time poor) customer can rely on, and thus, by massively spending on advertising, and not on ingredients, they can convert more paying customers than the small, but quality restaurant who nobody knows exists.

I want some way to have word of mouth recommendations work over the net, without it being game-able. perhaps a system where you must pay (a very small token amount) to review?

That would make it even more of a money arms race. Ultimately you want something like personal recommendations where people attach their own reputations to their recommendations; google is trying to push something on these lines with google+.

Being able to create fake reviews and fake recommendations is the same problem as spam in email (cept spam is easier to filter out). If sending email costs the sender some token amount (say, 1cent per 1000 email), it would probably stop majority of spam, because it just became cost-ineffective. That was why i was suggesting paying token amount for writing reviews - tho i have no doubts there are flaws with that idea….

Have you ever travelled? There are restaurants like that surrounding every airport, convention center, etc. because they know people will value convenience, speed of service, ability to handle large parties, etc. more than great food. The situation is similar near large office parks — it may be mediocre but it's the closest choice! — areas with lots of tourists or transient populations (how many college town date-night restaurants are great?), etc.

Actually, it's even simpler: Applebee's does not appear to be disappearing.

Now, you can be against Google ever making a UI change, or even against Google showing different results for "Italian restaurants" in New York and Chicago, but this doesn't really have anything to do with "organic" search results.

The scariest part of this is that, if you sell something using the internet, regardless of whether or not you see yourself as a “local" business - or think you’re competing with Google - Google sees you as competition. Searching for “Camera" or “Buy a Dress Shirt" gets you a nearly identical split of screen real estate as that of “local" searches.

You are very much being de-incentivized from investing in a page that is outside of google's universe, because you're only shot there is pure organic, which matters less and less.

[1] http://i.imgur.com/Bf74Dd7.png

User-agent: * Disallow: /

I have yet to experience the Google carousel for myself, but aren't these results all coming from Zagat or G+? So these may not be paid (or pay-for-prominence) results, but they're not the results of a Web search; instead they're the results of a search of a proprietary, Google-controlled database. That's not just giving local results more prominence, that's moving a large part of the page to a whole different kind of search: from Web-crawling to searching a Google-owned and -"curated" database. And in many ways that's more important than whether the results are 'organic' in the narrow sense of 'not play-to-play'.

It's fine to argue about whether Google should be showing anything but ten blue links (maps, images, calculator results...) But refusing to acknowledge the legitimacy of searching anything but web pages is just absurd.

It's not that organic search results are changing, but their VIEWABILITY, and therefore relative importance, is drastically altered

At Blekko we built a search engine (crawler, extractor, ranker, fetcher, the whole stack) and let people know exactly how we rank things, for users who created an account and logged in they could turn off everything except organic results. We have been moderately successful (we were the first to use a 'masked results' option [1] to show how much better our results were than Google's in contested searches) but there isn't enough outrage about that yet to build a business yet.

Most of our business (people who pay us money) comes from folks who either want to figure out how to game Google's results, or want some organic results to create a 'search like experience' much as Google does. When you think about it with only 7% of the page dedicated to organic results that is like 2 or maybe 3 actual search results and the rest of the page is a carefully crafted advertising vehicle. Sort of like a free 'newspaper' which has one article of editorial content and the rest are all ads. You can serve that market quite effectively with a relatively small index (a couple of billion URLs).

If you look at Google's financial performance over the years you can see how this evolution has affected their bottom line. Today you see it in declining revenue per click sorts of things. It feels to me as it did when banner ads went from this massive cash cow into something less useful.

One thing is true though, the world is changing yet again.

[1] http://techcrunch.com/2011/06/21/be-the-mark-in-blekkos-3-en...

I'm glad that folks like Blekko are trying to improve the search landscape but do you have any figures to back up the declining revenue per click?

I feel as though Google are becoming more and more focussed on profits (evidenced by their (not provided) data hypocrisy, role out of "enhanced campaigns, etc.) but their profits seem to grow[1] and they seem to focus on margins over advertiser or user experience with increasing focus.

I'm glad that folks like yourselves are in the market as I say but until someone develops better algos and better results (for the majority of queries) it's unlikely that folks will get angry enough to move or see a viable alternative.

In spite of these changes I haven't really seen a decline in Google's market share[2] drop consistently or significantly so I don't really see a change coming any time soon.

References: [1] http://www.blogcdn.com/www.engadget.com/media/2013/01/1-22-2...

[2] http://searchenginewatch.com/article/2275863/Google-Bing-Bot... (June, '13) "Google has the identical search share of 66.7 percent when comparing May 2012 and May 2013."

vs.

http://searchenginewatch.com/article/2269591/Googles-Search-... (May, '13)

  > I'm glad that folks like yourselves are in the 
  > market as I say but until someone develops better
  > algos and better results (for the majority of 
  > queries) it's unlikely that folks will get 
  > angry enough to move or see a viable alternative.

Now on the one hand I should be outraged Microsoft ripped us off [4], but I'm not. Its been interesting to see how much time and money they have spent making our strategy into a 'full size' search engine, and realizing that we were not going to be able to raise close to a billion dollars (which is what they will have easily spent by now) to take it to market against Google. The Bing challenge, the editors picks, the whole thing taken right from our playbook. And while I believe they are having the best success they ever have against Google to date (its a good strategy), there is another 3 years for it to run before consumers think that they are equal services. And there is a whole lot of dirty politics between here and there.

As for market share, its a curious thing. You can earn it or you can force it. Look for more and more "search defense" going on, for example non-Google searches getting harder on Android, or in Chrome. Conversely non-Microsoft searches in Windows products. With people like Firefox getting played in the middle (try to search on Google on Firefox on the standard Linux Mint distribution for example :-) Gone are the days where simply having a better product will let you win that prize. Way too much money involved.

[1] http://www.marketplace.org/topics/tech/weak-ad-revenue-pulls...

[2] http://news.slashdot.org/story/12/07/24/1553214/the-decline-...

[3] http://www.forbes.com/sites/roberthof/2013/04/18/google-tops...

[4] It would have been polite of them to have at least offered to buy us out :-)

If the above is true, Google is dedicating way more real estate to organic than ever. It's just a different form of organic.

On the one hand, that does level the playing field, because little guy doesn't need huge website and SEO to compete against big guy. But fundamentally, everyone is now handing over the control of their data to Google. Let's say you change your business hours (and as a potential customer, I do love Google's display of today's business hours) or phone number, how quickly does Google update all that? Probably very quickly, but it's now out of your hands.

And as that comment points out, positioning within those "local" results may be determined by ads (and they are not divided into paid and organic sections)--but we don't always know. Google is definitely trying to blur the line by moving away from the be basic search results listing.

I wanted to add that the problem of organic search results is always going to exist for any sort of business (as in both examples of the original article)--as opposed to information. Let's say I'm searching instead for a performance of Beethoven's 9th. Hopefully, people around the internet are linking to good performances, and Google will show me the "best" one. And it will have ads for ringtones and buying CDs of Beethoven's 9th, which is to be expected. But I can be fairly confident the organic part of the results are meaningful. With local search or business search, there is no such thing--Google is not indexing the real world, just what people put on the web. And when there is money on the line, what people put on the web is designed to make them money (SEO, ads, paid reviews, etc.).

[1] akharris: https://news.ycombinator.com/item?id=5972516

On the contrary, its completely in your hands: http://www.google.com/business/placesforbusiness/

The point of contention is that its a Google controlled affair instead of going through an external standard.

Now while it's nice that people can in theory update their business information using that site (that I myself had never heard of before either), but the reality of the matter is that, even in the 21st century, the modal local business owner just isn't very good with computers and the Internet.

The difference?

The website they get from some (probably also local) webdev company comes with customer support. Google has none.

Wrong or outdated information? One phone-call with a human being and it's fixed.

Try that with Google. They'll be talking to a robot and filling out form after form after form, in no-time. Believe it or not, but "figuring out this Google stuff" probably takes them at least a whole afternoon.

First, the author factored the screen space taken up by the google search bar into his calculations. Really? You want to press the back button each time you want a new search? And don't get me started on the fact that the author factored in the goddamn page margins as part of his percentage.

Secondly: the 7% of the page that is taken up by a map? Yeah, those are also organic results. In fact, they are probably more helpful to most people than the author's version of organic results (just links to websites).

Thirdly: after you scroll down (takes about 0.1 seconds) the percentages change significantly, as the entire box of sponsored results is no longer there.

So. If we eliminate the search bar and the margins, and we include the map in our percentage, and we scroll down a tad so that the sponsored ads disappear, you're looking at a page that is about 70% organic results.

FINALLY AND MOST IMPORTANTLY. Google did not become popular because it showed the highest PERCENTAGE of organic search results per page. It won the search wars because IT'S RESULTS ARE MORE RELEVANT THAN OTHER SEARCH ENGINES. Percentages DO NOT affect the relevance of google's results. It's all about relevance.

It's been about a month now, and I find the results terrible. I want to love it, but the first 10 results are pretty much never what I want. Sadly, Google still performs better on this front for me. If anyone has tips to make it better, I'd be very happy to hear them.

How long have you been using google? Probably long time. During this time you (unconsiously) learned how to ask questions "google way", now you need to learn to quack :p

I also found that DDG sometimes (not very often) does not find what I am looking for. Mostly it is because I know nothing of the subject ie. using a description of a "thing" I am looking for instead of its name, etc. - theese are the things google is better at.

Regardless, if you find such query than please submit it to ddg, they are very responsive and results do get better.

I understand this, but most of us are looking for an alternative to Google and not a completely different search engine.

I assume I could configure Adblock Plus to hide the Google ads, but at least for me and what I normally search for they're not really an issue.

Google Custom Search: http://i.imgur.com/PLZ79Vk.png?1

Startpage: http://i.imgur.com/5ui5LQc.png

There are barely any ads.

Also, after a year of non-stop use, i can tell you right now that Bing predicts what I'm looking for far better than Google ever did.

For example, if I search for "binary tree" I probably want Wikipedia followed maybe by StackExchange. Google does that, while DDG sends me deep into the netherworld.

All you need to do on a website with a search box is to right-click on that search box -> Add Keyword for this Search... and then add a shortcut.

The cool thing about it is that it is extensible. You don't need to settle for the fixed "bangs" that DDG provides. For example, I have a "dex" shortcut for searching an online dictionary of my native language.

Also, it takes less time to add a shortcut of interest than it is to read the DDG documentation page to find out the right bang for you.

If you care about privacy, then this is also a no-brainer, since you don't go through a third-party server anymore.

1. type ["binary tree" !w] in the url bar

than it is to:

1. type [wikipedia] in the url bar (I don't even know if it's a com or org, but DDG would be fine with just [wikipedia]. If I knew whether it was com or org, then I could skip the next step.)

2. click the wikipedia link in the now displayed DDG results

3. find the field to enter the site-specific search terms (for many -- but not all -- it's in the upper right corner)

4. type ["binary tree"] in the site-specific search box

1. Start to type wikipedia 2. Hit tab as soon as wikipedia.org is the first choice 3. Start typing the query and note that the suggestions are coming from Wikipedia, not google

Significantly faster than waiting several seconds for DDG to redirect

For me:

1) Wikipedia (in featured box), 2,3) Some Exchange/Notes migration software called "Binary Tree" that owns binarytree.com, 4) cslibrary.stanford.edu on binary trees w/exercises, 5) "Binary Trees in C++" from cprogramming.com, 6) "Binary Tree" at dictionary.reference.com, and 7) "Binary Tree" on mathworld.wolfram.com

...above the fold.

1) en.wikipedia.org/wiki/Binary_tree

2) binarytree.com - Binary Tree: Exchange Migration and Notes Migration Software

I find as more subject specific sites have grown, that generic search like ddg and google aren't as important to me.

I do more searches on sites like stackoverflow, yelp, maps.google.com, tripadivsor, etc. than ddg.

https://startpage.com/

It's basically google, but without the tracking. (Or so they say.)

And frankly, those search results are useless. If I am searching for auto mechanic, it is highly unlikely I am doing so to know the definition of an auto mechanic. Google may be creepy in its tracking, but at least by tracking me they can point me to a mechanic in my area.

It's rather simple.

When you type terms into the search box, just imagine the words, "Please show me advertisements for " in front of it.

It's like full circle back to Yahoo.

Maybe this is a result of listening to the data too much, and not depending on the greater vision enough. Or maybe it's the future and you're gonna love it.

I'm not _necessarily_ saying this is bad or unwanted, but it's a vast philosophical and product shift for Google. And it has implications for the trustworthiness and long-term usefulness of Google.

I do not welcome a future where Google invalidates the business models of businesses like Yelp by delivering their content in such a way that makes going to Yelp unnecessary. I don't want all of my websites to be subsumed by a corporate behemoth, so I get short term better products, but long term stifled innovation.

I find this SERP to be vastly more useful than how it used to be. Restaurant owners should too, I no longer will skip visiting your place because your website sucks or is not mobile friendly.

Initial search: http://files.calebegg.com/sushi-search.png

After clicking an item: http://files.calebegg.com/sushi-result.png

At least use the damn feature before you join in bashing it.

From Google's point of view, the question is: If you search for "sushi" are you more likely to mean "nearby sushi restaurants" or "a website that lists nearby restaurants"? Which results get you closest to your goal the fastest? Depending on your answer showing local results is absolutely the right thing to do for the user.

Not another website listing them, not dots on a map with phone and address (although that is also useful), I want the actual sites, so I can compare them, see their menus and whatever else they have decided to share with visitors.

I don't think I ever got a result like that from Google when searching for <type of restaurant> + <place name>, it's always been just a few restaurants mixed with way more restaurant-listing-sites.

But currently Google sort of tries to hide them, it takes you a few clicks before you get to a link to a restaurant site. So no quickly opening 5 of them in background tabs, for instance.

I find that quite funny, to be honest.

In chrome, with no login, I get a different set of results, including some local restaurants. Curious.

I think its almost taken for granted that all search is interactive. If the first try is not precise, refine the query and try again. Its not correct to compare the Google of today with the "organic" search engines of a decade ago. The world, and the web have changed tremendously since. It can be argued that Google is trying to help the consumer by showing multi-faceted search for vague queries (which can potentially narrow search requirements) or provide cues about forming the next query -- and if a company can make money doing that, whats wrong in it?

That said, the implication with Google results is that the higher up on the page the link is, the more relevant that link is to a user's search. In my experience, very few people ever get to the second page of Google Search results. I'm all but certain that if we looked at heatmaps of clicks across all Google search pages, we would see this quite clearly. It's the entire reason that SEO has been an industry at all.

It gets more troubling with products like Google Cars (3)(4). It is literally another layer of PPC results specific to cars. I've got to think that other car retailers (including manufacturers) can't be too happy about this.

(1) http://iampaddy.com/lifebelow600/ (2) http://blog.kissmetrics.com/why-the-fold-is-a-myth/ (3) http://www.drivingsales.com/blogs/InvestigativeReporting/201... (4) http://searchengineland.com/google-tests-new-car-leads-produ...

I actually don't mind this on mobile (but I also generally use something else for finding restaurants) since the majority of restaurant websites are basically impossible to navigate on mobile and all I really need to know is 1) where are they 2) what's their phone number. How organic that database is, I'm not sure, but it doesn't seem to be showing specifically paid placements.

A paid result can be a useful thing and might even be better than an organic result in many cases, but both have their pitfalls and it's not surprising for them to keep going this route more and more.

Google is a company, not a nonprofit. They are in it for the money. Why is anyone surprised?

Maybe I'm the minority, but if what I'm looking for isn't one of the first few results, I modify my query. Google is showing the stuff that makes it money, and the organic stuff we're most likely to need.

And at the end of the day, what we're talking about is having to scroll down on a page. Faux blog outrage.

You would think that bing would be trying to win the popularity contest by actually providing more relevant searches in the short term and then expanding ads in the future...

I guess what I'm saying is that pure organic search isn't necessarily the optimal way for a user to find the information they're looking for.

However, there's an important point here to be made, which has been said in numerous antitrust arguments against Google, and that is, "what if the product in No. 1, Google's, is in fact preferred by consumers and therefore is, to use language of the tutorspree authors 'organic'?".

It's an interesting question. I find much of the article interesting and of course the screen space dedicated to search is a hot topic, and Google's minimal style still remains in my favor, I just wanted to briefly object to the claim that somehow showing a non-ad Zagat page is "0%" (in the author's numbers) organic search.

> 7% is all that’s left for the entrepreneurs and restaurantuers who believed Google over the years when they were told that good business with well structured pages would be able to get in front of potential customers searching the internet.

I'm really curious how the author would propose to fix this. Even with all of the other stuff cut away, there are only still 10 blue links to give.

Also the ads on the right side are delineated very liberally - one could argue that the actual area is half as wide.

If you think the ratio is off, then write an article about it... but this is hardly "killing organic search".

Google penalizes sites with above-the-fold ads. It doesn't take much at all to earn the penalty. I think that's one reason why people are miffed with Google's excessive ATF advertising on their SERPs.

You can turn on verbatim mode on the search results page. Click on the Search Tools menu, then look in the All Results menu for the Verbatim option.

The verbatim option tells Google to use all your search terms, as-is, and (at least at present) the search results are much less cluttered.

To always use verbatim mode, remove the search terms from the URL in the address bar (leaving intact the other parameters that turn on verbatim mode) and create a bookmark.

Don't tell Google about this feature... I depend on it!

Nowadays you're supposed to buy ads to stay competitive. Google grew so much that it's now a monopolist behemoth, and that's why they're swimming in money. But because they keep all the nerds in love with them, they manage to get zero flak for it.

Even with the shrinking organic screen space, the first three results still capture the vast majority of the clicks, so results 4-10 receive fewer clicks regardless of the amount of space given to organic results.

I'd like to see a study where organic results 1-10 all appear on the page above the fold, giving users the ability to see all the results without having to scroll and measure click through rates at that time.

1. http://searchenginewatch.com/article/2200730/Organic-vs.-Pai...

How:

1) Google has steadily moved away from optimally presenting search results which they possess.

2) In being this connecting point, ads increasingly have taken over the space where organic results are published, much like ad space is sold first in a newspaper and articles are squeezed in around it.

Everything changes, or does it? Content is king, except finding content is a new king.

3) Google generally has the content we seek - in the form of search results. The results are squeezed around ads. This creates a cognitive cost to separate ads from results, instead of just finding what we need, which was the original promise of Google. This implies, at some point we're giving into reading the ads like they were the results?

Users have only ever been interested in search results from a search engine, not ads. In a way, good search results was a promise shared by AltaVista and Google. Being able to easily access them is what is changing.

Last year I was irritated enough to build my own search interface, focused on presenting results how I wanted to see them.

4) The cognitive cost had added up to be too much to scan Google constantly to find the one weather, or movie link I have looked at for the last 10 years, and becoming increasingly harder and harder to find.

My thoughts come back to the announced shuttering of AltaVista, in wondering whether we really ended up ahead. Google certainly has done things no one else has, getting search so good, and now finding what we want is a little more work.

5) Google is a business, and need a financial engine. As much as advertising currently seems to pay for the internet, I wonder if a day is coming where we're more willing to put our money where our mouth is.

It may take some time, for the majority of internet users to have 10+ years of online experience of doing the same searches and finding them harder to find.

What to do? I feel like brushing up my custom search interface and use it a little more.

My guess would be that some sort of rating site that better answers questions like "what car to buy?" should benefit from Google removing organic results from searches like that.

Example: http://imgur.com/AJtOPkf (this is zoomed down to 65% to capture all of it....)

They need to remember that there are alternatives just a click away, and serving too many ads degrades the entire internet.

People can use Yelp to find Italian restaurants in San Francisco but try typing the same in Wikipedia.

Is there any personalized search engine which lets you choose Yelp/Zagat/Tripadvisor when you type in that keyword? and maybe pull down Google map so Google doesnt feel left out ;)

I guess adblock and not living in US has some advantages.

Good luck finding that business model, and being able to reap the profits from it.

Now I find myself longing for a new company and a new disruption of what is essentially Google monopoly.

http://ppcblog.com/fbf0fa-now-you-see-it

http://blumenthals.com/blog/2012/01/31/is-google-intentional...

A few years ago this was the style http://www.ismoip.com/blog/wp-content/uploads/2011/01/Screen...

Before that http://cdn.userstyles.org/style_screenshot_thumbnails/58617_...

Now http://i.imgur.com/Wmdd0.png

Some of those keywords are worth tens of dollars per click so no wonder,the colors have been A/B optmized to get the most clicks from people not knowing they are ads.

Comparison:

Google: #fff7ec Bing: #f9fcf7 Yahoo: #fafaff DuckDuckGo: (255,212,0,0.18)

Not to say that Bing, Yahoo etc. are much better but I expect more from the "Do No Evil" Google rather than increasing the next quarter's earnings instead of targeting older people and people with bad monitors and hurting people who did a lot of good work to come in the first few in organic results but don't and/or can't pay Google for expensive ads. Also, Bing and the rest continue to mostly lose money and they can't afford to separate ads while the big guy continues to reduce the difference between ads and search results.

"Study:Contrast sensitivity gradually decreases with age" http://www.eyeworld.org/article.php?sid=818&strict=0&morphol...

The study the FTC cites[2] indicates that fully half of individuals didn't even realize the page contained ads.

[1] http://www.ftc.gov/opa/2013/06/searchengine.shtm

[2] http://www.seobook.com/consumer-ad-awareness-search-results