I worked at a company that heavily used the maps API in 2018. The 1,400% price hike really was an eye opener. It wasn't just the money involved, which was large, but the fact that it came so suddenly and with so much magnitude.
Google's choice to bump these prices made me realize that they could do the same thing with their compute cloud pricing or other APIs, again with little notice. That, in turn, led to the realization that we needed to remove our reliance on Google's entire stack, and quickly, or we'd be in a terrible bargaining position later.
This one choice by the maps team likely cost Google significantly in the medium to long term, if any other companies realized the same things we did.
Google's fundamental relationship with users is statistical, and I think it shows. They're very good at optimizing metrics; their search, for example has remained the leader 20+ years on. In that world, if you lose a few users or give a bad experience, well, there are plenty more where those came from, and the ones who leave will often come back.
I think they mistakenly bring that attitude to their consumer businesses, though. The stories of bad customer experiences are legion. Like you, I've learned that I can't really trust them. Which is true of all companies, of course, but it's different with Google. Amazon is even more rapacious, but I trust them to be long-term greedy, to not shoot themselves in the foot with some short-sighted action. For all their flaws, they understand that customer relationships have long-term value.
This is true per "equivalent" unit of generic computing, but some older instance types have gone up in price (presumably to encourage people to migrate their stuff onto newer hardware).
This is completely reasonable - but a pain when you have a box that works perfectly and you have to upgrade the software to cope with a new hypervisor driver/layer.
One of the things I really like about Hetzner is that they auction off the older dedicated servers for cheaper. It should make economic sense to do the right thing.
And maybe all their metrics say that in the grand scheme of things, shooting some customers in the foot is the better strategy compared to not shooting them in the foot. Sometimes via deliberate price hikes, sometimes via shoddy support channels and discontinued products.
The big questions are whether their metrics factor in customers who avoid google's products to begin with, knowing they might end up getting shot in the foot, or whether they have a moral obligation to not shoot their customers even if they can make more money that way.
enterprise and consumer markets are so different. The same management leading on both fronts is unheard of and very unlikely at larger orgs.
At Google's scale they are optimized and oriented for the consumer market with their horrendous pricing strategies, poor customer service, negligent backward compatibility and pseudo-monopolistic products and offerings.
As a consumer, I can live with the lack of trust. From a business perspective I cannot live with that.
I mean, at a minimum, their habit of using customer data from their platform businesses to inform development of their own competing products is pretty sketchy.
That's SOP in retail though. Walmart/Target/Home Depot/Best Buy/Whole Wallet, knows exactly what sells and use that to inform their own brands. And they kind of need to be tracking and analyzing those things in order to do their job.
Any store that uses a rewards, loyalty card or phone number when you checkout does the same. I'm not sure why Amazon is the bad guy when grocery stores have been distributing private brands of mainstream products and shopper loyalty has been used to optimize revenue streams. Amazon does it on a bigger scale, not sure the scale makes it right or wrong though.
Your theory is that if other large companies do something, it must be morally acceptable?
Even if that's true, which I'm not persuaded of, Amazon's enormous market power makes a big difference. As a consumer, I don't want Amazon to "optimize revenue streams" when that reduces the reward to market participants to be innovative and deliver high quality.
> Your theory is that if other large companies do something, it must be morally acceptable
You left something out, "as long as it doesn't effect me (yet), and large companies do it, it must be morally acceptable." I don't know why we defend them. Without Google we still go on. Without us, or support for their practices, they dramatically change or they disappear.
One pretty obvious way is how they treat workers. Despite being a customer for more than 20 years, this year I canceled my Amazon Prime membership and now avoid using them. If you're unfamiliar, you can find an ocean of material by googling "Amazon labor exploitation".
> Google's choice to bump these prices made me realize that they could do the same thing with their compute cloud pricing or other APIs, again with little notice. That, in turn, led to the realization that we needed to remove our reliance on Google's entire stack, and quickly, or we'd be in a terrible bargaining position later.
Yes. You are right. The whole cloud deal is a try to rule out on premise infrastructure and once it is gone and everyone is dependent on it (due to lost of knowledge, and for cloud providers ideally when hardware vendors are no longer selling hardware to non cloud entities) harvest the dependent businesses left at their mercy and charge dearly (including for all the years before that). It is actually not even a complicated plan but it is staggering how many people are putting cloth over their eyes.
These two comments make me see SaaS more like paying rent for housing:
If we become fully dependent on a service (shelter) and then the provider (landlord) jacks up the price, we may be quickly forced to switch services (homes) or be out of business (homeless).
In many places landlords cannot suddenly jack up the price, but can only increase it in small amounts year by year. One could similarly imagine rent controls on digital services, where sudden price hikes are simply not allowed, and terminating the rental agreement one-sided is similarly restricted. The practical downsides of that kind of legislation would probably end up quite similar to rent controls.
Yes! I've found that I sometimes will keep a subscription, even if I'm not currently using it, because I've been locked in at the old price, and can feel really annoyed when they increase the price by a significant amount or percentage and force me to adopt it.
About rent control, yup, it may have the ups and downs of real estate rent control :-)
I'm also wondering, with physical real estate, there is a tangible limit to how many people can rent the space, whereas SaaS seems to make up arbitrary user limits. But maybe I'm wrong, and there is a limited number of users because each user has variable costs—in the housing analogy, utilities like water, space, wear & tear on the place, etc., and in the SaaS space, customer service, server space...I can't think of any more but maybe they're there...
Ha, good point, perhaps that'd be like paying your landlord for utilities like water or electricity usage, but could also be paying for every time you have guests visit the house or you go up and down the stairs.
unfortunately when we switch services(homes) all of our electrical devices(code) now need to be changed to work with the different way the new service(home) is wired. At this point the analogy breaks down.
Hmm. Maybe it's a spectrum, where some situations are like moving homes and others are like like moving cities (need to find a new dentist, doctor, etc) or moving countries (need to learn a new language, learn new laws, get new electrical outlet adapters, etc).
I think as the size of the application increases and the API you are switching out increases in importance to the functioning of that application then the analogy of moving from home goes up from moving apartments(my small side project), to moving cities (my small side project turned into a startup that has been running half a year but isn't especially profitable yet), to moving countries (my application that many large organizations use with all sorts of different functionality built in to support the needs of those organizations)
Well said. As it becomes more complex and intertwined, technologically and socially (employees, customers, etc), then it becomes harder to change. Thanks for helping to refine the analogy!
A long term contract pretty much guarantees that the company will lose the institutional knowledge to run things on premises.
Think about it, if the company has sysadmins that have been with the company for some years, have racked and are running most of their infrastructure, and all of a sudden they sign a long term contract with a cloud provider, why would they stay?
It’s possible that the company do not have all that infrastructure at all.
I also don’t see the importance of ‘knowledge to run things on premise’ for most people.
Technologies change very fast and if I can pay a reliable provider with a fixed price for the years that I intend to use it, why do I have to know how to use it?
If I can hire a gardener to take care of things for a reasonable price, I don’t care if I have any knowledge.
This is a danger, although the cloud is still just VMs and networks that still need configuring and patching. Agreed people who would run physical hardware would be gone.
I think there's a risk here but I'm not sure it goes as far as you suggest. There are still plenty of entities, particularly larger ones, that buy their own metal because at some scales it becomes far cheaper to run in a COLO (at least acc to near monthly articles posted on HN). I'd therefore put the risk that only big vendors will be able to buy hardware as low. That said, as ARM or custom chips gain market share within cloud providers, there's a risk that x86 or standard suppliers will struggle, but AMD/Intel see this too and surely one or both will adapt.
On skills I think there's a small risk of people not knowing how to manage/set up their own racks, but doing a passable job there isn't that hard. On general sys-admin stuff, cloud infrastructure seems to have parallels to hardware infrastructure, so it shouldn't be completely unfamiliar to set up a hardware router/network/linux server. There are definitely skills and details missing (remote admin tools for hardware maybe), but this doesn't seem like a big gap to bridge if $$$ is on the line?
If those entities would be any risk, they would already be bought.
Or they will be long gone due to google/aws/microsoft price dumping. In meanwhile, the openly accessible technology / know-how will be gone or hired by them and there will be no one left. Just look around, how many people you know that are capable of handling on-premise server? What about farm of servers without using any of google/aws/microsoft technology?
This just isn't something new.
It is just another case of "historia magistra vitae". We had this, 30-40 years back, we had centralized environment (have you heard about mainframes before?), due to huge price pumping the decentralized environment came in (personal computers, internet, on premise servers, storage, ...). IT needed almost 20 years to get out of the hooks of corporations and now we are all diving in into the latest "cool" thing - centralization, now backed up by advertising revenues and shady practices like not showing your company on google search.
Let me show you my point and where it is going, lets do a tiny experiment - use your router/firewall/whatever you are fond of and block the following ASNs:
Now try to surf the internet. Casual style. As nothing happened.
Baidu (China) and Yandex (Russia) will still work from search engines. Duckduckgo not (shame on you). As any other you know. "The internet" as you know it will stop to exist for you. You know what you have blocked with those ASNs? Only 3 companies. Google. Amazon. Microsoft.
For complete experience you can also throw in AS714,AS6185,AS2709 and AS63293,AS54115,AS32934 (Apple, Facebook), but those two are not really relevant.
Maybe this will be eye opening how much we have f* up the internet, something that should survive 3rd world war is now mostly in the hands of 3 companies. Now imagine they pull the switch.
> Maybe this will be eye opening how much we have f up the internet, something that should survive 3rd world war is now mostly in the hands of 3 companies. Now imagine they pull the switch.*
I see people frequently quoting the bit about how the Internet was designed to withstand a nuclear war. But that quote was about packet switching. I.e. layer 2/3. Not layer 7. Application layer is, as you nicely demonstrate, very centralized, and was never immune to nuclear war.
Admittedly I know few, but not zero. That said they’re all relatively older and have been in the business for a while. I used to manage a team with 4 racks, ~40 machines at a COLO. When we hired for new people, it was usually older folks who had the experience necessary. Maybe you’re right and I agree the concentration is disturbing.
My only response is that when I look at bigger orgs and places like Fortune 500 cos I still see/hear about home built server farms, so those people must be out there. I think maybe we get blinded in startup/SV land and only see people using cloud infrastructure but miss a huge, silent mass of people.
Depending on the nature of the business it can be cheaper to run own servers even for smaller companies. I see that in Norway quite a few places see that and the pool of qualified people to run them is not small.
I wonder whether there will emerge some design patterns in apps to enable easy switching between on-premises and cloud infra.
Thinking about it logically, the safest approach is always going to be build for on-premises infra first (or standard VPSs), then modify the code to support cloud services in addition, to get the added benefits of scaling at cheaper costs.
That way you can always switch back to on-premises if the cloud provider starts behaving terribly. Perhaps there is a performance hit, but at least you have an option.
Worth mentioning that there are some use cases that are only possible using cloud infra due to scaling constraints.
This is the approach I took building my SaaS. It’s more difficult initially, but I had an infra that could easily be moved. I was using my own version of backend queues etc, so modifying the code to switch to Amazon SQS would theoretically have been possible:
> It is actually not even a complicated plan but it is staggering how many people are putting cloth over their eyes.
These are changes that are happening on a timescale much larger than the average lifespan of a cloud-based product/system. I think people building things in the cloud know it, or at least feel it, and so just don't care about long-term - because in the long term they expect to be rich and/or doing something else. In the meantime, ooh cheap compute, billable as opex - let's build everything on it!
AWS is already incredibly profitable while continuing to lower prices over time. What makes you think they would choose to run this scheme and forever ruin their reputation?
The point is that nobody expected Google to just hike their price for mapping by 1400% overnight. If they can do it with one product, what’s stopping anyone else in a similar dominant market position doing the same?
This got downvoted for (I presume) triteness, but there's an important underlying point: Amazon's actions (or anyone else's) don't have to look rational from the outside to really mess up your day.
In a world where there are three players, culture and reputation are not worth much, if not only at the beginning. Maybe 5/10/15 years from now the software and competencies to manage bare metal will be only behind the doors of those three companies, and at that point it will be economically convenient for them to act as a cartel and increase prices.
I don't see that happening. As someone who manages bare metal, there's not that much to it besides using a pxe installer. Didn't even use that before we got 20 servers, would just configure IPMI physically the mount the ubuntu ISO via IPMI and manually install the OS at the office.
Ubuntu is actually heavily investing into MAAS (metal as a service) and so is Redhat with cobbler and foreman. So I'm guessing there's enough demand to justify their development (as every major cloud provider uses in house tools AFAIK)
That’s one of the arguments for multi vendor sourcing. The same price negotiation tactic occurs in other industries.
It’s why google buys non intel chips. Come negotiation time there’s a serious threat they can leave. And there’s a serious rfp out to other vendors for pricing. Bringing their intel chip costs down.
Still raw from a gaming platform we built for progress/matchup on App Engine (GAE) early on in 2011 for a popular racing matchup game. We chose App Engine due to flexibility, scaling and price.
One day Google increased prices and our costs went up x5-x10 on AppEngine [1][2]. We then had to move to AWS over the next few weeks early on in launch. It caused push back on other projects and was generally a bad time that wasn't necessary.
That isn't the way to build trust and we have used AWS and now Azure more than Google Cloud for this reason in the decade since.
Google has great tools, but you simply cannot do shock pricing on the cloud, it is the biggest fear of everyone that uses the cloud or has to recommend it. Google made us look bad by choosing them over AWS, later we also use Azure.
We like to build our systems for horizontal scaling, on cheaper machines, we weren't even doing much but it was suddenly out of budget and left everyone feeling burned by Google. I love Google and their tools, but they have to be careful with the rug pulling and knee capping people that choose large projects to develop on their products and platforms.
Don't make people look bad for choosing your product and especially don't surprise people with pricing or dropping products. If it was a better planned price increase we could have taken time to move off more slowly or even optimized to run on it, but since it was so sudden it was shocking. Since that moment they have lost lots of projects to Google Cloud that would have gone there.
We literally never expected it because AWS was a force, Azure was still new, and we thought Google would be price competitive to AWS and Azure while looking to gain customers trust as cloud products were something that didn't seem like you'd want to do any drastic changes to customers as they can be long term customers if things go well.
I means it's always good to reduce reliance on anything, so I won't argue against that, but I won't agree much with your reasoning.
The cost we were actually paying on the Google Maps API was hidden. The cost was mostly paid using user data and advertising. As a matter of fact, the API is still free on smartphone ;). They just realized that it wasn't worth it to get that data on the web and thus increased the cost to the true price. The fact that there was already some competition to Google Maps which was much more expensive tells a lot.
That's not something that's going to happen on the cloud though. You pay the real cost, they aren't monetizing it from a side channel (though they may profit a tiny bit on the side by lowering their infrastructure cost, it's not the goal and I wouldn't be surprised if they didn't used their own cloud offering that much). The profit come from the price, not the user data.
It wouldn't be a good idea for them to increased theses prices, as competition exist/can exist, and they would just hurt themselves on the long term.
Unexpected pricing changes have already happened to Google Cloud offerings in the past. Google completely redesigned their App Engine pricing scheme in 2011. In practice this meant a 65x price increase for one of my small services. [1] Among the changes was that the 2008-2011 pricing was based on CPU usage, while in 2011 they stopped charging for CPU usage and instead introduced a slew of other pricing metrics, including the classic VM instance idle time.
While the pricing changes frustrated me, they did not drive me away. What ended up having a much bigger impact was that Google just deprecated everything. Some parts of classic App Engine still run in legacy mode, but get no updates and instead you get constant notifications that you should rewrite your app to fit the new Google Cloud model.
> Google's choice to bump these prices made me realize that they could do the same thing with their compute cloud pricing
At least in the cloud computing space, Google is an also-ran thanks to Amazon. They were able to play this game with Maps because they didn't have a healthy competitor. If they screwed with pricing as a distant second place competitor they'd be toast.
> This one choice by the maps team likely cost Google significantly in the medium to long term, if any other companies realized the same things we did.
It definitely did. What they did with Maps is exactly why I would never, ever consider running anything critical with Google.
"This one choice by the maps team likely cost Google significantly in the medium to long term"
Reminiscent of the Google Reader thing. Google Reader was used by, in global terms, a tiny number of people. But today those same people get to make purchasing decisions at a large array of well funded companies, and they're still bitter about Reader shutting down.
Google was the dominant provider of maps I'm guessing by a large margin. GCP is not the dominant provider of cloud services (they're third or distant third).
Yes, that's my point. There's clearly competition in compute. That Mapbox raised prices after Google did points to maps not being terribly cut throat (I think Mapbox got lots of low value load when Google upped prices, load that they then subsequently decided to shed by upping prices). Other map data providers were already more expensive than Google.
So that means it should be safer to rely on GCP (or in general on services where Google is not dominant) than something like Maps where it is dominant.
In 11 years of using it, I've never seen AWS increase the price for a service whose cost inputs they controlled entirely. It can happen, though. They increased AWS Pinpoint costs because their cost to send SMSes in India increased by 25% due to a regulatory action that increased the price for all SMS traffic in India.
Often, AWS's prices are _really_ high for new services. I think this is useful for two reasons: it ensures your early adopters are those who get the most value, and it gives you a big buffer when discovering how much it actually costs to operate. This usually guarantees that prices have only one way to go. For example, AWS IoT Device Management had a 90% price cut after it was introduced.
> Often, AWS's prices are _really_ high for new services. I think this is useful for two reasons: it ensures your early adopters are those who get the most value, and it gives you a big buffer when discovering how much it actually costs to operate.
I've thought about this concept a few times lately.
Like, I get the impression that a savvy business, when starting a new product with very little competition, should start with prices as bad and guarantees as low as they can get away with, because they can always improve their offer later, whereas going the other way runs into loss aversion and consumer alienation.
Eg it's better to start at 40$/month and lower to 30$/month after a year than to start at 20$/month and get a lot of angry users when you realize you need 30 to break even.
No, they have stopped reducing the prices of existing services once they become depricated, so they effectively become more expensive compared to other services but never increased.
Although reselling other products, such as RDS Oracle or MSSQL could potentially have increased if the licences fees went up.
But I don't think that is what you are referring to.
This really does seem like a shortsighted move given they’re seemingly pretty serious about Google Cloud, and their Maps Platform is apparently part of that: https://cloud.google.com/maps-platform
AFAIK AWS has never increased their prices for anything ever, Google Cloud once increased the price of a service by 1400%. That takes Google Cloud out of consideration for me for anything I don’t want to have to or can’t (because of vendor lock-in) move to another service with very little notice.
I went through the same journey and never built on Google Map APIs ever again. The unreliability hurt more than the money. There's something pure about using a capitalistic API (e.g. AWS, Stripe, Twilio) where you know they're making a good buck off your usage, whereas with Google it felt like a side business that they'd change on a whim.
I feel similarly about brokerage accounts that are upfront about charging a transaction fee or checking accounts that have a clear minimum or fee. When it's "free", I'm always scared that I'll end up paying another way and what I'm relying on will get washed away.
> That, in turn, led to the realization that we needed to
> remove our reliance on Google's entire stack, and quickly
> or we'd be in a terrible bargaining position later.
Always keep deployments vendor-agnostic. I'm very very happy on AWS, but I could literally deploy to Google Cloud or Rackspace in an hour if need be. I run my own Linux servers, Elasticsearch and RabbitMQ servers, and though I do use and love RDS, I could just as easily move to a self-installed MySQL server. And I don't use the AWS DNS services, rather I trust that only with my registrar who I am very very happy with.
That's exactly what you should expect when anything comes into monopoly. When you have no alternatives, any price could be the best offer for you. For the same reason, when I have to use cloud, I would try to avoid anything that deeply binding to a particular vendor by using IaaS or PaaS so that we can bail out when something went sour.
Google's choice to bump these prices made me realize that they could do the same thing with their compute cloud pricing or other APIs, again with little notice. That, in turn, led to the realization that we needed to remove our reliance on Google's entire stack, and quickly, or we'd be in a terrible bargaining position later.
This one choice by the maps team likely cost Google significantly in the medium to long term, if any other companies realized the same things we did.