> I personally don't see the outrage. AmaGoogSoft overcharges for data transfers because they know they can get away with it and that lowering it won't attract more customers.

I know a handful of scientists, myself included, who would consider cloud computing were it not for the expensive egress costs. The ease of spinning up lots of computing power for scientific modelling is useless if retrieving the vast quantities of raw output data is costly. I will have to investigate OVH though as a potential opportunity--thanks.

The cloud pricing model has, I would say, definitely been set for ecommerce uses, not scientific computing. Ecommerce uses use a lot less data, and if they are using a huge amount of data they are likely to have a corresponding amount of revenue to pay the high prices.

Scientific computing -- you've probably got a ton of data, and your 'revenue' from project-based grants doesn't really scale with the amount of data and other computing resource needs you have, at least not to the extent it does for ecommerce.

Possibly AWS could make money on scientific data at prices you could afford... but they probably make the _most_ money by targeting their pricing model at ecommerce, which has the most money to spend.

Possibly there's a business model for a cloud-provider focusing on scientific computing. I mean, there probably are such providers already I don't know about? Maybe? But it'd be a tough business, the people paying are pretty budgetarily constrained, their budgets don't neccesarily go proportionaly as their resource needs go up, they traditionally are (ironically) averse to innovation in IT, there might not be enough likely customers to give the provider the scale they need to make money, etc.

Capitalism!

Yeah I suppose my argument is a bit moot given Amazon probably makes plenty of money in the private sector without having to worry about academia. But if they could get the cost down to be on the level of buying and running your own hardware, they could see an emergent market.

In fact, the federal government in its current business-oriented mindset may even lean towards a contractor handled machine to replace ones like Yellowstone at NCAR or those at NOAA NCEP if the price was right.

It would make a lot of sense if grants covered such costs, even at current AWS prices.

Currently, I suspect a lot of IT-heavy grant-funded projects are basically relying on university infrastructure being provided without full (or in some cases any) project-based cost recoup, which universities are increasingly loathe to do. The indirect/overhead margin probably doesn't always really cover it. (And it's not like a university is going to let you use 'overhead' dollars for AWS, _even if_ it would actually be a cost savings compared to the local IT you are using instead!)

It's not really rational economic decisions being made all around.

Proper professional IT is expensive. People still think automating is supposed to save them money, but turns out, nope. Many industries are still relying on unprofessionally provided IT instead of paying for professional IT.

If you're an academic you could probably negotiate with google/msft/aws for a discounted if not free egress rate.

May I ask why you want to retrieve it? Wouldn't you just leave it there, and download only the conclusions?

I can think of a few off the cuff.

You're likely not running a model just for yourself. You have collaborators at other institutions that need data to compare or combine with something else.

Your grant may require you distribute the output for use by other researchers. That could either entail being hosted by you or by an agency or other entity. But you still have to get the data to them.

A reviewer for your publications may request the data.

That brings up another point that the review process can be upwards of a year.

Oftentimes you have to go through an exploratory data analysis, where you don't even know what your final analysis will entail.

How much raw output data are we talking about? What order of magnitude? Unless we're talking PB a month EC2 prices seem fairly reasonable. Also keep in mind intra-region communication between instances is free