One thing every company should do, tag each resource by team-projects and at end of week-month attribute cost to each team. Seeing a bill make the developer and manager more conscious of their choices.
My current consulting gig is at a government superdepartment with so many agencies agglomerated into them that IT treats them like external customers, with charge backs, contracts, the works. This includes ubiquitous cost centre code tagging of all resources.
What cost savings do you think this achieves?
Nothing. Literally nothing.
I actually tried to cut their cloud spend, I really did! They were burning through $250K per month. Now, a year later... it's $350K.
First off, every team negotiates a "budget" with some paper-pusher finance guy who has literally no clue whether the spend is ludicrously high or wonderfully frugal. Zero clue. None. No idea what a "core" is, or what the funny database product names in the cost breakdown even mean.
So you end up with websites where citizens occassionally upload cute pictures of wildlife getting 50x the opex budget of critical infrastructure and that's seen as perfectly fine. The budget is approved. The paperwork is done. It's fine now! Don't touch.
I tried to optimise a few of these systems, and I was literally screamed at. Mid-level managers on $100K a year were telling me I was "wasting their time" trying to cut their cloud spend by $100K per annum.
In some cases they were right. I had meetings where I was trying to convince a team to let me cut their annual spend from $5K to $4K. The result was that they politely pointed out that this mere 20% savings had already been eaten by the cost of the meeting itself.
This is the problem. The big-budget projects won the fight to get that budget and will never let go of their victory. The small-budget projects are not worth optimising because you lose the benefits to the paperwork overheads.
Meanwhile, using efficient shared platforms is also out the window because nobody wants to share any more. Why pay for the resources used by another team from your budget? How would you convince your boss to pay for that?
Sharing is caring, but charge-backs make for fiefdoms. This is mine, that is yours, go pay for it yourself.
So that's how you get VMs with 8 cores sitting at 0.1% load.
You're going to say that that's no so bad because of CPU sharing and overcommit. Bzzt. Not in the public cloud: those cores are 100% yours!
The public cloud is not VMware. You don't share. Everyone gets a slice to themselves. That's why it costs so much!
agree tagging is good, but you only identify costs by teams and not solving optimization part. All the way you will need tools for autoscaling or to take an advantage of spot instances
