Amazon also has been running internally on containers for years before ECS. But workloads for massive FAANG companies designed by FAANG engineers turn out to be quite different than most AWS/GCP customers.
Just like EC2 wasn't Amazon selling its "spare capacity during off-peak" but always a purpose built service with completely isolated data centers and network fabric from day 1, the connection between Borg and Kubernetes is tenuous at best.
Honestly, I would encourage you to evaluate ECS and Fargate. Forget Kubernetes and direct instance management. Try a construct like https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws... and see how much simpler it is than all the K8S overhead.
Source: I'm at Amazon but I work on none of these services and opinions are my own. I played with K8S and am eternally confused how it's as popular as it is. I guess it gives you the illusion of not having lock in?
I only have one thing to point out about Borg and that is large numbers of Borg developers started working on Kubernetes. They got rid of Borg mistakes and replaced them with all new mistakes. :)
I don't see K8S being about lock-in at all. Using a cloud provider gives you some sort of lock-in and you should be utilizing that providers strengths.
Using Kubernetes is all about the patterns that it provides and how it removes a certain class of problems for you. Deployments, scaling, logging, etc are some of the patterns it provides and the consistency matters. How many of us have worked at companies where deploying two services have been completely different? One team runs the jenkins pipeline while another ftps the files over. Now multiply that by several services and several tasks (logging, scaling, etc).
1. Is instance management on ECS really annoying enough to justify the higher fargate price?
2. Does ECS have something similar to pods, i.e. colocated containers which can communicate over localhost, share filesystems, etc? A quick search didn't turn anything up.
Amazon also has been running internally on containers for years before ECS. But workloads for massive FAANG companies designed by FAANG engineers turn out to be quite different than most AWS/GCP customers.
Just like EC2 wasn't Amazon selling its "spare capacity during off-peak" but always a purpose built service with completely isolated data centers and network fabric from day 1, the connection between Borg and Kubernetes is tenuous at best.
Honestly, I would encourage you to evaluate ECS and Fargate. Forget Kubernetes and direct instance management. Try a construct like https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws... and see how much simpler it is than all the K8S overhead.
Source: I'm at Amazon but I work on none of these services and opinions are my own. I played with K8S and am eternally confused how it's as popular as it is. I guess it gives you the illusion of not having lock in?