We did not use an NLB because it is expensive in its pricing model for TLS connections (and we prefer to let AWS terminate TLS). For 20 million connections, an NLB would cost $28,800/month. It's drastically cheaper to use an ELB, provided you can talk to AWS to have them scale it for you.

When I last discussed this with AWS, having them scale it for you meant that you actually had to contact them to pre-warm the ELB whenever you expected more traffic. It didn't work for our use cases where we didn't know when to expect more traffic.

Yup, that hasn't changed in our experience. If you're looking at bursty traffic beyond what AWS will pre-warm for you, then this likely won't work. We have fairly consistent loads throughout the day.

Back in the day, in my first job, I created load balancing using LVS. It was not easy to set up, compared to AWS, but it worked, and it was completely free, plus some man-hours up front. I wonder if it would be worth it nowadays...

Is it drastically cheaper to use an NLB without letting AWS terminate TLS? I thought AWS was expensive for TLS termination, generally, unless you need the client locality for lower latency.

Yes, if you terminate TLS its drastically cheaper. Without TLS, each NLB unit lets you have 100,000 concurrent connections. With TLS, a single unit is 3,000 connections. Similar pricing occurs with ALB's.

Terminating TLS yourself incurs some CPU cost and a bit more memory cost. How much CPU/memory is eaten depends on the efficiency of your code. Our Rust implementation roughly matches C code efficiency, so we could handle terminating TLS ourselves if ELB stops being feasible at some point.

You can also employ DNS round robin with health checks in Route53

ELB is pretty old and should be avoided tbh, use ALB and if you can't should switch to NLB ( for pure tcp LB ).

Same issue as NLB, they charge for units based on concurrent connections and it gets very expensive very quickly.

DNS round robin has the disadvantage of not guaranteeing that the traffic will be equally distributed. This can be a problem if there's a lot of automated traffic.

But, with that said, in term of price, it's unbeatable.

I have had success DNS round robin-ing a cluster of HAproxy LBs which handle SSL and then proxy to my backend websocket gateways which keeps the real load evenly distributed where it matters and also run health checks on the nodes.

The point is at scale you can't guarantee equal but something around there

That works well, but if you do ecommerce or a web app for which people write a lot of bots, then the traffic won't necessarily be that well distributed (mostly because users while run bots out of dedicated servers or such and will tend to only query dns once).

A bit of an edge case but pretty much the only issue I've had so far which is why I bring it up.

That's the first tool I grab for balancing _a lot_ of connections.