That's the only thing I was thinking with their A/B test. The calculator might immunize against unhappy customers later on. I think they could've looked at something like the percentage of customers who leave one or two billing cycles later.
Unfortunately, there's never enough time to run a proper experiment - we want answers now! Who cares if they're the right answers. Short-termism can't wait two months.
