I recall reading somewhere that basically everyone does A/B testing absolutely wrong, which means it ends up being "coincidentally" always picking what the team wanted to pick anyway.