Couldn't that just be gamed by allowing fast speeds for the initial connection of X size or Y Time and then dropping down to less once they guesstimate the test is over?

That proposed plan could probably be reverse gamed by Netflix by buffering streamed content with multiple separate connections. Or by including a way to check the speed mid stream. Or by allowing extended tests on fast.com. This is a classic cat and mouse game.

That's nice, but isn't it still a violation of net neutrality?

It's gaming the violation of net neutrality that already exists because of the ISPs.

Yep, but if anyone else notices they lost corresponding bandwidth they could set up their own such tests and demonstrate the disadvantage.

Depending on their public-spiritedness, Netflix may make it easier for sites to set themselves up with it.