My experience (as someone who was elected to public education administration) suggests it's often more about logistics. For example, adjusting how many bus drivers (and busses) are needed in a district to serve elementary, middle and high schools. Staggering start times reduces the need, but drives one of these options to be earlier than the others.

High school sports complicates logistics later in the day.

It's not just a question of learning; it's also a question of cost and resources.