The thing is, mathematics is nowadays almost the only science where someone can make a name for themselves at a young age, because they don't need as much funding (except to pay their own bills), research grants, associates to do grunt work etc. etc. So if the Fields Medal would really go to "the traditional pack", it would be especially unfair. Of course it's also unfair that many of the biologists, physicists etc. doing the grunt work for their higher-ups don't get recognized either until they manage to climb to the top, but such is life...