It is difficult to scale trust, for once: you would get people who keep submitting everything as spam, until overload of the "spamming" committee you are suggesting (if I understand you well). Many people are not that able to make the difference between spam and not, also (is efreedom spam or not ? If you are not a programmer ?).

I also suspect just taking the top 5 % will get a lot of false rejections, which is a real issue.

But when I say that there is a cost/revenue issue, it goes both ways: everybody thinks about the cost of spamming, but if the income coming from spamming decreases, it would also fight spam quite well. Decreasing this without decreasing google's revenue may be challenging.