The issue is that it needs the really really good on baseline before people start switching from other services.
But if you are able provide better baseline than others, then you might not even need the users to be "better. With users, you would be then exceptional.
I suppose services like Gmail benefit a lot from users who mark the occasional spam letter in their inboxes. While each such marking is inconsequential, correlated over many users, they allow to detect new and unusual waves of spam, and to start filtering them for everyone.
A similar mechanism could work for a search engine. It would be harder to game if it required high karma (many other users matching your judgment before it starts to count), and a paid account, like with Kagi.
But if you are able provide better baseline than others, then you might not even need the users to be "better. With users, you would be then exceptional.