A) How many of those people who run their own products with thousands of users come looking for a job with you? (or, perhaps, 'ran' in the past tense)
B) How do you know they programmed that app with thousands of users even remotely well? Did they store passwords in plaintext? Does it need to be reboot every 5 mins? Is it slow as molasses, but hidden behind load balancing?
Someone could have written awesome sites with 1000s of users and loads of revenue, but it might still have been done like crap, and doesn't belong anywhere on your network.
A) I was illustrating a point, but I will clarify it as it didn't come across: If someone demonstrates extraordinary ability in projects before the interview, I will not ask them to spend four hours demonstrating rudimentary ability.
Oh no, I was mostly commenting on my past experience. If someone shows past achievements, I'm going to adjust the interview to their level, rather than ask them rudimentary questions or ask them to implement rudimentary and time-consuming functionality.
You can get a feel for whether the candidate can program a simple blog by asking them how they'd do it, you don't have to ask them to spend hours actually doing the nitty-gritty...
B) How do you know they programmed that app with thousands of users even remotely well? Did they store passwords in plaintext? Does it need to be reboot every 5 mins? Is it slow as molasses, but hidden behind load balancing?
Someone could have written awesome sites with 1000s of users and loads of revenue, but it might still have been done like crap, and doesn't belong anywhere on your network.