From Note #1, http://www.paulgraham.com/swan.html :
"The biggest exits are the only ones that matter financially, and those are guaranteed in the sense that if a company becomes big enough, a market for its shares will inevitably arise. Since the remaining outcomes don't have a significant effect on returns, it's cool with us if the founders want to sell early for a small amount, or grow slowly and never sell (i.e. become a so-called lifestyle business), or even shut the company down."
Who are you serving? Investors or yourself? The SV model is that this is all that matters but a 50M business is still a damn good business and not a "lifestyle" business.
Also, read the rest of the quite-- lifestyle business was only one of the options.
It's quite possible to build a fast growing business that you never sell, that's not a "lifestyle" business.
The either/or dichotomy is disproven by your quote.