Hacker News new | past | comments | ask | show | jobs | submit login
"Attrition is a group statistic. It doesn't apply to me." (josephwalla.com)
185 points by arram on March 29, 2013 | hide | past | favorite | 110 comments



Special forces training is taken by a large number of people - some of these people are just chancing it, hoping they will be able to push through anyway but not really prepared or ready for it. Others, like your brother, are fully prepared. When looking at a base rate, it's important to understand the different divisions inside that base rate and the reasons people drop out.

In special training, the reasons people are likely to drop out are because they aren't physically fit enough to cope. This gives your a brother a good indication - he likely knows others of similar fitness to him who passed. The key here is that there isn't much randomness involved - either you are fit enough or you aren't.

Startups are a completely different ballgame. The main factors behind dropping out are a complete mystery - team composition, relationships, lucky deals with investors, hitting your marketing in at the same time some other big player pulls out or pitches in - the list is endless and impossible to even fully understand.

Basically what I'm trying to get at is that comparing something you have complete control over (special forces training) vs something you don't (a startup's success) is not a great analogy.

However you're still correct anyway - the best indicator of whether a startup succeeds is whether or not they try in the first place! So it's always best to try and ignore the baseline.


> In special training, the reasons people are likely to drop > out are because they aren't physically fit enough to cope.

That is the opposite of everything I have read about special forces training.

Anyone who makes it past the initial stages is extremely physically fit. The difference between the qualifiers and the rest is in the mind.


Still, it's in your mind. To make it more like the startup world, the gauntlet should randomly throw in bear attacks.


> The main factors behind dropping out are a complete mystery - team composition, relationships, lucky deals with investors, hitting your marketing in at the same time some other big player pulls out or pitches in - the list is endless and impossible to even fully understand.

What? There are some things beyond your control, but work with smart people on a problem people are willing to pay for on the leading edge of the current wave of technology. Dozens of good and bad things happen randomly--take advantage or weather the storms. Don't fuck up any of the obvious things, and work hard. Do this and you'll get paid.

Now, there's a whole other class of startup where you're betting that in six months, you'll have figured out the magic viral fairy dust that makes 200 million people want to visit your site before the money runs out. You might need some luck.


I think that this doesn't address the important element that things like special training courses aren't wholly about "succeeding" or "failing," especially when we're making statements like it being all about the mind. The Army, in its special forces personnel, is looking for a very specific group of people, and the challenges are designed to weed out those people from everyone else who is going through the course. Any good selective interview or course like this should be designed that the company/division is testing who the candidates are that meet their desired overall profile--in which case, not passing the test or course isn't a failure or demonstrative of a lack of ability on the candidate's part, but rather a lack of fit for that specific position or unit.


A wonderful point - too often overlooked.


As a former Recon Marine I can attest that there are two reasons that people would drop out or be dropped for selection.

The first wave of drops is how you describe. It is people that don't have the basic physical performance to continue. This in my perception accounts for a relatively small number of drops due to everyone going through prior screening.

The second wave of drops is very similar to the first, but the root is different. The failures are typically for physical reasons, but the source was mental. A lay thing to say is "He didn't have the heart." These people are typically labeled as "badge hunters" as they are looking for the bravado of the title, but honestly don't enjoy the job itself. The wrong types of motivation will quickly weed themselves out.


I'd argue that special forces training is not a matter of complete control, I'm sure that people bump up against physical limitations (eyesight, coordination, etc.) that would allow them to be a regular soldier, but they never know until they try.

Similarly, things like carefully choosing your co-founders, advisors and investors... doing good customer development, understanding your market better than anyone, give you more control over your fate.

Nothing is absolute, but the analogy seems apt to me: You're not a statistic.


I disagree. Statistics about start-ups also ignore how prepared people are, and how committed and talented they are.

Many personal achievements in life are a "complete mystery" to many or most observers, or to science, or to mainstream intellectual discourse, but they still draw on rare and special qualities of individuals. Even if you don't "control" the outcome, that doesn't mean it's not a function of who you are (in the broadest sense).


Of course - it's also a function of who you know, your timing, and a huge number of additional factors. It probably has a lot to do with 'rare and special quantities of individuals' as well. The difference is that if you can run 50 miles without breaking a sweat and do pull ups using only your pinky, it's pretty clear that you will make it in the special forces baring a freak accident.

However, you could have just created a successful startup. You could create an incredible product. You can still fail if Google launches a competing product at the same time before yours can gain traction. You can still fail if someone else comes out with a better and cheaper way to do what you're doing. You can still fail if a key part of your marketing fails you for reasons out of your control (Facebook blocking access to their API).

It's simply a function of the amount of chaos. Nobody can tell you if your startup will succeed or fail. I'm very sure that a special forces instructor will be able to tell if you'll make it after just a few minutes of watching you perform.


All start-ups face unforeseen setbacks and unlucky turns of events outside the founder's control. They are a constant. It's all in how you respond.

So yes, there's a high degree of chaos. This means to succeed you must be all the more resourceful and dogged and/or experienced. Maybe these things are harder to quantify than predictors of success in the special forces. I don't know anything about the special forces. The wording used above just strikes me as true for start-ups rather than being a contrast.


"The difference is that if you can run 50 miles without breaking a sweat and do pull ups using only your pinky, it's pretty clear that you will make it in the special forces baring a freak accident."

Leadership and team-building abilities are a significant success factor due to peer evaluations. You could be a 0.1%'er in physical ability and still get peered out because no one trusts you or wants you in their squad. Like a startup, your ability to build relationships is key to success.


Sounds like an attempt to posit away the fundamental attribution error.


A couple thoughts:

* Getting demotivated by statistics is the worst possible use of time and energy. It means nothing that the average reader of this comment has various qualities and abilities that, on average, are about average for a reader of this comment. Every reader is an individual, not an average. Start-ups, like people, and marriages, are so different that you just can't generalize across them in very illuminating ways. This is the same as saying there are many subtypes, and at best you could draw comparisons or generalizations within a subtype. Which is the same as saying if you scope your generalizations small enough and choose your points of comparison wisely, you have a chance of capturing a meaningful pattern that might actually tell you something about an individual specimen. Or you might just be over-analyzing.

* It's possible to create causality and attribute events to your actions, even retroactively -- it's called taking responsibility. You can even take responsibility for things you didn't know about, like an executive or official who resigns over misconduct below him. There is very little we have direct control over -- the positions of our limbs, for example, but only when we are paying attention to them -- but we are expected to exercise whatever indirect control we have. People in high positions of power and responsibility have very indirect control -- hiring the right people, putting policies and checks in place -- but they must accept this responsibility and milk it for all it's worth. When you take responsibility for something, you choose to account for (recognize, pay attention to) the causality between you and that thing. We exist in a vast interconnected web of causality, so it's all what you choose to focus on.

This isn't about disposition vs. circumstances, either. Your disposition and circumstances are both different from other people's, as are your goals, motivations, and strategies. It's about how much weight you should give a statistic about other people when trying to determine something about yourself. As you learn more about yourself and the world, and especially if you pursue areas where you are strong, statistics become meaningless.


I don't see any particular reason to become demotivated about statistics. Statistics are simply a tool, useful in developing an evidence-based understanding of the world. You can accurately generalize about many things, and there are even statistical tools to help you determine how accurate your generalizations are and what kinds of variability in outcomes there are. It's not like the field was just invented yesterday and nobody has thought about these issues.

If you're making decisions, evidence-based decision making can help you decide between different courses of action (A/B testing is an extremely simple example of statistics in action). Statistics can even give you some information about which factors can impact the likelihood of different outcomes. Maybe you think factor Y about your current situation (or your personality, or your product, or whatever) makes some outcome particularly likely. Is that actually true? Sometimes the honest answer is we don't have any way of knowing, because it's a unique situation. But I think people jump to that conclusion far too quickly. Often there is actually considerable information available, if you want to use it. I'd rather use existing data than cultivate some kind of "everything is my decision and I'm in control" mindset, when that would just be delusion in many cases: many things are beyond my control, but some are within it, and I think making effective decisions requires figuring out which is which.


"Demotivated by statistics": This is the topic of the thread. Someone quotes a "failure rate" and it is demotivational because it suggests you will probably fail, when in fact that is a naive and over-simplistic interpretation.


I think the linked article is completely the wrong way to respond to pessimism. It's also a rather naive approach, reminiscent of daytime-TV self-help stuff, or perhaps certain libertarian novelists. The correct response to pessimism is not to promote delusional, unscientific "magical thinking" that fantasizes about how any individual can accomplish whatever they want with enough grit and determination etc etc, riddled with survivorship bias and lack of critical reflection.

If a certain statistical interpretation is simplistic and naive, the response I'd like to see is a more sophisticated but still evidence-based interpretation. I don't think that "I'm a unique snowflake and I don't need evidence or science because of GRIT" is a good substitute.


Most of life is lived beyond where scientific evidence applies. The studies dry up, and what you have left is determination fueled by passion and commitment. Everyone is sitting around with the same feeble facts (e.g. percentage of companies in the US that fail annually, divorce rates, etc.), and if you are met with thinly-supported pessimism about your personal endeavor that you care about, you have to be prepared to supply thinly-supported optimism.

When life gets real, there is no comfort in figures. Real (and wonderful) commitments such as marriage and a challenging work environment have taught me that. And the more you excel, the more unusual you become and the more statistics will predict your failure.


Aww. I bet the person who won the lottery thought they were special, too.

Clearly, there is a grand misunderstanding of statistics here.

In order to make yourself an exception to the statistics, you're going to have to scientifically explain how you are going to become better. Everyone believes they're better. How are you, truly, going to go above and beyond? Do you have better connections than the average? Do you have the experience needed? Do you have a better idea which is downright guaranteed to succeed?

If you can answer Yes to all of those questions, congratulations, you are the average Startup Individual, and the statistics do apply to you. You're going to have to work really hard, have all the right things fall into place, and do everything right, and even then you still have an extremely large chance of failure. It is the nature of the situation.

Starting a startup isn't like playing the lottery. You have maybe 20% more control over your outcome. But 80% is still in the hand you're dealt, both as a person and as a company. Make the best of it.


I don't think it is a misunderstanding. For example, the overall rate of heart attacks could be something like 10%, but that doesn't mean that I personally have a 10% chance of having a heart attack. My actual chance is likely higher or lower depending on my family history and other factors. Group statistics give us valuable information, but they are not the whole story.


But, by talking about the "other factors", you are just re-configuring the group from which one would assess your individual probability of having a heart attack. You're right that group statistics aren't meaningful when the individual is not part of that group but, if that's not the case, then group statistics are the best source of predictive power.


People usually incorrectly assume that they are special. While it's certainly possible that there are extenuating circumstances that mean you aren't going to have a heart attack, in the absence of really strong evidence pointing in another direction, it's more rational to assume that the group statistic does apply to you.


One isn't a scientist observing one's own life. Being a realist is a good principle, but rationality is not an end unto itself. In the field of human endeavour, mental attitude is a huge factor. You can weigh out different cases in terms of odds, but the actor who dwells upon statistical improbability will have a much smaller chance of success than the one focused on actually achieving the goal despite the odds.


It's true that mental attitude is important, but you can still take a rational approach to situations like this. Look honestly at the odds. If you accept them, then cultivate the appropriate mental attitude to help you succeed, but if you do not start with a realistic assessment of the true situation then you are likely to waste much effort trying to replicate the paths that outliers have taken, effort that could have been better spent.

That realistic assessment should start by assuming that in the absence of further data you are a typical member of your class. If you have ideas about ways in which you are different to your class, try to validate them with actual data before making judgements based on them, because another feature of typical members of the human race is that they frequently invent spurious reasons to believe they are atypical.


"Have all the right things fall into place..." is a misconception that is actually far from the truth in my opinion. Most successful startups win because they go after SUCH a large market that they're able to be quite wrong about a lot, but they're right about a few key things (one of which is often thinking it's a massive market when others don't).


Yeah, there are a lot of determining factors which really boost the probability of success, and that is certainly one of them.

I'd say most people think they have a huge market though. How do you separate out the ones who are right (and thus the special person for whom the statistic didn't apply) from the failure cases (who believed in essentially the same preconditions but were incorrect/implemented wrong/didn't move quick enough/had some extenuating circumstance/didn't get funding/had the wrong partner/hired the wrong dev/didn't pivot/failed).

That's why experience is such a key factor—if you've already done it 3 times you have a lot higher chance of making the right decisions along the way.

I'm not saying it's entirely luck, but it's not deterministic either.


Contrary to most commenters here, I've recently started to embrace "being a statistic." I think completely understanding where you stand (on the normal curve) is not only important when helping you prepare for taking an exam, approaching a girl, or launching a startup, but also important for the many (many) inevitable failures. You will be able to deduce what you did right, what you did wrong, how to improve, etc. given where you started.

I think that embracing your inner statistic is a good, if not great, idea. Don't be depressed because you're not that special snowflake, instead try to understand where you are, where you want to be, and what, exactly, you have to do to get there. If there's a 99% chance that you won't get from where you are to where you want to be doing whatever it is you're doing, either amend your goals or do it a different way.

Sometimes there's no shame in folding.


Understanding the truth will always guide you right; denying it will lead you astray.


> Sometimes there's no shame in folding.

Actually, a correct fold is one of the most skillful things you can do in poker. That's why James Bond will never be a good poker player.


Why would you ever fold if you always have the best hand?


His brother is correct: statistics are a group property, or rather, they are they description of what happened with a particular group.

When when you start making predictions you're talking probability not statistics. Statistics may inform probability but there can be issues with this (eg bad sample, insufficiently sized sample, failure to identify the underlying characteristic(s) such that samples are actually flawed and so on).

The problem we have with a species is that we conflate the two. This comes up with highly political issues such as racial profiling. Imagine a situation in a southern border state where statistically more illegal aliens are Hispanic in both absolute and relative terms (meaning absolute number and relative percentage of their given populations). I say this not as a fact but as a non-judgemental scenario to make a point: law enforcement will likely, consciously or subsconsciously, make different suspicions based on such external characteristics.

Is this fair to the individual? Of course not. It borders on an assumption of guilt. People of Middle Eastern descent who regularly fly I'm sure are cursed with this kind of suspicion when almost all of them have done absolutely nothing wrong.

Anyway, as to special training, a relative of mine was married to a guy who went through SAS training (and passed). To those who think this is simply an issue of being sufficiently physically fit, you couldn't be more wrong. Now you have to be sufficiently fit for it to be possible to pass but after that it's a mental issue.

They put these guys through mentally strenuous situations: sleep deprivation followed by interrogation, psychological profiling and the like.

His brother has another point though: on an individual level you should never assume you're going to fail (or succeed for that matter) based on group statistics. Engineering is currently male-dominated but no woman should ever take that as evidence that they can't do it as an individual. Nor should any man assume a specific woman is less likely to be capable, etc.


If you don't think statistics apply to you as an individual, you are essentially claiming to be different from the statistical average in some meaningful way. It may help to articulate what that is.

So maybe most people fail this class, but I think I'm smarter or more determined than the average person, so I shouldn't be discouraged from trying.

I say this because if you don't actually believe you're above average in some meaningful way, you very much should be discouraged by statistics. I don't think myself stronger, more determined, or more desperate than the average navy seal candidate, so even though I'm an individual I shouldn't actually expect to pass the special training that most fail. Similarly, I should expect to pass basic training because most do and I don't think myself any worse.


Whenever you use statistics you are generalizing and likewise the statistics typically don't apply exactly to most people. Everybody is above or below the mean in some way there are almost always data points two standard deviations from the mean... They help us simplify how we think about large populations.

So many little differences can have huge effects on us as individuals that its really only necessary you are different. Failure can only truly be found in seeking to be the same.


I don't think your two racial profiling examples are so much a problem of conflating statistics and probability, as they are a problem of confusion of the inverse (http://en.wikipedia.org/wiki/Confusion_of_the_inverse). It is the case that if you randomly select an illegal immigrant in the US, then the expectation (p ≈ 75%) is that they will be Hispanic. That's not a misapplication of statistics; it's just what the numbers say.

However, that does not mean that if you randomly select a Hispanic person, the expectation is that they are an illegal immigrant. The probability is in fact closer to 15%.

The imbalance of those numbers for people who "look like terrorists" at airports is even more preposterous, of course.

But statistical invalidity is not the only reason that racial profiling is bad. Even if 99% of Hispanic people really were illegal immigrants, it would be unacceptable for police to target them based on race. Not because it would be an invalid conflation of statistics and probability - it wouldn't. The problem is that in order to protect individual rights, the police and the courts must not act strictly according to expectation. We shouldn't imprison someone, for example, just because we reckon there's a 75% chance they were the killer.

And if 99% of Hispanic people were illegal immigrants, and so police hassled every Hispanic person, then we would know just as sure as we'd know that they'd have a 99% success rate, that they'd be violating the civil rights of the other 1%.


You don't even need racial profiling to show that statistics harm individuals. Look at car insurance - in most countries you will have a massive premium to pay,just for committing the crime of being under 25. Even if you are 24, had your licence for 6 years, drove hundreds of thousands of miles without an accident, your insurance will still be higher than that of a 26 year old person who has just passed their drivers test and doesn't know anything about driving. Why? Only because "statistically" you are more likely to be in an accident. Well, so are men, yet it is illegal to make their car insurance higher because of their gender in almost all EU countries. You may not have done anything wrong, but are still "punished" for being part of the statistic. It makes me really upset sometimes.


Well, they also help individuals. Think of the person who is 26 years old, has just passed their drivers' test, and then totals their car and kills the other driver. Insurance shells out a few million in liability claims. Their insurance will go up, obviously, but they're not going to be financially wrecked, and it's the rest of society that bears that burden.

The way to think of it is to think of your whole life statistically, as a series of events where you win some and you lose some. And then don't get upset over the ones you lost, because there are many others where you ended up with the winning end of the deal and you just don't remember it because humans are wired to dwell on losses more than gains.


Sorry but I just cannot agree with your first statement - even if an 18 year old driver causes a crash and kills the other driver, the insurance company is still going to pay 100% of the claim, unless the claim goes over the insurance limit on your specific policy(which in EU is unlimited for death and bodily injuries). So I do not see how paying more when you are under 25 is a benefit.


I thought that, too, with car insurance for an under 25 year old... The price difference actually was about 25 euros a year (Germany)


I am from Poland and it's similar there - the difference is negligible. But I am studying in the UK at the moment and the difference is exorbitant - I would pay almost 3000 pounds per year for insuring my 2.0L Peugeot Partner, and my friend who only passed his test few weeks after turning 26 is paying 400 pounds per year for insuring his 1.6L Golf. Just because he is older, and not because he is a safer driver.


Nothing wrong with stats. They are what they are. Problem is when we then apply them to individuals. Then we are unfairly stereotyping the individual.


Car insurance is the prime example of unfairness and being punished for the actions of others, purely because of statistics.


Lots of care insurances provide benefits to people that deviate from the statistic minimum in a meaningful way - few miles per year, ... In all other ways, yes - you get to pay an price that's averaged over a reasonably large peer group that you just happen to fall into. That's how insurances work: They redistribute risk. They protect you from extreme outcomes by charging a low, steady price. All in all, you usually pay more than you get out and only a few actually benefit financially. Take a look at large car rental companies: They don't actually insure their cars against theft. They're large enough to do the risk arbitration themselves. So I guess you just need to find a large enough pool of people that agree with you and then form your own insurance - with lower fees.


>He explained that statistics are created for groups, not for individuals. As an individual, it's not relevant to you. He went on to pass the course on a broken ankle. That makes me think about some startup founders. Their companies should have died, since most die. They should have given up, since most give up. But, somehow they willed their company to exist. Startup failure is a group statistic, it doesn't apply to you.

Yes. Also known as "survivorship bias".

Fact is, the brothers of all the other people who attended that course could have (and might had) said the exact same things to their brother. And most of them would be proven wrong.

Group statistics are indeed about groups. It's our way to predict what the most likely outcome is for individual members of a group taken at random (or all other things being equal).


>Fact is, the brothers of all the other people who attended that course could have (and might had) said the exact same things to their brother. And most of them would be proven wrong.

Exactly. But those people don't talk about it a decade later. And that's selection bias.


This is one of the Five Hazardous Attitudes that make you an accident-prone person, a risk factor identified by the FAA

http://www.avhf.com/html/Evaluation/HazardAttitude/Hazard_At...

Reduce your risk and improve your expectation values by rationally using decision theory and correcting the systematic sampling biases in your head.


No, it isn't. This isn't him saying, "It won't happen to me" just because, it's that it doesn't apply because he's taking the necessary precautions. He's doing everything he can- and other people won't affect HIS rate of failure.


Yes, they will, and that's what makes him failure-prone.

When outside influences do come along that affect his expected outcome, he will be disrupted instead of scientifically understanding the situation, and his role as an individual who is not the center of the universe.


You are way too deterministic.


No, I'm a Bayesian.

When you know there is sampling error (luck) it makes sense to blend a little bit of the distribution of the group in with each individual.

For instance, if you look at baseball hitters for a season ordered by batting average, probably the guy at the top of the list got lucky (sampling error) and the guy at the bottom of the list had bad luck. The real batting average is a hidden variable that we can only see through sampling.

A good estimator of an individual's batting average can use the group distribution as a prior against the individual distribution as an observation -- this regresses the individuals back towards the mean, which is a realistic way to perceive uncertainty.


As a baseball stat head, I can extend your analogy.

In late 1990s, when Quinton McCracken first published DIPS theory, it was postulated that pitchers had zero control over their "luck based" factors (BABIP, in this case; expanded to include LD%, LOB%, HR/FB%, etc.)

Now there exists enough data to identify outliers, to whom the theory of DIPS (Defense Independent Pitching Stats) doesn't apply. Matt Cain, for example, is acknowledged to outperform his DIPS.

No matter what the statistics suggest, people will believe they're the Matt Cain. And some will be. By definition, they're the outliers that our statistical models do not account for.


As others, currently 'above' this comment, point out: not necessarily. You may have valid arguments that explain why you can validly assert you differ from the average.

For instance, as RyanZAG points out, if you are well prepared for a test, while you know others are 'just chancing it', then your chances of succeeding are much higher than the statistics would have you believe.

To state it more generally: the reason a statistic often cannot be used for an individual expectation, is that the statistic is based on persons completely unlike yourself. A relevant statistic would only include people like yourself. In his example: those that show themselves more aware of the difficulties that will be encountered, better prepared than average, etc.

As an example: my life expectancy is vastly different from the commonly stated 'average' life expectancy of people in my country. Firstly that is 'from birth' and I have already survived quite a bit. Secondly it includes smokers and I don't smoke. Thirdly it includes people that engage in all kinds of other risky behavior that I avoid. The average life expectancy may be 72, while mine is 87.


If you put your mind to it, you can achieve anything.

Except compute an uncomputable function. Or transmit information faster than the speed of light. Or simultaneously measure the speed and momentum of a particle with errors whose product is less than half the planck constant.

But apart from that, anything.


Do you really believe that, or are you being sarcastic?

Because I hate to tell you, someday you'll collide with real life. If you put your mind to it, can you become president? Chances are simply, no. A lot of people have put their mind to it, like Romney, McCain, Hillary, Kerry... it's not like they were lacking in motivation or desire.

So... no, sometimes, no matter how hard you want something, you are not going to achieve it. I know there's a big self-help movement, believing in attracting positive energy, believing in yourself, and whatnot... but a lot of people can't seem to always tell the difference between realistic expectations and self-delusion.


> Romney, McCain, Hillary, Kerry

They probably just didn't want it enough. If they'd really thought positively, if they'd just visualised themselves in the position they wanted to be in, they'd all be president now.

And if you're going to say "but what about their opponents?". Well, if they believed in themselves enough, then they'd be president too.

If Romney and Obama both believed in themselves, then they'd both be president.

Now that's the power of positive thinking.


So you were being sarcastic.

Having said that, positive thinking and motivation are very important factors in goal achievement. It's often the case that when someone fails to accomplish something, it's because there was simply something else more important to them that inhibited their effort.


In other words, positive thinking is very helpful, possibly even a necessary precondition for some outcomes (like becoming president - ain't gonna happen unless you really do want it), but it is neither a predictor nor a principal factor.

On the other hand, it is arguable that negative thinking can be a fairly strong predictor of failure. That is perhaps the most useful take-away from the otherwise relatively useless field of positive thinking: if you keep thinking you're going to fail, keep visualising and expecting failure, chances are very good that you will indeed fail, even if you had all the other necessary factors for success.


Touché. :)


The point of these types of stories is that the right mental state is a prerequisite for success--but not a guarantee.

Yes, it's true that most people who want to be President, won't make it. But it's also true that every single person who thinks it is impossible to become President won't make it either.


Hello? Hello? Anybody home? Think crazygringo think


The issue with this statement is that you don't actually want to achieve "anything". You want to realize your potential.

Before you do anything, you must discover what you love and what you're good at. You can then do that thing. It feels like "anything" because if you had a choice of anything, that's what you'd choose. But you can't actually do things that you wouldn't want to do anyway.


> Or simultaneously measure the speed and momentum of a particle with errors whose product is less than half the planck constant.

This is easy to do actually. Unless you define error as a value that depends on multiple measurements. I prefer to just go with standard deviation and keep the ambiguity out of it.


Reminds me of Steven Jay Gould's classic "The Median Isn't the Message":

"If the median is the reality and variation around the median just a device for its calculation, the "I will probably be dead in eight months" may pass as a reasonable interpretation.

But all evolutionary biologists know that variation itself is nature's only irreducible essence. Variation is the hard reality, not a set of imperfect measures for a central tendency. Means and medians are the abstractions. Therefore, I looked at the mesothelioma statistics quite differently - and not only because I am an optimist who tends to see the doughnut instead of the hole, but primarily because I know that variation itself is the reality. I had to place myself amidst the variation."

http://www.cancerguide.org/median_not_msg.html


He certainly had a way of bringing out the beauty in truth.

What he's saying here really is useful, since he's also recognizing the variation. He knows that he is not the median or mean, but a point somewhere in the field among other points. The median does not exist; it's only an abstract measure of the wide range of variation and commonality.

However, it would be foolish (or, as he puts it, optimistic) to place yourself blindly at the top of the bell curve without evidence to support that.

As Gould (bless his heart) says in the paragraph following that: "I possessed every one of the characteristics conferring a probability of longer life: I was young; my disease had been recognized in a relatively early stage; I would receive the nation's best medical treatment; I had the world to live for; I knew how to read the data properly and not despair."

He was a scientist. He had evidence and reasons for believing he lied where he did among the statistics. He didn't choose to believe that, he deduced it.

If someone has good reasons for believing they're going to be better than the average, more power to them. They're simply making an intelligent conclusion. But you can't choose to be successful by sheer willpower alone. You have to make a case for it; build evidence that proves it. I don't see that here. I don't see it with most people who believe this sort of anti-statistical nonsense. All I see is a misunderstanding of statistics; a lack of respect for the variation and where you are likely to lie within it.

You have to understand that before you try to do better.


You still need a backup plan, I assume that if you fail special forces selection you get to keep your job in the regular military.

If your startup goes bad, make sure it doesn't leave you in an untenable financial situation regardless of how optimistic you are.


That is actual wisdom. It is okay to lie to yourself about your odds of success, if you have done your homework and have a viable Plan B.

Having the "best plan" is usually only possible in hindsight. My observation is that the most successful people are not necessarily the ones with the best plan, but those who act decisively on good plans and know how to pick themselves up with an even better attempt after a failure.

I do not really see how failing to enter special forces has a huge downside. Even not succeeding offers the opportunity to learn.


Although I like the sentiment, I disagree with the accuracy of this.

Attrition is a group statistic. It doesn't apply to you if and only if you are exceptional in some way. Otherwise, it applies to you even more.

In the specific example, his brother knew he was so much harder than his peers that he could do something incredible, like run for miles with a broken ankle. That mental toughness is how he was exceptional, and why he could be confident that he would be one of the people that made it.


You're probably giving him too much credit. I've seen interviews with people before they embark on these kinds of courses, and every single one of them said that they really wanted it, that it was the most important thing in their lives, and that the group statistic didn't apply to them.

Some of them made it. Many of them didn't.


Ignoring base rates is a great idea until you are hit by a base rate. And by definition, you are probably going to be hit by a base rate. Everyone who succeed and failed all thought they were an special individual.


Most people don't win the lottery. Statistically speaking, you almost always lose. So why play, right?

Sometimes, its okay to ignore the statistics and hope for the best. Sometimes, its okay to stop hoping and start believing, even if the odds are grim.

Selection bias does teach us one useful thing. That you don't win if you give up.


Ignoring the odds only helps if you are flying the Millenium Falcon through a statistically overly-dense asteroid field, and that's arguably only because you have the sister of a Jedi (and daughter of a Sith) on board.

In every other case, you are better off using your resources on something with a more reliable return on resource investment.


If that were true, there would never be any entrepreneurs. Employment is safer than a startup, for example.

Now weighing your comment against really long odds like the lottery, its definitely more correct.

The point I wanted to make is that its fine to hope in things that are seemingly unattainable, and to be somewhat ignorant of the realities. Imagine, for example, how bad college basketball would become if players started realizing just how long the odds are in the path to wealth through the NBA.

It turns out, a lot of our economy runs on hope.


A lot of college ball-players knowing that they aren't going to make it to the pro's, transition into coaching, parlay their athletic prestige into a corporate position etc.

Just because you are gambling with your life doesn't mean you can't gamble better. Problem is when people get delusional about what they can achieve vs. the reality. Most people are born and remain losers. My motto in life is to preserve my capital, so I can lose for as long as possible.


The kids who start playing at a young age don't generally start out dreaming of selling out for slightly better odds at some point later in life. They have their eyes set on their heros and they go as hard as they can. Its only later that they become realistic in their expectations.

I am trying to weakly correlate this concept to startups. You have to start out dreamy-eyed enough to start. You can deal with reality later.

When Scott Mace and I started CalendarHub and applied to YCombinator in the same year as Justin Khans Kiko, we weren't shooting for lifestyle business. We were shooting for Get-Bought-By-Google or bust. Reality set in later. That was my start, at least I started :).


The lottery is controlled purely by the odds, the outcome is stochastic.

Playing my own devil's advocate: C3P0 states the odds for successfully navigating an asteroid belt, but Han doesn't want the odds, mainly because C3P0 hasn't stated the odds for Han Solo succesfully navigating an asteroid belt. Han works best when he doesn't know how often other people fail.


The Force is not a Field of Luck +1. She wasn't piloting.


It is, however, a Field of Plot Device. She wasn't piloting but she was essential to the story, so the probability-function collapsed in a way that preserved her ability to contribute to further events.


I did say arguably. And Jedi do make other people operate "better", at least from some of the scant reading I did.


The lottery is one where everyone pretty much has equal odds of success. Individuals, especially in startups, tend not to have equal odds at all. It's a game of skill, not luck.


> It's a game of skill, not luck.

I'm gonna guess that you didn't found a company at the Nasdaq peak in April of 2000.


It's a game of more skill than the lottery, sure. But it's still largely a game of luck.


I don't exactly disagree, but I would say there are enough things outside of your control that luck does play a part. However, to your point I believe that (with great skill, appropriately applied) you can bend the odds in your favor.

What I really wanted to do in that comment was espouse what the article is selling... positivity/hope.


in my high school, in 1989 in India, my maths teacher was into single digit lotteries, which give a 10% win chance for lowest prize money.

with his maths, statistics, patterns, trends, he increased his odds to 30-70%, I have seen him when he is 100% sure, he would be travelling 100 miles to different places just to buy all the lotteries


> with his maths, statistics, patterns, trends, he increased his odds to 30-70%, I have seen him when he is 100% sure, he would be travelling 100 miles to different places just to buy all the lotteries

Those payouts must've been pretty decent to justify so much time and effort. (Or perhaps he was, unfortunately, not also into economics.)


at todays conversion rate one lottery was about 4 cents, and price money was about 20 cents. these lotteries were preprinted with the numbers and available only at lottery vendors across the town, one vendor would have about 20-100 of the same number, so if he believed he would win, he will buy one number each for the predicted number and one for one random number and then buy those 2 in volume of hundreds across the town. it was pretty addictive


So let's see... You say he had a 70% chance of winning, so if the prize money was 20 cents, each ticket had an expected value of 0.720=14 cents but a cost of 4 cents so a net of 10 cents. I'm not sure what you mean by '20-100 of the same number' - he could only buy 20-100 from each vendor? Let's say he can buy 100 at a vendor (if they're the same number and so the entire batch will win or lose, then that renders the expected net value a little worse a fit but whatever) , that's 10 cents 100 or 10 dollars.

In the US, minimum wage tends to come out around $7, so if he can hit one vendor per hour then I suppose he'd be showing a reasonable profit. But you also need to take into account wear and tear on vehicles, risk of accidents, planning your life around the lottery... I suspect with those included, it'd still be net-positive but not really that great a use of time and effort. Personally, I'd be doing it more for a fun write-up than to make money.


I'm sure most people in his selection group felt the same way, and most of them failed. We only write blog articles about the exceptional ones.


I feel like not being realistic about your odds is a very dangerous game to play. Sure, you can maybe grit your teeth and bear it. But you also might totally burn out doing something unrealistic.

This applies to more than just start ups: It's very rare that you can succeed through sheer stubbornness, many times you'll just end up hurting yourself.


I think he's trying to say that there's a difference between your odds and the population odds. The key is to know thy self.


Startup success and lottery wins are very different things. You don't get better at lottery as you buy more tickets over the years; but you get better at your trade as you do it more and more, even if you fail at each previous attempt to start a company.

If you see it as a lottery, then it doesn't make sense for people to try starting their own companies after several failed trials when they reach some middle age, as they really don't have much time left. But, it is just the opposite, because they are actually much better at seeing pitfalls and bs, and are a lot more likely to be successful.

So, the statistics do matter, you just need to know the kind of statistics that apply. As a rule of thumb, in real life, it is usually not the lottery kind.


Classic survivorship bias scenario. Nevermind that all the other people in the course went in with equally inspiring slogans like Failure Is Not An Option tatooed across their buttocks. Some people fail, sometimes it's going to be you.


We're doing motivational posters here now? Okay, we can do that.

I recommend: the Tao of Programming http://www.canonical.org/~kragen/tao-of-programming.html


Related reading: The Codeless Code http://thecodelesscode.com/contents and ESR's Rootless Root http://www.catb.org/~esr/writings/unix-koans/


60% of people drop out[0] - before you knew this, you had an 60% chance of dropping out. Now you know the statistics, you can go about exerting effort to end up in the 40% that don't drop out. And that number - 60% to 40% - tells you a little bit about how much effort you're going to need to put in (not much).

90% of startups fail[1] - now you know that, you want to put yourself in the 10% that don't. 90% to 10% is a bigger gap to jump than 60 to 40, so it's going to be a lot harder.

0,1: not actual statistics


This is The Biggest Lie Entrepreneurs Tell Themselves (tm).

You hear about the fact that only 10% of startups succeed, but you tell yourself "I'm better than them". pg himself reinforces this notion by saying things like "some founders have a 0% chance of success, others a 95% chance".

pg is right, in a sense, but there is very little way for you to know whether you really are "better" than others. Very few people actually know what makes a startup successful, so there's that. On top of which, the people who are taken into account in the 10% statistic includes, among others, people who are building their 5th startup, the sons of highly influential politicians, personal friends of movie actors, etc. It includes people 30 years in a particular industry, with tons of insider knowledge, who decided to build a product in that industry. And it includes other 21 year olds with stars in their eyes, and nothing more.

This isn't to say "don't build a startup". That's a different conversation, and depends a lot on your goals in life.

But most people are in the startup game at least partially for the money. And for the purposes of calculating your potential earnings, unless you have a good reason why you're special, pay attention to the damn statistics!


My oncologist told me "Statistics are not people."

I was about to undergo a bone marrow transplant. At the time, mortality within the first year was 90%. But I was much healthier initially than most other patients, because I had different medical history.

Plus, I'm one tough SOB.


> But I was much healthier initially than most other patients, because I had different medical history.

That just seems like you need better statistics. More sophisticated cancer-prognosis models typically do take initial health and medical histories into consideration.


What if he broke more than his ankle?

Obviously he was mentally and physically prepared for the course. But a course is predictable. Startup economics are more like combat, you can be the best soldier but an IED finds you.


"Who cares what statistics show? Look at medicine. 80% of people with pancreatic cancer die within 5 years and 95% of appendectimies occur with zero complications. But we both know cases of pancreatic cancer patients that lived and appendicitis patients that unfortunately passed. Statistics mean nothing to the individual" - Dr. Cox (Scrubs)

That quote has always stuck with me since I saw that episode, if you enter a situation knowing statistics of outcomes, it can affect how you perform.


And maybe it should affect what (if not how) you perform. If you are probably going to die soon, maybe you would feel your life best spent in a different way than if you had decades left.


Meh, I agree with him... if only because I'm one of those guys who cares about how I perform. It's like being in the gym... I'm not going to be the strongest guy in the room, but what I do care about is how progress.

At the end of the day, I don't care how others do it outside of learning from how others didn't succeed and those who did.


Between this article and the beekeeping one, HN isn't making a good showing on the day for scientific literacy.


"...we've found a very serious problem: no matter how carefully we select the mean, no matter how patiently we make the analysis, when they get here something happens: it always turns out that approximately half of them are below average!" -Richard Feynman


This is a very acute observation. It applies to you, but not in a way you think. In fact, as an individual it probably means the opposite to you.

Check out Douglas Hofstadter's GEB, for a really enlightening take on the topic. Check index for "Aunt Hillary"


This is more or less the same point that Stephen Jay Gould made in "The Median Isn't the Message:"

http://people.umass.edu/biep540w/pdf/Stephen%20Jay%20Gould.p...


Classic case of not understanding statistics and probabilities...


Extreme determination is a rare and powerful thing.

Combined with some talent, luck, humility and hard work, one can accomplish anything.


I've always viewed statistics as information about aggregates. Anything can happen when N = 1


You are not a special snowflake.


So is survival rate.


This is why there are so many graduate students who can't find jobs. The prospects (acceptable for STEM, dismal for all else) are published, but people who are used to being the "1 in 10" fail to realize that, when it counts, they likely will be. The other 9, it turns out, are Ivy-educated smart people just like them.

This might apply to VC-istan as well, although I think founders are less delusional about it than employees. Founders become EIR if the thing flops; employees think the business is "fully de-risked" (because they're told that to justify the 0.0x-percent equity slices) and that they're guaranteed to actually get the executive positions they were promised (ha!).




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: