The 10x engineer exist but not because he or she is so great but because they wrote the shitty code in the first place so it takes a new person 10x longer than the author to do anything.
the exponential cost thing is probably somewhat true. If the people who come up with the software actually knew what the fuck they wanted they would save the software from having to guess wrong and redo the code. Unless the software runs on a million dollar jet or something finding a bug during testing should not be 100x more then finding it during code. Infact I can make a decent argument that it is cheaper to pay a tester to find bugs then to waste an engineers time looking for them.
I don't know why anyone down voted this because there's definitely at least some truth here. A lot of times the stereotypical 10x rock stars in an organization are really smart and productive people who ship features and move on to the next feature, leaving maintenance to the people who are left behind. They get the glory, but in many cases (totally anecdotal of course) they did it by taking all sorts of shortcuts and kludging things together in ways that aren't scalable or maintainable.
In this case, the 10x productivity boost comes from incurring technical debt that they personally will never have to deal with. Many organizations need these people in order to actually ship stuff, but their productivity comes at a cost that's paid for later, by other people. Maybe they're 10x up front, but it ends up being more like 1-2x when these costs are taken into account.
I'm more worried about the much more tangible 0.1x (or the feared but very real -1x) engineers, who exist in many organizations.
I downvoted it because people will go to any rhetorical lengths, including "un-defining" the very word intelligence to death, just to believe some people aren't more intelligent than others.
Oh, new rule! We can concede there are "10x" devs, but they have to write shit code! Give me a break.
They ARE a thing. Work for any large and successful software company and you'll find (or hear stories about) at least one. Look for the people that are consistently put on mission critical projects.
There are more types of extraordinary developers, but 10x is the only one I hear consistently mentioned.
I.e.: Let's explore every possible angle from which we could detract from the phenomenon. Leave no stone unturned!
By my count, I've worked with 6 of these mythical creatures at 3 jobs in 15 years. Is that rare? I don't think so. The fact is, in a lot of cases the team or even the entire company would not even exist as they did, without these people. And everyone knew it because it was blindingly obvious. What would id software have been without John Carmack? Did you know that the first version of Microsoft Math, which included a fully general algebra solver, was written by one guy as a side project? Nobody else on that whole team did anything like that. Just that guy. The 10x dev is the person all the other devs are intimidated by, strictly on the basis of the code they've written.
I've been a fan of Laurent's work in this area for a long time.
What I find most interesting is that Laurent's work is clearly about the pervasiveness of myth and the need to apply rigorous thinking to what we believe. It's not about the specific claims; those are (fascinating) examples.
And yet, every time, people get really worked up about the 10x issue. They ignore the broader point entirely and focus on that one claim. (I find it particularly interesting that they focus on 10x and not any of the other claims Laurent debunks. Perhaps because it's easily understood? Perhaps because it's easy to think of anecdotes? Perhaps—and I suspect this is the real truth—because they secretly believe in their hearts that they're among the Chosen, and Laurent's threatening their clubhouse?)
Laurent doesn't say, "there are no 10x programmers." He says, "the studies that claim 10x programmers are so weak as to be laughable, the notion of measuring productivity itself is controversial and poorly defined, and do you even know which ratio you're using when you say 10x, anyway?" (He's not quite so informal about it, but that's the gist.)
In other words, the 10x claim has yet to be defined, let alone subject to rigorous analysis.
But out of the woodwork come these people who say, "I've seen 10x, therefore it must be true!" (I can't help but believe that these are the same people who jump up and down saying "the plural of anecdote is not data.") To be blunt: irrelevant. Laurent didn't say "there are no 10x programmers." Sit down already. Read it again.
You know what? The 10x claim doesn't matter. Laurent's not even saying it matters. His argument is bigger, more interesting, and more valuable than that.
He's saying that people—us programmers, specifically, who so wish ourselves to be rational thinkers—believe things because we want them to be true. Not because of the "evidence" and "proof" we're so keen to demand when we hear something we don't like... but because they're the myths that surround us, and they make us feel good.
(And now I hope morendil will chime in and tell me how I've misinterpreted him. ;-) )
You're quite right that people tend to miss the broader point.
Could be my fault, could be that more people - on this site in particular - ought to be familiar with pg's Disagreement Hierarchy: http://paulgraham.com/disagree.html
LOVE this post. thank you. All these so called "analytic" minds making leaps of faith that make religion look well thought out. There is data that says the less you believe you are likely to be wrong, the more likely you are to be wrong.
Same goes for so much that passes as argument on Hacker News, especially on topics about which participants are largely ignorant.
The biggest problems is that no "law" or "rule of thumb" of software engineering is always right. So "debunking" by finding an exception isn't really useful. The best answer to most of these questions is: It Depends (tm). There are situations where the right person making the right calls makes a huge difference and there situations where anyone can make a call that would turn out reasonably OK. There are situations where buggy software has little impact on the business and there are situations where it can result in bankruptcy. This is partly why this is difficult to research in any sort of rigorous fashion and why we should definitely maintain some degree of scepticism to any broad claims.
x10. Anyone here do the TopCoder Algorithm SRMs? There's clear evidence of x10-ish performance difference between the "red" coders and the average coders in there. Does that always apply? No. In certain situations can certain developers blow the average away? Absolutely yes. This is true for things other than software development.
The author talks about "engineers of equal experience". No such thing. There are no two single developers with "equal" experience.
Bug fixes. It's a no-brainer to see the cost of fixing an issue by the same dev in reasonable proximity to when the code was written is a lot cheaper than someone else fixing that bug 6 months later. It's also a no-brainer to see that the cost of fixing a bug in nuclear reactor control software that can cause a core meltdown is cheaper before the core actually melts down. In between there is a lot of grey area but I think there's a lot that can be applied while still being safely in the no-brainer region, that is our intuition, that is our learning/pattern recognition over years of software development.
Exactly: the test of a heuristic is its average case performance, not its worse case. If you select for finding a really bad case in "proving" it wrong, that's not very informative.
"The trope of 'software as a highly abstract, intellectually demanding, nearly mathematical activity' contributes to a myth of software development as a profession 'naturally' dominated by males: math is the subject of its own interlocking system of tropes, myths and stereotypes that paint it as a manly pursuit."
That logic seems to be all backwards to me. Of course developing software is a highly abstract, intellectually demanding activity. Claiming it’s not is as absurd as claiming women are somehow incapable of doing exactly that.
> Of course developing software is a highly abstract, intellectually demanding activity
I don't mean to sound harsh, but "of course" and "it's absurd" aren't much of an argument.
My advice is to dig into this a bit deeper. What do you think makes software "a highly abstract, intellectually demanding activity"? What would the world look like if that were NOT the case, what would it look like if it WERE the case?
How does this claim coexist with that, which has been bandied about for decades, that software programming is on the brink of being automated away?
How does this claim square with sites like the Daily WTF, which provide evidence that many gainfully employed programmers do not, in fact, use their brains very much? With the more or less weekly announcements of "security breaches" caused by people ignoring basic best practice, for instance storing passwords in plain text?
How much mathematical(ish) ability is actually needed to put together a Web page, a bunch of CRUD fields, and some backend database template?
Why is it so easy to get a job writing software even if you have no formal credentials at all? How is it that some people can get these jobs even if all they can do is write a Web page, and that badly?
It's possible that our experiences differ: that for you software has been abstract and demanding. (I can see people who write kernels for a living getting the "of course" reaction.) Whereas I've often rubbed shoulders with the world of "IT programming", with self-taught games programmers and Web programmers and people who turned a half-baked idea and their gift for the gab into multimillion dollar businesses.
Whenever I did my homework and dug into the facts, I found that things that "of course" held true weren't all that obvious. I'm trying to encourage people in the profession to get into that habit.
> as absurd as claiming women are somehow incapable of doing exactly that
Yes, I too see that as a false claim. Do you deny that there are people out there who are in fact making that claim?
>How does this claim square with sites like the Daily WTF, which provide evidence that many gainfully employed programmers do not, in fact, use their brains very much? With the more or less weekly announcements of "security breaches" caused by people ignoring basic best practice, for instance storing passwords in plain text?
I take many people being bad at programming as evidence that programming is hard, not that it's easy. Maybe it's evidence that getting a programing job is easy, but that's not the same thing.
> I take many people being bad at programming as evidence that programming is hard, not that it's easy.
We are in violent agreement. Note that "hard" is not the same as "abstract and intellectually demanding". Running a marathon is hard. Becoming a US Marine is hard. Many things are hard that do not primarily require quasi-mathematical skills.
For that matter, finding gravitational waves is hard but may be just as much about investing enough money (think space-based laser interferometry) or just plain luck.
The ways in which people suck at programming are much more diverse than just a failure of abstraction, otherwise we would all be writing Haskell.
These ways include failure to ask what the user or sponsor wants, failure to make sure we've understood what the user or sponsor said, failure to communicate with other members of the team, failure to question the things we learned in school and always took for granted. All of these are common failure modes in the biz, none of them are a failure of abstraction.
In fact excessive or premature abstraction is a widely recognized failure mode of software engineers, and the myth is responsible in good part for that.
You didn't just say "highly abstract", you also said "intellectually demanding". You only seem to be addressing the "highly abstract" part now. All of the failure modes you mention sound like failures of intellect to me.
Also, common failure modes of people working as programmers != common failure modes of people trying to become programmers. Even if most people working as programmers understand the abstractions involved well enough to function (and I'm not certain that's actually true), the abstractions may still be too much for many other people.
>In fact excessive or premature abstraction is a widely recognized failure mode of software engineers, and the myth is responsible in good part for that.
Part of really understanding abstractions is knowing when not to use them. And since, as you say, many people have trouble with that, understanding abstractions must be tough.
I take it as evidence that we are doing it wrong. I can only hope that in 50 years time people will look back and wonder how we managed to get it so wrong.
I honestly think a lot of you have gone plainly insane from years of Internet arguing, and can now write verbose devil's advocate arguments for absolutely anything.
Can anyone name a finding of software engineering research that isn't folklore? I've looked many times. The field is strikingly weak. It is typified by tiny sample sizes, subjective analysis, and zero replication.
Software engineering research is very tough, because despite the formal name, it's a social science, and no two projects are the same. Civil engineering holds up to physics and repeatable tests. No two software projects are the same. They have different teams, training, management structures, technical environments... Similar to psychology, it is very hard to draw rigorous conclusions for the real world, and most lab tests are of oversimplified situations.
I also get Citogenesis. This also gets abused in many fields. Think Cargo Cults, and stories of performance improving every time the lights go on and off, and how we're all evil because people in lab tests electrocute their test subjects.
That said, it is hard for anyone who has been in the world for a while to disbelieve the 10x programmers. We all have met them. They really are that much better. And they really can solve problems that others can't. They may not be better for every situation (Example: Following a structured testing methodology that requires a lot of human input) but they do exist. It would take a lot of evidence to prove that they DON'T.
Try "Making Software: What really works and why we believe it".
I'm about halfway thru reading it, some of the findings presented are that TDD studies either aren't very good or give inconclusive results; that a number of code complexity metrics don't predict bug rates any better than file size does; that bug rates are correlated with how well organizational structure matches code structure; and that competent PHP programmers do in fact exist.
I have the book and have read several (though not all) of the papers. I didn't see anything worth exempting from what I wrote above. The state of the field is just very weak. Even the good researchers, like Lutz Prechelt, aren't producing anything that comes close to justifying changing one's mind based on evidence. (The PHP vs. Java article struck me as fluff; so many other variables come to mind so easily.) I'd be happy to be wrong. Counterexamples are welcome.
I did like the paper on code size very much. The principle that code size is the best measurement of complexity and a good predictor of error rates is probably the finding I'd name if I had to answer my own "name one that isn't folklore" question. At least that one has multiple studies behind it. Even so, most of them (that I've seen) aren't very good.
"singling out some myths for critique rather than others gives a misleading impression"
Article author here. Agree with your first para, so the above has me puzzled, a bit.
Would you please expand: what impressions do you think the article gives, that are misleading? (And if possible: what do you see or hear that makes you think they are misleading?)
Edit: What I said there was rooted in earlier discussions and not the current article. Sorry; my confusion. But I'll leave the explanation of what I meant anyway.
You place so much emphasis on dismantling the 10x claim (writing an entire book about it) rather than software research in general, that an observer might reasonably be left with the impression that there's something more "mythy" about that claim than others. If so, I'd say that's misleading.
Folklore isn't ipso facto false. If folklore gets repeated as persistently as 10x does, I'd say that speaks weakly in its favor. (Please note that I said "weakly".) Because of this, I don't think it's irrational to continue to favor that view, even after debunking those studies. For one thing, the opposite claim is even less supported by data. For another, it fits loosely with my experience and that of others I talk to. Experience in the field may not be controlled measurement, but that doesn't make it rational to dismiss it.
I do think there's something to the 10x claim about individuals; I think interaction between individuals is even more important—100x, one might metaphorically say—and that question is even less amenable to formal study. I doubt that we'll ever see convincing quantitative results on either of these things. Building teams will remain an art, and folklore and myth are the stuff that art is made of.
> dismantling the 10x claim (writing an entire book about it)
"Leprechauns" is by no means entirely about 10x developers.
In it I also tackle things like the "exponentially increasing cost of bugs" claim, the Cone of Uncertainty, the "software crisis", the successive mythical reconstructions of the Waterfall bogeyman, the limitations of the "empirically based software engineering" movement and specifically the problem of discipline envy.
If it had only been the 10x thing that turned out to be ill-supported I wouldn't have gotten my knickers all atwist. My beef is precisely that when you look closely most of software engineering looks awfully like pseudoscience.
This article is, in a way, Chapter 1 of the next book, in which I'd look beyond "academic" myths, and try to confront a broader picture of how the software community thinks about itself, and how that perpetuates some of the problems we've been bitching about for decades. (And some of these problems turned out to be non-existent; other very real ones barely rate a nod from academia. That is also part of the problem.)
Thanks for the clarification. It seems we disagree less than I thought, and I've deleted my mistaken impression from the root comment.
I may have asked you this before, but do you know of any finding in the research literature that you don't consider pseudoscience? The only one I know of that might come close is research on code size. I've heard that the literature on code inspections is good, but that may just be another myth.
While I understand that you are not going to debunk all the myths, I think the article would be much stronger if you at least battered the "10X myth" a bit
Software development is a craft driven by heuristics. The conditions for every project are different. It's difficult to identify contributing conditions. These conditions include technologies, abstraction, business factors, trends, development talent, talent of peers, market position, maturity of the market, funding, knowledge of current requirements, mutation of requirements, complexity of the requirements, interacting agents, etc.
I can see why it would be so difficult to find correlations.
"The best I can do, therefore, is this: my advice is to open your eyes to the pervasive influence of culture on all the things you deplore; to acquire and use whatever tools, from semiotics or literary criticism or whatever, seem to be handy in understanding the water of myth around you.
And when you get there: write about it (or sing, or make movies). This, really, is the key thing. Not only do we need to talk more, not less, about how we develop software, we also need to talk about the things we say: we need critique."
if only the discussion on hacker news would actually take this advice instead of arguing arcane bullshit. Hoping a few will take this tack. Please surprise me.
Well, at least he's doing his best to prove the myth of all programmers being humorless autists who take everything literally to be true...
Yes, sure, we're talking about exactly 10x when we talk about 10x programmers. Etcetera, etcetera.
Even the most fanatical Agile evangelist gets both the tongue in cheek humor (with a serious undertone) of Zed's "programming, motherfucker", but not this guy.
To be fair, sometimes it is hard to tell when those in the Ruby/JavaScript/Web 2.0/NoSQL/"opinionated" crowd are being serious and when they're joking.
Some of the claims or statements they make in a serious manner are so far off-base that they appear to be jokes to those with more experience or higher standards.
The "JavaScript is like Scheme" claim is a good example of this. A lot of JavaScripters will openly express this idea in complete seriousness. Yet anyone who has actually used JavaScript and Scheme will know that this really isn't true at all. The true intent of the person making the outrageous claim is hard to discern. They might be serious, or they might be joking. Often, it's safest to assume that they're being serious, until they explicitly admit otherwise.
I don't mean to make fun of you, but: http://badassprogrammers.com/ basically demolishes that line of argument (and in the bargain confirms that Poe's Law does apply in Zed's case).
Particularly poignant is the way they redacted "autistic idiots" to "socially awkward idiots": if you're going play the "politically incorrect" card, play it with panache instead of being hypocritical about it.
Tons of 10x hate in here already. Is there a reason this is such a stumbling block for people? Is there a reason that isn't tinged by sour grapes? I'm like 99% convinced that I've worked with such people and seen what they can do. I'd be willing to buy that it isn't simply the individual, but the individual plus the right circumstances. But I absolutely can't say that I've never seen a single developer write a surprisingly large amount of good code in a surprisingly short amount of time, because I have seen it.
"some developers are surprisingly good" must be the weak form of the 10x hypothesis, and to quote someone else on the internet "where there is a strong version and a weak version of a hypothesis, the weak version is usually trivially true". (The strong version is that the weak version is always just trivial and the strong version is never true).
I think it's important to distinguish 10x "developers" and 10x moments that a given developer has.
It's like 1.000 game, a hat-trick or triple-double - sometimes any given player has the potential (though not all may be given the opportunity) to make their mark.
Lebron James does not make 10 times the contributions of the other players, but he makes the team incredible. The number attached to the 10x engineer is largely bullshit as well as not measurable. I've worked with engineers who make everyone around them much more productive through excellent library code or laying down dope infrastructure that everyone else can leverage. I've also worked with engineers who mentor other engineers extremely well giving those productivity gains in turn. That doesn't mean she is slinging ten times the code or tickets or whatever every one else on the team is; she is just enabling productivity gains by being an excellent team player. The notion of 10x is a misnomer at best and at worst cause people to focus on other measures that are ultimately meaningless...leading to burnout.
Oh, ok. Look everyone, now it's the number that's the problem. How many more ways can we think of to discredit this notion, which is clearly so in need of discrediting? Get to work, people.
I think the 10x label is just something that is easy to say and comprehend. But the real impact of a high-performing developer is probably way more than 10x. It's not really about lines of code. It might be that this person makes a breakthrough which changes your industry. It might be the difference between staying in business or not.
I've had the pleasure of working with very high-performing developers and the positive impact they can have on a team is amazing. I can see how some people don't believe it if they have never worked on a team where one person amazed and inspired them.
the exponential cost thing is probably somewhat true. If the people who come up with the software actually knew what the fuck they wanted they would save the software from having to guess wrong and redo the code. Unless the software runs on a million dollar jet or something finding a bug during testing should not be 100x more then finding it during code. Infact I can make a decent argument that it is cheaper to pay a tester to find bugs then to waste an engineers time looking for them.
the software crises story about defence i think is real: http://www.reuters.com/investigates/pentagon/#article/part1