Hacker News new | past | comments | ask | show | jobs | submit login
Don't measure developers, measure projects (anaxi.com)
164 points by jeanlaf on Nov 26, 2018 | hide | past | favorite | 62 comments



Don't automate what you can't first do manually. Your best developers might be helping, supporting, designing and architecting your next solution.

One commit, one PR, might prove more valuable for your business than hundreds of other tickets closed.

Remove the KPIs and talk to your team members in 1-on-1s regularly. Give responsibility and trust to your team, and they'll be self-empowering.

Add KPIs and you'll take away their responsibility. Trust will be lost. Probability for innovative solutions will go down, your best developers will leave.

It is all about trust and responsibility. Hire for team spirit, conscientiousness, communicative skills and diversity.


The phase of a project with a flood of PRs and closed tickets is the easy part. The real work is done before and after (planning and debugging), in people’s heads.


I saw this happening. Putting metrics puts pressure and people wont innovate and leave. You can't be innovative when there are metrics on your head.


Agreed! I couldn't have said it better myself. It's really hard to quantify all of this without remembering that no two metrics are interchangeable / comparable.


Of the metrics listed, the only one I will say never to use is lines of code. As a very wise manager told me, "Code is a liability. Functionality and business value are assets." Measuring developer productivity by looking at how many lines of code they've written is like measuring the health of a business by how much money it's spending.

If you can delete code and retain the same level of functionality (for example, because you found a library that will automate much of the task), then that's a huge win. Every piece of code that the team writes is one more thing that has to be unit tested, integration tested, documented, and maintained in the face of future changes. If they can offload that work to a library, or find a way to accomplish the same task by using existing code or by reconfiguring the application, that's almost always preferable to writing new code to accomplish a particular business need.

Of course, one can go too far in the other direction. It's easy to fetishize using external libraries for everything (like we see so often in the JS ecosystem). But that's where I would argue that the analogy of treating code like money shines. Sometimes it is worth spending money to have something custom made for your business, rather than buying an off-the-shelf component. Similarly, sometimes it is worth writing code to do something custom, rather than integrating a third party library, or reconfiguring an existing application. But that choice should be consciously made. Rewarding developers for lines of code written moves that choice out of the realm of conscious decision-making and into the realm of reflex. They get paid for more lines of code, so they're always going to reach for the solution that results in more lines of code, regardless of whether that code is necessary or appropriate.


I agree that lines of code is a terrible measurement of productivity.

I don’t agree that short is always better. Sometimes it’s better to write the code so that it’s easier to understand than to use smart tricks to make it short, because the next person looking at the code might not be on to your tricks. You can argue both ways on this of course, but I prefer clean and easily understandable because I’ve seen what happens when you don’t do that and a junior programmer has to fix something 15 years later.

I also disagree that libraries are better. You seem wary of JS, and that’s good, but no libraries are magical and none of them can be trusted to do something for you, unless you look under the hood.

Especially not open source libraries. If you don’t know what is going on inside the libraries you use, then you’re heading for a world of pain. Of course some libraries are better than others, but using them blindly is wrong. And if you have to read up on them, then they aren’t really quick fixes.

We had a developer who used a library to handle decimals. We have our own library for this, that we’ve build because almost none of the open source libraries for decimals are very good at handling memory allocation and floats and computers aren’t 0.00000000000 they are 0.00000001436 etc. But our developer didn’t know we had our own library for this, so he used a public one. One that uses around 10 other libraries, one of which didn’t handle memory allocation the way you’d want.

So suddenly our decimals were wrong, but the computer didn’t know they were wrong, because technically the library worked it just wasn’t specific enough. And it cost us a lot of money to clean things up afterwards.

This is the extreme example, but it’s never been more relevant than it is today. Especially not in a world where less and less programmers seem to actually know how computers or even compilers work.


>I also disagree that libraries are better. You seem wary of JS, and that’s good, but no libraries are magical and none of them can be trusted to do something for you, unless you look under the hood.

I wouldn't go that far. You use glibc, right? Have you looked under its hood? If you're writing Java, have you really looked inside Spring or Hibernate? In the .Net world, how many people have really looked at the ASP.Net source code? How many Python developers look at the code for Requests? Even in Javascript, I can't remember the last time I had to crack open jQuery and take a look at its internals.


> But our developer didn’t know we had our own library for this [...]

I guess you also addressed this problem, right?

Distributing knowledge inside the company is an issue, that's even more crucial in software engineering.


I assume that's “especially not-open-source libraries” and not “especially not open-source libraries”


No, I meant to say that open source libraries are especially dangerous.

I get why you may object, but open source libraries have failed us a lot and by contract the .net framework library hasn't.

There are a lot of great open source libraries, that are both well documented and maintained, but there are a lot more that aren't.


Then i don't understand why you associate not knowing what's in a library with specifically open source ones. They tend by nature to be more open to introspection by any member of the community.



Completely agree! Love the quote "Code is a liability. Functionality and business value are assets."


Well some times a lot of effort is expended on "functionality" when its not a good use of time.

I worked at one company where a level 2 Engineer was bucking for promotion and spent almost a year (wall time) and ten plus man years redeveloping a system in Oracle that was functionality but added no real value.


That reinforces my point. This developer didn't add any functionality. Functionality implies value. It implies that the software is functional towards a business goal. So, in my view, this developer added a bunch of code without generating any new functionality, increasing your liabilities while leaving your assets the same.

And yes, I've seen this same behavior as well, at other large firms. The best way to get a promotion is often to launch something, so people launch, and don't worry about whether the thing they've launched adds any new functionality, or is yet another reimplementation of an existing framework that will make some junior developer miserable.


The Goal has a bunch of insights translatable to software engineering. This is one of them.


Very much so, and an easy read.

Unfortunately so much development is "build it and they will come" rather than analysing and improving a system or solving a specific issue - which may or may not involve coding.


> As a very wise manager told me, "Code is a liability. Functionality and business value are assets.

Obligatory: https://www.folklore.org/StoryView.py?story=Negative_2000_Li...


It is reasonable to use as an indicator of scenarios where you should pay closer attention.

If someone produces an unusually high amount of code, that might indicate that they are producing a lot of functionality, but it might also indicate that they are churning out mounds of crappy boilerplate that should be abstracted better.

If someone is producing an unusually small amount of code, that might indicate that they are very unproductive, but it might also indicate that they are very efficient and are finding simpler ways to express functionality.

All of these things are things that you want to be aware of. The mistake is treating metrics like this as something that is correlated directly with performance instead of just tools to help you understand the bigger picture.


> As a very wise manager told me, "Code is a liability. Functionality and business value are assets."

I'm not sure he is drawing the line in the right place. Business value requires functionality which requires code. Business value would be an asset for obvious reasons. Functionality and code would be liabilities. Functionality and code that doesn't provide business value is waste. You want to minimize both functionality and code. Maybe it is just the analogy breaking down though.


Business value requires functionality which requires code.

I don't think business value necessarily requires functionality. Business value is generated by code operating inside a specific business process. Only part of that process is software; the rest of it exists in the minds and the actions of the people interacting with the software. In some cases, it's easier to change the business process than it is to change the software.


One thing I have noticed when working with juniors is that the wrong things are considered for story point estimation and those story points are the metric of success.

Juniors tend to be overwhelmingly concerned with basic literacy. First its a "does it work" and then it graduates to "how to write the code". This often comprises thinking about tools, frameworks, dependencies, and code style. These aren't deep problems that moves a business forward. These are competency problems that keeps a person employed. This is a distraction that consumes developers with busy work instead of thinking and planning about how to make a product better or how to release faster.

Worse still, too much emphasis on the particulars around the how of it can blow out a 30 minute task to a 2-5 day task with all the unnecessary scaffolding and extra process. This is particularly frustrating when realizing these concerns can be immediately eliminated when the proper automation is in place. It is possible to really get good at spinning on mouse wheels and cranking out those stories a little bit faster, but it still isn't moving the business forward. Nothing forces out strong developers faster than cliches like: "That is the framework way" or "You're reinventing a wheel".

Some better metrics to really measure real world success:

    1. does the change solve a business problem
    2. does it pass the project automation
    3. does it reduce total code instructions
    4. does it close out defects
    5. reduced execution time
    6. improvements to project automation (faster developers)
    7. improvements to documentation
 
Once the proper metrics are in place the concerns of the juniors are addressed economically. For example, if reducing instruction count is important the code gets leaner and the how of it shifts from code style to leaner code. Code style is important because developers need improved literacy from uniform code expression, but once the goal of expression is leaner code the uniformity shifts from expressive style to optimization concerns.


I'm not sure what you mean with "reducing total code instructions" and "lean code", but I'm not sold on the idea of making code less readable which often happens when you try to reduce code by being smart.

Reading code is mostly way harder than writing it, especially if you read code from someone else. Optimizing for readability is the game changer for maintainability.

* Use expressive function and variable names.

* Have clear consistent structure across your project(s).

* Use simple code where possible.

* Do not write one-liners that do multiple transformations. Each line should have one purpose.

* Document design decisions in code. Don't document "what" your code does, that should be readable by itself, but why you made the decision to do it like that.

* Documentation library style comments for complicated functions helps a ton to know what the function is supposed to do.

* Add tests that specify the expected behavior. This not only helps when changing code, but also when reading.

It's not readable code XOR documentation, it's both. Readable code is the "what", documentation is the "why". Even if the docs get outdated, the why can still be insightful.

Non of the above suggestions usually reduces code, but it will help everyone else, including future-you in 6-months, to work with your code.


Less code should not mean either less readable code, or cleverer code. It should just mean ... less code.

Using abstractions, removing dead code, refactoring to share common code paths, etc, all reduce the amount of code you have, and all typically increase the readability of your code.

This doesn't seem at odds with your bullet list for maintainable code.

A useful maxim is: working software is an asset, source code is a liability. More code means more maintenance cost, so accomplishing business outcomes with less code is good, to a point.


A simple rule is that more instructions mean more execution time and more people time. The first of those is not always true, but the second of those is generally always true.

The total instruction count is the number of instructions from the source code that are actually executed. This isn't the lines of code that people read. For example a loop may comprise 100 iterations and 3 lines of code. That loop will likely result in 300 instructions. If a loop contains a child loop than the instruction count is something like:

    (iterations * overhead) * (iterations * overhead)
I am not a fan of OOP programming because there tends to be a lot of overhead. I understand this overhead helps to bind things together at writing time and share resources at execution time, but at maintenance time years and hundreds of developers later its often a catastrophic mess. I would rather the binding be an explicit call to a reference that I can read opposed to reasoning about flow control that isn't evident from reading the source code.

I am also not a fan of pronouns, which in some languages is the this keyword. If I have to guess what a reference is contextually I am wasting time that could be better spent making a code change.


Why would you think less code means the code is less readable?

Does cramming 5 lines of code into 1 line make it less code? No.

Reading code is harder, so the less code there is to read, the better.


As I said in my comment, a lot of the stuff I see as positive for making code more readable more often than not adds extra lines as comments or as code, because you keep it simple. You are right that cramming 5 lines in one, doesn't really reduce code, but I have worked with enough programmers that thought it's a good thing.

Sure if you can reduce code without making it more complex, that's a good thing.


Noooooooo. All of these metrics are a mistake, don't measure any of them. Measure the things that actually matter about your product and whether or not it's getting better or worse.

Measure outcomes. Only outcomes. Try and make sure your metrics capture the real outcomes and not just what you see in a lab. This usually means you need to use field data to get that information.


In larger organizations it can be very difficult for team managers to measure business outcomes of their work directly. Talking to customers is a difficult process, if at all allowed. Information from customers percolates through many layers of support and sales staff, product managers, business stakeholders, project managers etc. By the time the information gets to you it is tainted by bias and politics. Not that it would have helped you much since your objectives are already set by your hierarchy, and again are focused on your ability to execute, not business outcomes.

That may not sound like a very productive environment, but that is the environmental in which most of the world’s software is written. For those engineering managers that have to operate in it, measuring your ability to execute is the best you can do. You might as well do it correctly (which is hard).

And if you’re one of the lucky few working in a less dystopian environment, don’t be too quick to gloat. Enjoy it while it lasts, you’re just one acquisition or reorganization away from joining the rest of us in corporate mediocrity.


I work for one of the world’s largest companies. If your organizational structure is a problem for connecting outcomes with engineering then it sounds like you’ve already found a great problem to solve and so go ahead and tackle that instead of spending time on bad metrics!

(Full disclaimer: I am also currently working as one of those project managers you mentioned. If your project managers aren’t communicating clearly about how engineering impacts outcomes, find new project managers.)


So, based on your public profile, it appears that you work at Apple, which is not at all representative of the experience of the vast majority of software engineers. You also seem pretty young, so you’re unlikely to have a lot of experience prior to your current job. It kind of shows in your reply, too. For example you suggest that a manager should focus their efforts on changing the organization around and above them. That is very bad advice in almost every large company. First, you should focus your energy on helping your team succeed within the organization you have - not the one you wish you had. Then, if you have any bandwidth and willpower left, you can attempt modest amounts of positive change in the organization, starting with the absolute worst hair-on-fire problem you see. But keep your expectations low, because most attempts to change large organizations fail. If you are too unhappy with an organization to tolerate it, and you don’t have the power to fire the people in it, the solution is generally to quit.

You appear to be one of the lucky few not in this situation, but you don’t have enough experience to realize how lucky you are - or how quickly your situation could change. You think everyone is happy with their organization at Apple? Think again. Like I said, you’re just one reorg away.

Please refer to the last paragraph of my previous comment, it is extremely relevant to you.


It isn’t by accident that I am in a functional org. I select where I work carefully, and I walk when they aren’t functional. You should too.

Managing your team is table stakes. Managing 360 isn’t a sign of lack of experience.


First you said that if I don’t like my org, I should change it. Now you’re saying that if I don’t like my org, I should quit. Which is it?

Perhaps you should just stop trying to give me unsolicited career advice, and actually address the comment you’re replying to? Just a thought.


The post looks reasonable to me. The point he's making is that no metric is useful for measuring developer efficacy/productivity & the underlying unstated reason is that dev works tends to be random; problem A is nothing like problem B & there's no way to objectively measure the quality of solution for problem A against another developer's solution.

He's instead proposing metrics that you can use as a high level barometer of project health. What outcome are you looking for? You can't mean the final shipping SW because that's usually way too late. You want to figure out if your velocity is going to let you hit your ship date well in advance of the actual ship date so that you have time to deploy process fixes/hire more people so that you do hit the ship date.


> He's instead proposing metrics that you can use as a high level barometer of project health. What outcome are you looking for?

None of the proposed metrics capture project health. Test coverage is the closest and that still isn’t great. (Unit tests are important but you can have 100% test coverage and a 0% working product. In fact, this is how every project begins before the first line of code is written.)

No one gives a damn how many pull requests happen or how many days apart they are.

1) Does the software work yet?

1a) How much of it works?

2) Does it do what we want?

2a) .... everytime? Less? How many 9s?

Report on the things that matter. Track the things that matter. Everything else is actively harmful and distracts your teams from doing what matters.

Also, when you ship the software you haven’t finished, you’ve begun. It is absolutely still useful to keep metrics on outcomes and track improving them over time. It is absolutely not “too late” and any planning that assumes software is done after you ship it is flawed. Features can become done, software never is.


They're useful to measure in that _changes_ to them are indications that something might be happening. Use them as canaries.

For example, your velocity has changed. Why has it changed? Are you bringing new people up to speed? Have you suddenly hit some unexpected complexity? Your mean time to resolve issues has gone down. Are you releasing code with simple bugs you should have caught? Have your team started picking them up sooner?

Whenever something changes there's questions you can ask, the answers to which might well help you do better. Using metrics as an early indicator of change can help.


1) If you need metrics to answer those questions you have a larger problem than metrics can solve.

2) That's an argument for literally any possible metric. You could make an equally passionate argument that looking at and keeping track of free/busy times on calendars or soda consumption or how full the garbage cans are on the floor as an indication and early warning sign that something changed. It doesn't mean those are good things to track. Pay attention to the things that matter. Align whole teams and organizations around those things. Don't let them get distracted with metrics that don't matter.

3) Metrics are a double edged sword and influence behavior. If you're tracking the wrong things, you will get the wrong things.


People would like to have simple proxies for measurements instead of doing actual work. I totally agree all those metrics are useless and only real indicators are "is it working yet" and "are users finding it useful". Because I can make loads of pull requests that are perfect but are not working for end users, and have 80% test coverage for functionalities users never use.

Funny is that as project leader one should click through project as in using it to see if it is working instead of wasting time on finding proxies. If leader/manager does not know how to use system or never clicks through then he is useless...


Indeed. Plus, I would add that measuring the outcome (definitely a must-do, not arguing on this) is more about the product. The article was more focusing on metrics for the engineering organization, how it performs as a team. Could you say that your engineering organization is collaborating effectively, only based on the outcomes?


Yes. Divorcing teams from outcomes and their products is an anti-pattern. Focusing on metrics for an engineering organization not linked to the things that organization builds is harmful and will produce the wrong results.

Evaluating teams on outcomes will force them to collaborate to make something great. Evaluating teams on collaboration will just make teams do whatever it is you’re measuring.


A friend of mine was looking at the metrics for a project he was managing and saw that the bug rate of commits after 4pm was much higher. So then he introduced a new rule against committing after 4pm, resulting in a big drop in the bug rate. He chalked it up to less committing while fatigued and rushing to get things done to go home.


That's really interesting. Was there any drop in overall productivity/delivery pace as a result? There are definitely times when I wind up cranking out buggier-than-usual code later in the day/night to meet deadlines or get things done that otherwise wouldn't (at least, not on schedule). Sometimes when I timebox myself too much like that and it winds up dragging projects out days or weeks longer than they would have been had I not just blocked out a few hours to get the dang thing working and then cleaned it up afterwards; this is particularly true for complex projects where it can take a large portion of the day simply to build the mental context necessary to actually start executing on what needs to be done. If I cut myself off at a certain arbitrary time, the time spent building mental context earlier is totally wasted.


If I cut myself off at a certain arbitrary time, the time spent building mental context earlier is totally wasted.

If what you have to remember to make a certain change means you have to keep 27 things in your head at once to avoid making bugs, you might want to consider re-organizing your code so this number is smaller.


I think the title of this post is great, but the body doesn't quite deliver. The best content I've found on this subject is this book:

https://www.amazon.com/gp/product/0932633366

It talks about the different between informative and motivational metrics - the former being just for awareness while the latter being intended as explicit targets for employees. While it's easy to bleed the former into the latter, there's nothing inherently wrong with measuring game-able statistics as long as the incentive to muck with them is minimized. Easier said than done, but not impossible.

The article is pretty dismissive of story counting, which is a semi-popular approach to estimation:

https://martinfowler.com/bliki/StoryCounting.html

(Anecdotally, we had someone look at our teams' previous sprints and found that # stories / sprint was more consistent than points per sprint over the time frame they reviewed. YMMV.)

The focus on velocity-related metrics seems smart - Git Prime did some analysis on this and found that frequency of commits was a useful measure:

https://blog.gitprime.com/prolific-engineers-take-small-bite...

I don't see how these are any less game-able than klocs though.


Thanks for the book recommendation! Will definitely read it :).

Concerning Gitprime, I completely agree with you. And they do say that the point is to game their metrics: https://help.gitprime.com/general/cant-you-just-game-the-met...


Is Agile solving this problem now? or has Agile gotten out of fashion?

    * release often, release early, each release should function though with less features.
    * break large goal to small steps, one step at a time.
    * sprint every 2 weeks or 4 weeks.
    * 15-min stand-up every other day
    * etc,etc
I read some agile books in the past, and now I'm experiencing real Agile in sw development for the first time, so far I found it is quite effective comparing to my past projects, assuming you have some great scrum master and project-owner that are really good at the balance of arts.


If people actually do it, it helps a lot.

In my experience, if business stakeholders are seeing real value shipped to customers every week (or 2 if you must) then they don't get very antsy. Especially so if they are using a release-early, release-often approach to the product, where they see a project as an opportunity to explore what customers really want.

The problem starts to come in when the feedback loops are long. If they ask for something in January and don't see results in June, then they've got 6 months to wig out. And one of the ways they'll try to manage their fear is by getting up in developers' business by imposing all sorts of metrics.


in our case it's a long term internal project, it's embedded system development, we release bi-weekly internally before it reaches 1.0 for customers.

embedded product is notoriously hard to manage and I found Agile process actually helped quite a bit, via Scrum's 'story points' method.


First, Agile is a collection of ideas and principles, and not a monolithic thing.

Scrum (what you mostly describe) does not address this, really. Except that it attempts to reduce metrics down to a primary metric (there are others): Story Points. These are the primary concern of a Scrum team, and are developed by the team members as an estimate of the size of the stories (tasks) as they see them. These are not given by anyone else. These are not used by anyone else. The team then evaluates themselves (if they do this correctly) by constructing a control chart [0] to see how they're performing. If they trend down, they identify the cause and share the reason with other teams so that others with a similar cause can address it more readily. If they trend up, they identify the cause and share so that others can attempt to duplicate the success.

Scrum does not address how to do personnel performance evaluations so it doesn't stop management from trying to use metrics for the purposes described in this article. For example, story points (if used as an evaluation metric for personnel) become useless as people will just inflate their numbers in a continual game of oneupmanship. At which point a large amount of the value of Scrum is lost.

The Lean school of Agile is a philosophical descendant of Deming's work in Japan post WWII and the Toyota Production System (called Lean as it spread). This school of thought does address the issue discussed in the article in that Deming's views have you reducing your focus on individual performance and increasing focus on performance of teams and projects (title of the article is related to this). Deming and others recognize that the system in which work is performed effects the outcome more than the workers themselves (NB: His focus was largely on manufacturing, but this works in service industries and to some extent in our discipline as well). Since the view in that school is on the system, it does address the problems described in the article (if you get management to understand it). So by focusing on the project level (really you should probably go higher, department or even enterprise level) you can understand where problems are occurring, where there is waste worth cutting or addressing, etc.

That said, I do not recall seeing some of those points I made (gathered from Deming and other sources) in any of the books I've read on Lean as it relates to software. It could just be that I have forgotten or missed that information (may have glossed over it as I had already internalized it) or was in sources that I haven't read yet.

[0] https://en.wikipedia.org/wiki/Control_chart


> If you don’t include story points or some equivalent, this might be the most misleading metric you could use.

I'm sorry, but if you think "story points" is something useful, your organization is already doomed.

Story points is an inflationary currency. When your developers realize that somebody important in the company cares about said currency, they will make sure that their work has more of these "story points" in "sprint planning".


Maybe it's just because I'm more confident that I won't get fired, but I've not had any trouble telling the higher ups to stop asking about story points.

Story points are a way for your teams to improve themselves. If they don't understand that, then teach them. And never report the points (directly) to anyone above the project lead or perhaps one level above that. Any higher and they are too disconnected from both the team and the work to understand what they're seeing.

The only thing that higher ups should care about are trends up and down. Your team, after say 6 months, should be pretty steady in their performance if they're using story points correctly. Downward trends should be identified by them and their project lead early and addressed, and if it's an issue that may impact other teams they should communicate that up and out. Upward trends should be celebrated but also examined to see if some change could be duplicated elsewhere (an improvement to your build configuration resulting in faster feedback, adoption of a new development tool or technique, etc.).


> Story points are a way for your teams to improve themselves.

These points have nothing to do with actual value produced to the business, or the developer's worth / skill. It's also very easy for a developer to begin to inflate the estimation of tickets they believe they will work on, creating a false sense of confidence.

Alternatively, if people work at a "shop" where tickets are so fungible that people pick up whatever the top ticket on the board is, it tells me that they are working on low-value, repeatable things where expertise is unnecessary.

It's also not clear that increasing story points (even if they were somehow related to work / hours perfectly) would be useful. I've seen teams drop down to do minimal "work" for months and produce tremendous value because they were reorienting the product to do things successfully. An organization that looked at that labor input, sans objective outcomes in the product, would be doing themselves a disservice.

Story points, tickets closed, lines of code: ultimately all of these are a belief in the labor theory of value. But that's the thing. I don't want labor, I want value, and the tragedy that is input-focused measurement is just due to it's easier to do than measure product outcomes. It's always more convenient to say "We did 24 story points this week" than "Our users hate our software, and here's how we know it."


> These points have nothing to do with actual value produced to the business, or the developer's worth / skill.

This is true.

Story points don't translate to value. They can't. A nail could be worth 5 cents, or it could be a million dollars. Value is determined by the customer and their need.

Story points translate to cost. Customers determine the value. They examine the cost. And they determine whether the value is worth the cost. Is that 5 cent nail worth 5 cents or is it the nail that'll save the project?

Story points are about developing an effective and consistent estimation tool (how much will this cost or how much time will this take to produce). If your team is inflating their story points, they're idiots.

Assign a number to a story or task. If the team has developed consistent and reliable estimates (this takes time to happen) then this number translates with some reasonable fidelity to cost and time to complete the task. If the numbers are unreliable indicators of cost and time, then the team needs to examine their estimation methods. Perhaps bring in other people with more experience to help them.

The team (themselves, no one else) has a history of completing so many story points per cycle (sprint, week, month, whatever). So they know that if the customer has asked for an estimated (their estimate) 50 units of work and they can only complete 20 per month on average, then it'll take 2.5 months on average to complete the work. This lets them, you know, plan. And discuss with their customer the priority and objectives. If the customer is only paying for one month of labor, the customer will have to reexamine their objectives and priorities.

The customer never sees the story points. They see the cost and time estimates the story points turn into.

The managers never see the story points. They see the cost of the projects and the income from customers.

The only people who should see and care about the story points are the developers and the project lead. Maybe a statistician if they can't do the math themselves.


> Story points translate to cost. Customers determine the value. They examine the cost. And they determine whether the value is worth the cost. Is that 5 cent nail worth 5 cents or is it the nail that'll save the project?

There is absolutely no evidence that story points actually correspond to cost in a product organization (as opposed to a contracting billable hours situation), because the long-term scope of a project is unknown. When does a project actually "end" with respect to its cost vs value production? Is this accounted for via story points? Are all of those things known when starting the first iterations? I would submit the answers to these are "we have no clue."

Something that does however correspond to cost is well, cost. Team's have a known salary burn rate, and this is completely independent of how many imaginary points they make. Any organization whose value output is being measured where two weeks is actually such a large timeframe that it needs to be broken down into more granular estimates is 1) delusional about the predictability of their actual value output and 2) pipe fitters working on easy, well-known features.

There are many ways of estimating projects as a whole that are equally as (un-)reliable for estimation and take way less time, and these can be coupled together with financial rationalization from the business. If two week increments can't have significant variance for the business to make a profit, and this business is in the extraordinarily high margin software industry, they're working on the wrong problems.


Okay, how does this doom a project? It's just an arbitrary score to begin with, so what's wrong with inflation?


Because now "real" velocity goes down.

There's a negative Pavlovian reaction developers have every sprint when 1) "commitment" was not achieved, or 2) story points decreased. To avoid that pain, developers will pad points. This means that when things go according to plan, large sections of a sprint will have no work or people will try to appear busy.

The easiest way to avoid that gamesmanship is to not measure things that don't matter, which is all inputs. What matters is outcomes, which can be in the form of money, usage, satisfaction, etc.


I've only worked on an XP project once, but if we finished early, we either did some refactoring or pulled down another card and started on the next task.

You do need some time now and then for refactoring and that's how we did it. Not everything needs to be formally estimated and scheduled.

Story points are supposed be a tool to help management with realistic scheduling, not a way to push developers to work as fast as possible. If there's a lack of trust between engineering and management, it sounds like you have bigger problems.


> Story points are supposed be a tool to help management with realistic scheduling, not a way to push developers to work as fast as possible.

There is no empirical evidence that story points are more effective than other forms of estimation. (There's also no evidence that any form of work estimate is related to business value, which is the original criticism of measuring inputs).

Secondly, the perverse incentive around sprint boundaries exists and can create bad management practices. If management is actually looking at those estimates retroactively, it means that the team is being evaluated on the performance of their estimates, not on the timely delivery of value. If management is not looking at them--either in the long term or the short term at a sprint boundary--why bother in the first place?

Additionally, if it was routine for a team you were on to finish ahead of schedule, my guess is your teammates and/or manager was padding the estimates. In which case, we're back to what I consider a situation where both the company and employees lose because of disengagement.


Getting secure connection warnings.


Maybe the author fixed it - I see no such warning (FireFox 63), but what did catch my eye was a 3.6 MB jpeg that took 16.3 seconds to load (25 Mbps connection) -- admittedly, haven't read the actual article yet.


Oh let me check this! Sorry about that.


Fixed!




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: