I know lines of code as a measure of productivity has been criticized to death here and elsewhere but today I went and did some analysis on past two years of my team members contributions and saw the team members that I know they are productive are modifying code a ton more than others. It can be subtraction of code too but overall more changes correlated with my perception of their contributions. My perception is pretty informed because I'm their manager and I know this team and this code base very well.
Counting lines of code, commits, changesets or any other simple metric will destroy your culture.
The team _will_ find out, and then instead of contributing to the success of the business in earnest, they’ll be doing stupid things like maximizing their changesets or racing for “easy” large changes like deleting a module.
It doesn’t matter whether those values do or do not correlate with reality (IMO, if they do, it is for relatively junior engineers only). If you give off the smell of measuring people like that, you will ruin any collaborative team environment and you risk never being able to recover that.
There’s a good chance you’ll chase away excellent engineers with this sort of low-effort metrics management too.
Massive amount of code, particularly when written by a single engineer more often ends up being a liability, not an asset.
There are exceptions, but these are extremely rare.
Look for code created in collaboration by multiple engineers, with contributors or reviewers across multiple teams. This code is more likely to be an asset.
(Unless created by Google for external use or by xOoglers. Such code outside Google is a near guaranteed liability.)
This results in yet another opinionated framework in which that particular developer is extremely prolific. That developer then keeps making massive changes to the framework, deprecating and renaming parts every so often. At some point other engineers, who initially had been excited, get tired of the tyranny, rebases, rewrites and having to fix their functionality broken by framework changes. Then, the opinionated developer gets into disagreement with the management (that is getting more and more concerned by the constant churn and lack of new features). Being up to speed and fully convinced that he is the most prolific in the team, he decides that the management is unreasonable, easily finds another job and abandons the project. The codebase dies.
Sure that's one possible outcome if that developer is a poor leader.
The other outcome if that developer is a good leader and interested in developing a community is that a community builds around the core foundation that was laid and begins to take a life of its own.
Look at React. Started by a single opinionated developer who laid the foundations and built a prototype. From there it's taken on a life of its own and evolved. It was not designed/built by committee from day 1.
On the balance sheet I'd call functionality an asset and code a liability. Everything else bring equal more code is more to maintain as things change, search through for bugs, etc.
You're absolutely right. It's also true that counting lines of code is foolish and almost certainly counter-productive. But to your point, the argument is simple. People who ship lots of value will, more often than not, also ship lots of code changes (not necessarily net additions). The oft conflated group who ship lots of code, but little value, usually won't make it past code review (and, hence, don't actually ship all that much).
Contrary to some other comments, I'd argue it's the people who ship lots of value, but very little code, who are rare to find in practice.
Yes, for relatively small projects the above is true - software engineers who ship lots of value, but little code are rare. For slightly larger projects they are not so rare. Systems or integration engineers do less coding, but can make the whole thing actually work end-to-end. Engineers that change projects within the company can have drops in their contribution rates, but their wide knowledge can prevent many-a-mistake from happening.
Looking for healthy collaboration is the key, this is where the magic usually happens. Counting lines of code is a poor proxy for this.
> Counting lines of code, commits, changesets or any other simple metric will destroy your culture.
Counting them alone won't, but incentivizing people based on changesets will. IMO there are plenty of cases where counting LOC is useful to a manager. It's a discrete piece of data than can be combined with other data to make informed decisions as a manager.
It’s very high risk and low reward. I have heard of a very valuable employee quitting on the spot when they discovered their manager was doing this kind of analysis.
It has greatly demotivated me to learn that productivity was measured by commit count in one of the companies I worked in. Interestingly, I was the most "productive" engineer there at the time. Yet my code was garbage (no offence to garbage intended). This oxymoron showed me that there's no path for me to improve in that company and I soon left.
Nothing is just a discrete piece of data. If it even barely feeds into something that feeds into something that feeds into something that feeds into something that affects developers’ compensation, then it will get gamed.
According to Goodhart's Law, when a measure becomes a target, it cease to be a good measure.
For example, I have just written a Snowflake stored procedure that auto generates SQL code and then executes it. I am going to call this stored procedure from multiple places in my Python codebase. Now if my manager would look at the number of lines of code I would still write the stored procedure, however, rather than auto executing the SQL code, I would just copy and paste the auto generated code into my Python codebase increasing my LOC for that week.
> Now if my manager would look at the number of lines of code I would still write the stored procedure, however, rather than auto executing the SQL code, I would just copy and paste the auto generated code into my Python codebase increasing my LOC for that week.
Not everyone panders exclusively to optimizing grading metrics. I wouldn’t do that even if my manager looked at lines of code because I care about the quality of my work. I’m confident a lot of people share my sentiment.
You wouldn’t, but maybe some other people on your team would, or maybe some other team. Even if it’s just a small minority, all of a sudden you’re performing worse than them, despite doing an objectively better job. Hence, it’s not a good measure.
Sure, maybe if your manager passively mentioned looking at the stat, you wouldn't. If your manager brought it up in performance reviews as a reason you weren't getting a raise or being promoted, or suggested you might be laid off if you didn't get your LOC metric up, you would immediately change your behavior (or hopefully quit).
If you are already good, productive professionals, than what is the point of counting lines of code at all?
If you are trying to use lines of code to measure productivity, the only reason is because you think some members of the team are not being productive. Sure, the good and productive team members won’t try to game the metrics… but the bad programmers, who you are trying to find, sure will.
The good programmers won’t game the system, but that doesn’t matter because they are going to be fine no matter how they are evaluated. So what is the point of measuring lines of code? You aren’t gaining any new information.
LOC definitely positively correlates to productivity, it just isn't everything. One year I modified over a million lines, because we decided that we wanted to stored some generated data in a config file that was checked into git, rather than in the DB. If you can ignore certain commits like this, the correlation is probably even better, but use it as just one data point.
Another major confounder is that super deep investigations that fix major problems often have relatively trivial solutions at the end. But the effort to debug and fix them (that is largely hidden by conventional metrics) is enormous. The people working on these types of things are often your best devs (because you know they will actually get to the root cause), but if you are only looking at LOC you wouldn't know that.
No, it's not easy to manipulate. You're thinking in linear terms which isn't what lines of code changed is useful as. You can't use it to distinguish which of 4 different productive programmers is most or least productive. However it's extremely easy to distinguish the non-productive programmer in the group of 5.
And it's not easy to fake, because as soon as that developer start merging in a ton of bogus or useless edits to push up their numbers then everyone on the team, including the manager will call them on their bullshit.
It's just like shift times for workers in manual labor jobs. You can't tell which of the people who are meeting up 8 hours a day is most productive based on if they where there for 8 hours and 12 minutes or 8 hours and 25 minutes. But you sure as hell can easily see that if someone is showing up 4-5 hours on a nine-to-five job, then something is wrong. And if that person tries to "fake it" but showing up for 8 hours but spends half the time slacking off, but coworkers and managers will call him on the bullshit.
Let me commit node_modules! And automate regular updates. Baam I've got a huge loc count. True, I ha e to argue why I do it and I say for auditing and reliability and maybe win. Once that is through I reformat the code even so slightly (rewrap comments?)
And when I then do actual work in between I lean towards the solution giving me more lines. Longer variable names, more line breaks, ...
There are so many ways to manipulate that, one I know it's a key metric. Sure, it's hard to go from weakest to top contributor that way, but good enough to get away from last spot who is being fired. At least till all play that game. By then the project goes towards a wall.
So yes, for a one off case to identify engineers one should talk to it is a somewhat useful metric, but I would look at more business relevant metrics like amount of closed bugs or similar first (while of course they can be manipulated easily as well ...)
Creating bloated bullshit has even more value in terms of measured changes than the initial bloat because it also creates the opportunity to fix it up later for even more changes.
Not really. One can always keep refactoring and churning things around. Search-replace to rename some long function name into something even longer. If you’ve managed to enforce the 80 char rule, this would likely result in things linted differently. Boom, massive change. No one can object to a good refactor.
Your entire point revolves around the idea of coworkers ratting out and being smart enough to know. Not every place is so cutthroat and pro-employer this is a given.
The idea the manager would call out individuals is laughable too. Most managers barely know how to code themselves. Give it another few decades before your manager looking through commits is a standard, you're more likely to see alert tools be prevalent before that.
- patents written / rewritten / meetings with lawyers / issued
- mentoring of other engineers
- PRs reviewed
- talks given
- meetings attended
- RFCs / ADRs written
- customer calls attended
- quality of code (the original and best reason that LOC is bad.)
Et cetera, et cetera
If you are truly an engineering manager, unless all your direct reports are fairly junior, I sincerely hope you realize soon that LOC is a pretty terrible measure of engineer contribution.
I am old enough to remember when the industry finally came around to accepting LOC is a worthless measure of engineer productivity in the late 90s. I am not exactly shocked but dismayed that this conversation needs to be rehashed.
Number of tasks closed is a better measure than LOC and it is still a terrible measure.
They shouldn’t just “accept it”. They shouldn’t care at all!
Equivalent would be saying “our sales rep only said 2300 words today but that is ok, we can accept it he was visiting an old customer not cold calling like the juniors”
I think that this can sometimes be the case, but I think it's mostly due to a couple of factors that end up making this sort of analysis not very useful at a smaller timescale. First, tasks that end up requiring a large number of lines of code changes (e.g. a large new feature or a giant refactor) will be more likely to be done by people on the team who are the most experienced and successful, whereas smaller scoped tasks tend to be assigned to junior engineers or new team members. This makes the association between lines of code and productivity for an individual engineer a bit circular; if you're the one assigning tasks as their manager, the engineers you think are more productive will be assigned the larger scoped tasks, so of course they'll have more lines of code. The other factor is that summing up changes over a long period of time pretty much by definition ends up being biased towards people who have been on the team longer; even if two engineers made the same number of per unit of time, the one who will has been on the team longer will obviously been the one who has made more changes, and that's not even taking into account the fact that people's productivity should ideally increase over time given the increased familiarity with the codebase, the tooling, the problem domain, etc. If the longest tenured team members are the most productive, you'd expect them to have the most changed lines of code based on that correlation alone.
LOC modified is a fine rough estimate. The issue is that it's a rough estimate and shouldn't be used in things like who to promote or fire without deeper investigation.
Usually LOC modified correlates with actual productivity but there are lots of exceptions. Like if someone copies an open-source library directly into the repo, commits a formatting change, is working on a critical section of code, is working in a different language, is writing new features while everyone else is fixing bugs, has been told they are going to be measured by LOC, etc.
The fact that you actually analyze your team members' contributions, know the codebase, and measure productivity in more ways than LOC is a good sign. The problem is when managers exclusively measure LOC to gauge productivity and make decisions. While LOC and productivity correlate more often than not, I bet you can spot at least some cases where the LOC is not an accurate measure.
Maybe for junior engineers and that’s it. I want my seniors and leads mentoring, doing code reviews, designing, communicating with the business about impact and the like, training, and yes, coding, but not as much as juniors.
I think the other big factor that hasn’t been called out yet, is that a senior developer is more likely to be able to change the requirements entirely to write less code.
As I’ve become more senior, I’ve found ways to push back on product requirements that add little value for the users, but tons of complexity in the code. If I don’t follow the requirements exactly, I can usually write ~10% of the code. But you need to learn how to do that with lots of experience.
It takes the same amount of time, because I need to dive in deeper to understand the problem and how it interacts with my features, lots of planning and meetings to change the scope, but the final output might only be 5-10% as much code if we can reuse things that already exist in the codebase.
—
For example, product team wanted something to be bold that we had set up to be within dynamic plaintext. It would’ve taken a rewrite of that entire UI component to be able to make some bits of text dynamically bold. But I was able to push back and say “what if we just rearrange it slightly so that the bold is unnecessary”, and the designers+business were ok with that too!
If I hadn’t pushed back, my more junior teammate would’ve written 2k+ lines of code to support that requirement. But instead it was probably like 10 lines to support the alternative formatting. That 10 lines took lots of time to understand the requirements, but was much better for code cleanliness and consistency in the end.
Agreed that the more senior you are the less time percent-wise you will allocate to just coding but for strong developers this doesn’t mean your output drops compared to juniors, it drops compared to what you could do if you were only heads down coding. It’s easier in many ways to have higher output as you become more senior if you stay close to the code, know more about the code base, and can be much faster, all the best devs I’ve loved working with are like this. Many of the less productive senior folks that drop in their output and don’t code much (i.e. modify < 100 loc in a month as a normal occurrence) become architecture astronauts which is not imho what a senior dev should be doing as they will lose touch and eventually their guidance is less useful on the complex matters where they are most needed which in turn means their team and company impact is not what it should be.
I've worked with a ton of architecture astronauts, and while a bunch of them are just shit coders, you can definitely create new ones out of otherwise good coders with the wrong environment.
Those seniors that maintain their output are only able to do that as long as they maintain their knowledge of the codebase. As soon as you lose control your output goes from 10x to 0.1x in the blink of an eye. At that point your options are to look like a fucking idiot because your juniors are out-doing you, or to optimise for blowing smoke up the manager's ass. Easiest way to do that is large sweeping rubbish architectural changes that look good on paper.
Some people will take a stand, but most just play the cards they're dealt. Every great engineer I know has had both 0.1x projects and 20-50x projects. Knowing why that's the case is more important for a senior/lead dev than coding ability IMO. Get the best out of others and all that...
I wrote about a blog post about this: https://shubhamjain.co/2020/05/17/being-fast-matters. Basically, as your get better, you also get faster. It's an unpopular opinion but great programmers can often perform wonderful in a short amount of time. Hell, it took Linus just a day to start using Git for managing Git[1]. It's a product of experience, thinking clearly, and being knowledgable about good tools for the job.
People will retort: "But, but, but... what about pushing junk?" Well, if you've decent code review system how will that get merged? Pushing junk is often a less of a problem provided you've a good engineering culture. The best teams I worked with were just plain fast. The worst ones had plenty of engineers hiding behind meetings, reviews, and bike-shedding.
I think the key point is modified, not just net new lines. About 15 years ago I worked for a genius manager who had his underling write a script to measure productivity this way, and they forgot to account for removing lines of code. They were so pleased with the one guy who just loved to copy paste crap over and over, versus us who were actually refactoring or reusing existing code.
This can be approximately true on certain teams but falls over for others, like people working on diagnosing issues, security, or performance. Those jobs are engineering-related but generally require spending a lot of time thinking about code rather than writing it, so the contributions from it are not easy to see in a codebase directly.
> did some analysis on past two years of my team members contributions and saw the team members that I know they are productive are modifying code a ton more than others
Depends what you mean by "productive". If you mean "appear to be busy" then yes. But by using this metric you disadvantage people who measure twice before cutting once, people who think deeply about a problem and find a simpler way to solve things.
In my experience, more junior people tend to spin their wheels a lot, and out of that spinning comes out a lot of code. Code that is often broken and then is is fixed again, and again. And such people appear to be more productive and busier, at a first glance.
Fascinating take. Anyway, I’ll likely be contributing deletion of at least 1000 loc from a codebase I maintain, sometime in the next couple weeks. My rate of code change in the month leading up to that will have been approximately zero. A week of that was on the road. The rest was the real work, and quite a bit more of the real work preceded that week on the road. No lines of code changed for weeks on end. That work was instrumental in being able to make a -1000 loc change. Glad I work with people who communicate about the work being done rather than communicating with commit logs.
Lines of code that contributed to linting problems
Lines of code that contributed to security issues (as reported by the static analyser)
Average complexity measures
CI/CD build failure rates
Lines of code reviewed
Gitlab has plugins to generate reports like this, and if you have enough data (multi year preferably) you can use this as a springboard for deeper analysis of members of the team, I find all these metrics together (non taken as absolutes because there’s many assumptions built into each one) can give a good overview of individuals in a team
Build failure rates encourages frustration as long as you don't provide ways to run the exact tests in isolation locally, under the same rules as CI. Meanwhile, branch triggers and pre-push hooks trivialize this further.
Most of these metrics are strange given they are trivial to check pre-merge. Meanwhile, running these statistics pre-merge discourages small commits and sharing code prior to having ideal builds.
The moment this gets out is the moment devs will game your statistics and any benefit you gain goes out the door. And almost every one of these, even combined, is easy to game.
I cannot tell if you are aware you're setting devs up for failure or not. Half your mentioned metrics are tackled by hard forcing your devs to run branch CI prior to merging and giving them a slap on the wrist for not running CI checks locally pre-push.
If they are important enough to measure, why not hard force them and focus on the remaining metrics instead?
See where I said “none taken as absolute because there are assumptions in each one”
And
“Use these as a springboard for deeper analysis of devs”
These are only used as high level metrics to guide further analysis, we are keenly aware of the faults of these metrics - but the alternative of no-metrics or worse, “gut feel”, is slow and riddled with more problems.
It doesn’t mean a dev is bad, maybe it means there’s some tooling failure, or organisational problem, or personal problem they need help with. These metrics give visibility into anomalies, and their imperfect nature doesn’t mean they are useless.
There’s also like 20 more metrics but I’m on my phone and couldn’t recall them all right now
Also I don’t know where all of this nonsense about “forcing” to run CI/CD before branch merges and “slap on the wrist” - all our devs can run the test suites locally in docker and inside their IDE, nobody is forced to do anything in the pipeline severs, and also nobody gets a slap on the wrist for CI/CD failures, actually I like to see larger changes with more failures because it means the dev is trying and I consider this normal behaviour, your incorrect assumptions about our test environment and how we interpret these metrics sound like you’re projecting.
I’m not a manager, I’m a dev (well senior dev, that’s why I know about the metrics), but I am measured by the same standards as everyone else. When I became aware of then I viewed my own reports, and found areas I can improve on. I like it.
The metrics actually showed no surprises when I saw them, had I been asked to name the worst 5 devs, I would have named the same people that the metrics would have highlighted.
So what you're saying is that it's useful for devs to be able to view their own stats for this on a no blame basis, and you have used your privileged position of access to them to improve your own performance, but you don't let the other devs do the same because you want to keep using these as secret metrics to manage them. Do I have that right?
No, I was asked my management to review the reports and help them interpret them, I would gladly give all the devs access to the reports, but that decision is not mine.
I raised this suggestion with management, but since we had a handful of devs who were drastically underperforming we decided to help them out first with increased mentorship, in order not to embarrass them in front of their colleagues.
I don’t know why you’re assuming the worst and malicious intent, it’s a great place to work, especially when we have a good vibe and all enjoy working together.
I think what contributes to this positive environment is not employing developers who instantly assume malicious intent, and aren’t paranoid about being discovered as incompetent. These people are usually the ones most opposed to metrics, and also are the ones who the metrics show are the worst performers. Instead of objectively assessing the facts they attack the messenger, full of emotion and vitriol because of their insecurities and inabilities.
There are many factors that go into firing a dev, if someone needs help we mentor them and grow them, but if someone has a bad attitude, assumes the worst and is full of negativity, then we are very quick to fire them.
Collecting and then using metrics like these in secret and without transparency for those being measured by them is unethical. The decision may not have been yours, but it's one you've chosen to go along with and support.
Maybe your company is still a great place to work in spite of that. Or maybe it's only a great place for you to work, and your colleagues would feel differently if they knew about this. I can't say for sure because I don't have any further insight into your company. But that's also not the point about why it's unethical.
Nor is it having a 'bad attitude', assuming the worst, or being full of negativity to be opposed to the secret use of metrics in this way.
Ah yes, the ethics argument, completely subjective and therefore impossible to refute, the last refuge of the exposed.
I see the metrics as a chance to provide visibility and support, you see it as a way to “manage” the devs. I see a chance to help, you see tyranny.
I see my viewing of my own metrics as a chance to introspect with an open mind and a willingness to see fault in myself and improve, you see an abuse of authority and a land grab to get ahead.
Then you say “I’m not assuming the worst, or full of negativity” when clearly you are, you just demonstrated it.
So like you, let me fall back on the ethics argument:
I think it’s unethical for a software manger not to take machine collected metrics and to manage on “gut feel” and intuition, just as all gut feel and intuition are not explained as rationale to all decision making (thus are also “secret metrics” - i.e. collected without transparent knowledge of all reasoning)
I think it’s unethical not to have secret metrics and inform every person of every thought and data point you have, so there aren’t any secret measures. (You better call your bank, your insurance company and loan companies and marketing companies who are all collecting secret metrics on you, in the sense their rationale, data points, and algorithms are not 100% open and are therefore secret)
I think it’s unethical to distribute reports that might demoralise some devs without helping them out first, especially if we know some of these reports might look unfair, and we are only using them as springboards for further investigation, like I have mentioned many times in this thread.
I think it’s unethical to complain about ethics and not address any of the points in the above discussion with objective facts.
But ethical things are ultimately subjective, so our conversation ends there, we will have to agree to disagree.
I don’t understand this response, all of us judge each other all of the time on secret, unspoken metrics, that’s human nature. Our entire culture and society is partly built on what is not said.
Your bank is scoring you by machine, so is your insurance company, and loan companies, are their algorithms 100% open and publicly available? No. Because they are secret metrics. Your bank knows more about you than you know, and so do the loan companies, I know, I used to work there. The datasets the loan companies have are insane, I could see if you are in financial distress or not, the value of your house, whether you were a “complainer” or not, your favourite music, whether you’re a foodie or not, and heaps, heaps more, I think 250 attributes on each person in the USA at one company.
What difference does it make if some metrics are collected by machine and not fuzzy, vague, human intuition? I think it’s an improvement.
Yes the metrics inform the seniors decisions partially like I said above, and yes they are taken with a grain of salt, like I said above, they are inputs for deeper investigation, that’s it.
Here’s an example that happened just the other day: I noticed heaps of CICD failures for one dev and I reached out to him. He told me he couldn’t easily run the test suite, so I spent an hour peer coding with him and showed him how to use VSCodes debugger.
1 day later he called me up and told me that “using the debugger was a life changer” and he was thrilled to be working so quickly, the faster feedback cycle from code->debug had restored his enthusiasm for work.
I think these are excellent metrics for the engineering team to discuss at retro in aggregate.
But not so good for comparing Joe and Mo. Or even saying “how is Joe doing”. Unless the reviewer is very skilled in navigating that data without bias. Most people I have worked for wont be and will ruin the employee relationship by making some comment about how “this could improve” when the employee things “ok this is BS but better go along with it, and maybe i’ll open CV.docx tonight and give it a spring clean”.
Bullshit is the biggest turn off for me in jobs and I will leave a job with too much (some is to be expected and a side effect of getting groups to work together). Team leaders tracking my time and commits closely to judge performance is this. How should they judge? Best way is get people to pair often. You will know what they are actually doing that way or can talk to someone who did. Any real laggers will be found naturally, and then find out why and fix (might be onboarding sux for example).
People do hate paring I know but it doesn’t have to be all the time. Even an hour a day of someone jumping on to help will spread boats loads of useful information.
Because it's really easy to throw code at a problem.
OTOH it's hard to come up with an optimal minimal solution.
> Lines of code rewritten (changed N days after)
Means: Your devs commit unfinished, not thought out crap.
Or: Management is a failure because they change requirements all the time.
> Lines of code removed
More is better, right?
Because cleaning up your code constantly to keep it lean is key to future maintainability.
But could be also a symptom of someone with a NIH attitude.
Or, management is a failure because they don't know what they want.
> Lines of code that contributed to linting problems
If it's more than zero that's a symptom of slacking off and / or ignorance.
Your local tools should have showed you the linting problems already. Not handling them is at least outright lazy, and maybe even someone is trying to waste time by committing things that need another iteration later on.
Or, your linting rules are nonsense, and it's better to ignore them…
> Lines of code that contributed to security issues (as reported by the static analyser)
Quite similar to the previous.
Ignorance or cluelessness. Maybe even the worst of form of "I don't care", while waiting whether your crap passes CI.
In case someone doesn't know how to handle such warnings at all you have a dangerously uneducated person on the team…
Or, what is at least equally likely: Your snack-oil security scanners are trash. Producing a lot of the usual false positives (while of course not "seeing" any real issues).
> Average complexity measures
Completely useless on it's own.
Some code needs to be complex because the problem at hand is complex.
Also more or less most of this measures are nonsense. The complexity measure goes usually down when you "smear" an implementation all over the place. But most of the time it's more favorable to concentrate and encapsulate complex parts of your code. Hundred one-line methods that call each other are more complex than one method with hundred lines of code. But the usual complexity measure would love the hundred one-liners but barf at the hundred line method.
If there's someone who constantly writes "very complex" code (according to such measure done by some tool) this can mean that this person writes over-complicated code, as it could mean that this person is responsible for some complex parts of the code-base, or is consolidating and refactoring complexity that was scattered all across the place. Or maybe even something else.
> CI/CD build failure rates
This means people can't build and test their code locally…
Or someone is lazy and / or ignorant.
Or, equally possible, your CI/CD infra is shit. Or the people responsible for that are under-performers.
Or your hardware is somehow broken…
> Lines of code reviewed
How do you even measure this?
Just stamping "LGTM" everywhere quickly would let this measure look good. But is anything won by that? I guess the contrary is more likely.
Also someone who's constantly complaining about others code would look good here…
OTOH valuable and insightful code review is slow, takes a lot of time and effort but does not produces a lot of visible output. That's why I think it's disputable whether this can be measured even in a meaningful way.
There is only one valid way to asses whether someone is productive: You need to answer the question whether what this person is doing makes sense in the light of the stated goals.
But to answer this you need to look at the actual work / things produced by the person in question, and not on some straw man proxy measures. All measures can be gamed. But faking results is much harder (even still possible of course).
As with anything it depends. I'm definitely a code churn out machine (when I'm motivated at least), but I'm not going to say that's a good thing. I've seen others like me who leave a wake of tech debt, and others who don't. Who knows which one I am? People seem to like me though.
As others point out though, the moment something becomes a metric it ceases to be useful. Check out the book "The Tyranny of Metrics" for a breakdown of it.
So fixing indentation or spaces, changing case, renaming functions systematically are all things that can be done with a quick search and replace across the entire code base. I have a feeling the most experienced/"productive" engineers are the ones confident enough to make those changes, but that doesn't necessarily translate into more lines = better programmer.
That was almost exactly what happened to me a couple of years ago. The company hired a terrible guy with an architect title. What he did was some really simple 3rd party integration sending some transaction details to an accounting provider. Instead of reading from db, he decided to intercept those before saving to db coz some of those were not saved at all (only a problem for accounting, but most of the customers did not use accounting providers anyway). So, he spent over a year on that and moved on to another project. I took the project over and lots of complaints from beta testing customers. My manager and I spoke to him about it and never get anywhere. "Look, I can spend another year on this, and it might never really work. How about let me scrap his sh*t and start over?" I proposed. Months later, green light I got. So I deleted all his code solely for that part, around 5k lines and replaced with merely ~300 lines, never had an issue since then. Soon we hired a new Product Lead, the first thing he did was asking how many lines code we wrote in the past year. Errr, okay, so I pulled out stats from source control. Ouch, it was minus ~3k lines.
Most good writers will tell you that shorter is almost always better. You want to express your thoughts in the most succinct way possible. Review your own writing to remove as much as you can, then hand your draft off to an editor who'll try to cut even more.
And yet, many writers are still paid by the word, and many book contracts still specify target page counts. Because even though shorter is better, the shortest possible length of a given text still serves as a rough measure of that text's breath and complexity.
Is code any different?
Yes, paying or evaluating programmers by lines of code is going to lead to misaligned incentives. However, I'm not convinced the dynamic is any worse than for prose. Bill Atkinson spent a week removing extraneous code from QuickDraw; an author might spend a week cutting extraneous subplots from a novel. Word count is still a useful metric for the author.
Perhaps the difference is that non-writers tend to be more familiar with the writing process than non-programmers are with coding. If an author was working on revising a novel, no one would ask them how many words they'd written that week.
Or maybe word count measurements are in fact making everyone's prose worse and we should do away with them too.
I worked at a print newspaper. They have target word counts not because it’s a great metric because that’s literally the amount of space they have for article on the page. Same for magazines. There is a target number of pages in the issue. Any business that publishes in both print and online still has that constraint.
Online daily newsletters are designed to take so many minutes to read, which is calculated directly from the word count. So word counts often have utility.
Coding sometimes has space and memory constraints but often it works best to optimize for maintainable code, which might be shorter or longer depending on the case.
I think a key difference is that code needs to be maintained, often by someone other than the author. A great way to write a lot of lines of code is to write something you have to fix or update each week. On the flip side, you could spend weeks chasing a nefarious bug, costing millions of dollars, and fix it by changing one line. Put another way, the analogy with writing prose is a good one for "programming" as you do it in school where you are often crafting relatively small pieces of code and then calling them done, but is less applicable for "software engineering" where code lives a long time.
It’s my understanding that Tolstoy was paid by the page (or something like that based on how many words he wrote). It’s an interesting analogy to code in that Tolstoy’s writing is wonderful. His style is delightfully monotonous. It’s interesting to note that the incentives you’re talking about don’t have to result in bad code just different code (in the hands of a good enough programmer).
Many authors had similar things (Dickens was serialized with portions printed weekly/monthly) and they often read better if you read them that way. Reading a Dickens novel cover-to-cover is somewhat like binging a TV series; it's too fast and you start to notice repetition, etc.
The code is different in the sense that good abstractions and function definitions, when not used in excess, will severely cut down the amount of lines. However, given the same amount of time and not a lot of time spent on thinking about abstractions, odds are they will end up with fairly similar LoC counts. Just developer A will have written more features, whereas developer B will have done the same thing over and over, without having the same features.
That's where your example of writers should take hold too: good writers stop being paid by the word and start being paid by number of sales. The same should be done with devs.
That said, the entire thing gets tossed out of the window when considering significant portions of developers will overthink abstractions, fiercely hold onto overabstractions (which end up bloating LoC), have discussions with each other ad nauseam, slow down code reviews over petty differences, and more.
Also if you write an abstraction then your teammates use it, the measurement on either you or your teammates or both is going to be skewed whether you’re going for high or low LoC.
> And yet, many writers are still paid by the word, and many book contracts still specify target page counts.
That doesn't really mean anything, after all some managers still judge productivity by LoC. They both can be bad.
And at least IMO, using word counts for books is giving very wrong incentives. I've stopped reading non-fiction books because absolutely every single one i've tried to read over the years always *ALWAYS* gets way too wordy, spending entire chapters in what one could fit in a blog post. And i'm writing that as someone who also tends to err on the "wordy" side - both when writing and when reading (i.e. when in doubt, i think it is better to write more). But books tend to overdo it to fill those contract pagecounts.
Right, and that's why I said at the end of my post, it may just be that length is a bad metric in both cases.
However, I'd posit that if you went into a forum of professional writers and asked how they felt about word count measurements, they'd be a lot more divided than we are regarding lines of code. Or, to pick another example, NaNoWriMo seems to be pretty popular among aspiring authors, and that's all about the wordcount. Maybe it shouldn't be—or maybe there's something we're missing.
For example: it may be the case that lines of code is a useful goal during certain phases of a project—maybe while prototyping, for example. Just like how a target word count is useful when writing a rough draft, but obviously stupid while revising a draft.
Although, an author might set a goal to get below a certain word count during the revision phase—and that again might be a useful target for programmers refactoring code.
LOC is the refuge of the incompetent. At Apple, when Gil Amelio came on board with Ellen Hancock etc, one of the first things they asked for was a total LOC estimate for Copland. There was a lot of pushback because engineering knew it was meaningless but they said they wanted it anyway. I don’t know what they did with it.
The ex-NeXT management who took over from them never asked for LOC.
For me LoC is a good initial metric. I don't know why everyone refuses it outright. Yes, there're cases where someone will spend month just to delete 1 line and that would be productive. Or when someone will autogenerate 10k LoC spending 5 minutes. But keeping it in mind I still use LoC as initial metric when I need to estimate PR complexity or project complexity. I guess it should not be the only metric and it definitely should not be the metric which is used to estimate someone's productivity, at it's very easily gamed.
It's good because I need few shell commands to measure it.
It's useful for telling how much effort went into a software system, and how much it will cost to maintain it, but it's not useful for telling how much effort went into a task, even when it's not being gamed.
It also doesn't tell you how much effort should have gone into a software system. Spend six months writing your own message queue? 16384 lines of code. Spend two weeks wiring up your system to RabbitMQ? 1024 lines of code.
Right. I can see it as a relative metric. A 20k LOC project is probably less complicated (complexity, features, screens, whatever) than a 200k LOC project.
Not always. LibreSSL is probably smaller than some applications but much denser with complicated code than a simple system with lots of CRUD screens.
But it’s an easy to grasp metric. It’s also quite easy to figure out. It’s automated and doesn’t require weeks of study by specialists.
But as we all know, using it to manage productivity of individuals or teams can be heavily misleading.
imo those times where you spend a lot of time to fix something with one line of code are actually an indication that the code is pretty bad. It shows that it was hard to understand, that there was massive complexity in the system and that one line was probably additive and added a workaround or extra configuration of some kind. I've never spent a massive amount of time for ~1 line of code and been happy about the effort. If someone had only architected this thing better or explained it or documented it then it would have been trivial to add that line.
Sometimes things have subtle bugs that it takes a long time to track down. This happens less often when they're well written, but it still does happen. And when it happens, the bug fix is often just one line. It's a frustrating experience, but sometimes still worthwhile.
Also, though, I don't think that needing only 1024 lines of code to wire your system into RabbitMQ is any kind of indictment of the quality of RabbitMQ or of the code being wired into it. It would be even less of an indictment if it was only 128 lines or 16 lines or 1 line. But sometimes you need to grok a whole new paradigm; if you've never worked with AMQP before, you probably aren't going to be able to write those 16 lines in half an hour.
Similar comments apply to other domains with well-developed libraries for them. OpenCV, BLAS, regular expressions, LALR parsers, SQL databases, GNU MathProg, and so on: if you haven't worked with them before, it's probably going to take you a while to learn enough about them to apply them to your problem, and then you're going to produce an extremely small amount of code to actually use them. But that doesn't mean your time learning about them was wasted; quite the contrary.
It's usually a sort of shibboleth (or the opposite of one?) - somebody asking about LoC, especially a new manager, sends up alarm bells of "this person doesn't understand coding."
Because it's so obviously easily gamed. There are some very specific contexts in which LoC is actually useful. A codebase with 1M LoC is vastly different to deal with than one with 100,000, no matter the language.
Goodhart’s Law: when a measure becomes a target, it cease to be a good measure.
I imagine this is in response to the rumors that Twitter employees had to print out the code changes they did in the last month to justify their jobs. This, and stories like it, are what I thought of when I heard that.
What I remember reading on Twitter at the time, was that devs were expected to print out about fifty pages of diffs relevant to the last six months of their work. Rather than printing out all changes from the last month.
The printout part makes me question the idea that code was being counted. That part is easy to do with tools which Twitter undoubtedly has.
I've never worked a job where I couldn't print out 50 pages of diffs for six months, although I have the sort of temperament where I'd probably sandbag or quit if asked to.
There's little point in speculating on this until a clear picture emerges. I can't imagine a winnowing process for a new takeover that didn't involve some kind of portfolio justifying the worker's salary. Not at Twitter, which was notoriously overstaffed and had a reputation for rest-and-vest.
> Has there been any definitive reporting on this?
I'm not aware of any such reporting, just rumours.
But it seems obvious that since they scrapped together this list in just a single week, they can't have done anything much more complex than use some simple metrics.
Well, it is a damn good story. People might have gone overboard on 2010, but every 18 months seems like an acceptable rate, to get new people introduced to the tale =)
The difference between those posts and this one is context, since in the past week we've seen LoC being used as a metric to determine mass layoffs at a prominent tech company.
> since in the past week we've seen LoC being used as a metric to determine mass layoffs at a prominent tech company.
Do we actually know that? People were asked to print their code, and a mass layoff happened. But AFAIK, what feature of the code was judged (if any) is kept as a secret.
No metric can or should be used in isolation to evaluate productivity.
LOC is obviously very flawed, but to be completely honest it has been useful as part of a heuristic for figuring out where something has gone wrong in a department (after applying basic heuristics like removing packaged code, vendored stuff, etc.)
For example, if engineers are constantly committing and then deleting huge amounts of code (somewhat like this example) then it’s good to dive in and make sure you don’t have some issues creating that situation. This can range from product managers who aren’t validating their requests before engineers start working on them, to interpersonal issues such as unqualified juniors writing large amounts of buggy code and leaving it for senior engineers to fix.
These metrics can also be warning signs that something has gone wrong. If an engineer’s number of commits have plummeted and LOC committed has gone way down, it’s a sign that you should ask more deeply how their work is going. I’ve had situations where programmers were being pulled into 20+ hours of meetings each week and being given weird research assignments from random C-level executives that took away all of their coding time. I’ve also had situations where people weren’t getting along with each other in private and DMs, and the first warning sign was that their contributions stalled.
So while LOC and commit metrics can absolutely be misinterpreted in the wrong hands (e.g. as a proxy for laying off Twitter employees without trying to understand the context of their work) I’ve actually found them to be helpful as one of several early warning systems that something is going wrong with a team or project. Work with people for a while and it’s not hard to recognize their normal output and when it has stalled for unexpected reasons. Use that as a signal to start a conversation about how they’re doing.
There’s an old aerospace joke about how the cost per pound of airplane equipment has been continuously going up and at some point the only way to maintain the line is if some of the equipment weighed 0 lbs. Hence the introduction of software.
I'm not saying that measuring programmer's productivity by lines of code written is good.
This story, however, is not a story of someone who deleted 2k lines of code.
It's story of someone who wrote, let's ballpark, 4k lines of code and then rewrote them, which resulted in 2k lines of code left. Not sure how to account for that but let's say it's 5k lines of new code written in total which ended up as 2k loc.
This story does not prove that you are a great programmer if you write little code. It actually shows that even the best of the best (like Bill Atkinson) write a lot of code and cannot arrive at the ideal code at the first try. Writing the first 4k loc was a necessary step to gain knowledge that enabled him to refactor the code down to 2k loc.
For one of the most frustrating projects I ever worked on, my net contribution was in the vicinity of -1k to -2k LOC in a service that weighed in at around ~5k LOC in the end.
The guy who started the project came from an enterprise Java background, and his code was the stereotypical design pattern pileup. I only jumped in because he was having trouble actually wrapping it up, and, with every commit, I deleted a bunch of code and replaced it with just a small fraction of what was there before, and he reviewed the changes and agreed they were an improvement.
Yet, even though he genuinely agreed his code was overly complicated, he never internalized the learnings, and kept repeating the same mistakes over and over again. He was fixated on the idea that his coded needed to be extensible (without a clear notion of how or why) and kept adding abstraction after abstraction for every new feature he added.
So, yes. It is uncontestably true that you often need to write 4k LOC before you understand how to write the same thing in 2k LOC. But it's also very much the case that there's a baseline of skill you need to write those 4k LOC instead of a 10k monstrosity.
Oof, this sounds sadly familiar to me. One of the most damaging developers at my company had a huge line of code count. I feel bad because he loved coding, and worked long hours, and always added lots of unit tests... But the unit tests were massive copy-pastes of each other that tested the trivial stuff (setFoo(), getFoo(), check the result) and often didn't test the complex logic that was harder to test but more important.
He watched C++ YouTube videos by experts and preached practices to us all - some of which were genuinely good ideas - but he lacked the judgment of when to apply what. He made massive changes without seeking team consensus, never accepted feedback, and totally lacked the desire for concise code that seems natural to most good developers.
Sadly his manager was enamored with him and kept him around far longer than we should have. I still run across his code to this day, overcomplicating and getting in the way of needed refactoring.
Cleaning up his code has given me a couple of -2000 LOC days...
> He was fixated on the idea that his code needed to be extensible (without a clear notion of how or why) and kept adding abstraction after abstraction for every new feature he added.
Sad to say, usually the only way this lesson can be learned is through being fired. And not always then, but at least it is no longer your problem after.
There are any number of counter examples to most of these metrics, but I think the extreme counter examples are often in bad faith. You can’t convince me that a JavaScript engineer who consistently authors 1 commit a month is productive. At some point, you need to stop talking about stuff and actually build stuff
Of course you can’t determine the best person by lines of code (package.lock files lol) but you can often make a guess at the worst person
If they mostly are telling other people what to do to make the other people write better code, then yeah, they can be productive to the work of the company. In a good job, I will save more time by tweaking other peoples plans than any direct execution I do. There is a reason the best management is management by walking around, talking to people, finding it what is actually going on.
That’s a manager then, not a software engineer/developer role if your whole job is directing others and not doing any coding yourself, and I’d posit that if you are doing that for too long, you probably shouldn’t be tweaking anyone else’s code eventually since you will lose touch with the system if you aren’t coding and your advice shouldn’t be at the program level anymore.
Not if your job is only to make people better programmers, not responsible for product deliveries but responsible to code quality, code brevity, long term maintainability, that sort of thing. If you keep pairing with people and pointing out better ways of doing things, ways of doing things that they might have considered impossible but have some hackish solution of a category they haven’t used before, or of building things towards your desired long term plans but not specifically asked for by product/customers, the job is only code and coding, but also enculturation of a group into a common vision that can make for very high performing groups of developers.
I have never seen a job like that where someone doesn’t code a bunch themselves and they are still a useful member of the team in the long term. I’ve hired a lot of engineers of various seniorities and also never hired anyone in the role you described and don’t see why I would. I expect the principle engineers or senior engineers to provide guidance but also to code.
Seems like whenever I write a bunch of code, it's always bigger before I've completed the test/debug phase. I put this down to revisiting all the algorithms while finalizing the code, and making adjustments/optimizations where it makes sense.
But he didn't write -2000 lines of code, did he? He wrote a new algorithm that was 2000 lines shorter than the old one. So, in principle, there was a positive number of lines that could have been reported.
Of course, loc is a bad goal for productivity, but it can still be an interesting measure (similar to test coverage): If I count lines of code added, commits, or tickets closed, I cannot compare any two developers. But if I count that metric for the same developer, or the same team, I can certainly spot changes in productivity. (As every metric has its outliers it would probably be. best to combine all of them.)
So if I was managing a team and wanted to know if it would be beneficial to optimize test execution times, I could just rent some bigger machines and see what happens. I could also do a similar experiment with organizational aspects like the size and schedule of meetings or the introduction of a dedicated on-call contact for support requests, etc.
No, you can't. There is barely any correlation between the amount of new lines of code I wrote in a day and how productive I was that day.
As a senior technologist who bills his time as a consultant, I can tell you that I (correctly) get paid the most for days where I write 0 lines of code.
But also, my most valuable code was code where I spent a week thinking and writing maybe 5 lines of code a day, and my least valuable code was when my output was limited by my typing speed.
> As a senior technologist who bills his time as a consultant, I can tell you that I (correctly) get paid the most for days where I write 0 lines of code.
Maybe convince management not to rewrite their core system.
Or listen to the requirements from a new system and propose an existing tool that solves 90% of them. Or explain why the library they were going to use solves a different problem and how they'll regret choosing it.
Or talk to folks and figure out why my proposed new design for a system doesn't cover all the needs of the old one (and eventually that some iteration of it _is_).
Or sketch possible algorithms for a difficult problem on a whiteboard / notepad until I figure out a good-enough solution.
Or talk to stakeholders about their needs and concerns regarding a new system I'm helping design.
Or talk to customers. Or help prepare a pitch or customer demo.
It's very rare that a customer feels it's worth paying me rate for writing code. Though it does happen, and when it does it's usually very interesting code with a very interesting story behind it.
Alternative proposal: Count the number of commits devs are making. Since some devs make more frequent commits than others, for each dev review some fraction of their commits at random, and rate how substantial the randomly sampled commits are. Then multiply average commit importance by number of commits to get an overall productivity score.
Thoughts? I think the biggest counterargument is that it incentivizes churn like making a commit and then reverting it later.
Programming is the reification of decision making processes.
Therefore, a language that allows a programmer to turn thoughts into correct code faster is a win.
The size of the code as writter is only slightly interesting: almost everyone can type faster than they can think clearly. A more compact notation is preferred for reducing the delay between typing and testing, but not to the point of impairing readability.
Yeah, and that has another problem: developer who code fast but don't think things through would have plenty of issues opened against their code. They would probably be fixing them as well so their close count would be much higher than for a programmer that made the right choices to begin with.
You could then go with features only tickets but those vary in size and complexity. Now, if we would put complexity as a measure (on which the team of developers should agree) and then measure the number of issues on that implementation we might get a better measure of that person's contribution.
That said, once that would be in place, even that can be gamed.
I think there is no silver bullet here, as much as we would like it to exist.
>But if I count that metric for the same developer, or the same team, I can certainly spot changes in productivity
I can't imagine this working in any scenario. Even different tasks within the same project on the same day will involve enormously different amounts of coding, testing, and analysis.
> If I count lines of code added, commits, or tickets closed, I cannot compare any two developers. But if I count that metric for the same developer, or the same team, I can certainly spot changes in productivity.
No you can't.
Sometimes I will be working on coding related tasks for months, and then mostly on design tasks for months. I am not more or less productive in either, but I will close a lot less tickets and write 0 lines of code when I'm doing design-related work.
Same with oncall. I probably won't write much code but I can close a hundred tickets in a week.
Metrics are just a stupid measure of an engineer's productivity.
The first thing I look at when reviewing a pull request is the total lines of code added and removed. If the number is over a certain amount (it differs depending on the language and task) I will immediately reject it and ask for a one-on-one. Too many additions or deletions are a signal that something went wrong somewhere. Either the dev went off piste or the task wasn't properly specified. Or sometimes an auto-formatter has gone on the fritz and "prettiered" everything up.
I had the same thought. It’s one of those stories that we’d love to believe, just because it’s so incendiary. But the original source seemed to be an unsourced tweet, and it’s not a good idea to believe those.
I've been a developer professionally since the mid 80's, and working for other people since the early 90's, and never once have I ever seen LOC used for anything but code reviews where "lines per function" had an upper limit. Never heard of anyone using it, nor worked with anyone who encountered it as described.
Where are these LOC counting managers I hear so much about?
Of the several engineers I've managed at my startup, the ones who wrote the most code got the most stuff done. It's a correlation and there are exceptions, but it's generally the case, at least for new software. A project won't complete itself without a lot of code being written or modified. Although I think total number of PRs is possibly more useful as a metric than total LOC (assuming neither are being gamed).
This is sometimes true for early stage projects. There are exceptions, sometimes you can have an engineer creating a library or a framework without any need. An existing library would have done a better job faster, but that engineer just wanted to make a new one. As a result you can have a massive framework, known only by that engineer in which that engineer is very effective. But no one else is. And overall it is a liability, if compared to an existing framework.
In general, the more code you have the more code you have to maintain and massive codebase churned out by a single engineer is more often a liability, not an asset.
I know people who write quickly huge amounts of very poorly thought of code that will be buggy and will have a bunch of completely useless functions, reimplemented functions that already existed within the project and similar stuff.
You might be unaware that you are not such a great manager.
I like to call this "net negative development"! Valuable contributions to an established codebase can involve a tiny amount of code but a massive amount of understanding, so it's easy for a developer to clean up more in old code than they add in new code.
There must be a point where this doesn't hold as well - new feature projects or, especially, newer projects themselves - where adding code is more correlated with value.
Once on the interview in the part "what do you want to know about the company and our team" candidate asked me what kind of new code my best engineers are adding to the codebase. Almost immediately I answered that the best engineers here are mostly removing code, because it is much harder problem.
Gave myself an RSI writing -500 LOC in vi. Only discovered the block indent instruction partway through.
It was copypasta of a bad n² idiom for collision detection between two search criteria. So first I had to replace it all with a function, then change the function. Made the app actually usable though.
I would love love love to see a study on the correlation between lines of code (not net, total changes + and -) and co-worker assessment of aptitude for people in the non-inclusive band from Junior to Architect (aka, people who are really expected to write a bulk of the software).
My favorite PRs of late have come in at numbers like +280/-1257. (I replaced a lot of janky flask handholding with the equivalent fastapi machinery, to much joy.)
Isn’t reasoning things from first principles exactly how you learn?
I think what you meant was that people love learning lessons the hard and slow way instead of taking others’ word for it.
I’m sort of conflicted on this. Yes, it is faster to take an expert’s word for it. But after a couple of human generations of that, do we actually lose understanding?
I think we can only stand on the shoulders of giants standing on the shoulders of giants to some N giants until the whole acrobatic totem pole comes tumbling down. The game of telephone through the ages produces belief, not understanding.
Another option is making giants more giant, but that is an even slower process than one person learning from first principles: humanity has to find a way for our brains to learn those principles faster. Maybe that will take millions of years of evolution. Or maybe something like neuralink will provide a shortcut so it only takes thousands of years.
