> a P0 (Blocker) bug is a ship-stopper which has to be fixed before the next release can happen. P1s (Critical) are important but not something we’d stop a release for, while P2s (Medium,Low) represent pretty much anything that will only be fixed when our development team has cleared all P0 and P1 bugs
> It’s important to note that a bug’s priority shouldn’t be confused with its severity, which exists as an entirely different dimension... ‘Severity’ to represent a (somewhat) objective assessment of a bug’s impact to the functionality of the system. From Critical representing major crashes or hangs, to Minor functional issues or purely Cosmetic blemishes. While a QA engineer will classify a bug’s Severity at creation time, it’s the PM who assigns the Priority at the time of triage based on their knowledge of the client’s business and product requirements
I don't see the benefit in having severity and priority as 2 distinct dimensions. Given a specific priority, what additional benefit do you get from having a severity label as well? Based on the author's description, I can't think of any.
If the goal is to better communicate what "type" of bug it is, using an enum classification (eg, cosmetic vs crash vs data-corruption) would be more appropriate than a numeric scale.
Because as the quote said, it depends on the client requirement.
"if you click anywhere in the left half of the screen during the 5th screen all the settings get deleted" might be a very severe/critical bug, but if the client says "thats fine for me don't worry, but can you fix the wrong color on screen one where the logo is purple instead of fuschia that's not acceptable for my needs" then this one has a higher priority, even though it's much less severe.
Your job is not to build the perfect/best product, it's to build the one your client want and is willing to pay money for.
The mixup between priority and severity only comes from the modern era when so many product get made for internal consumption, or where the user is not not the buyer.
> "if you click anywhere in the left half of the screen during the 5th screen all the settings get deleted" might be a very severe/critical bug, but if the client says "thats fine for me don't worry, but can you fix the wrong color on screen one where the logo is purple instead of fuschia that's not acceptable for my needs" then this one has a higher priority, even though it's much less severe.
The problem here is the made-up definition of severity.. basically, you decided that crash or losing settings equals severe and cosmetic stuff equals not severe. Why?
I could easily give it a different definition. For example, a crash that nobody cares about is not severe. A small "cosmetic" slip up that can cause big damage to brand is very severe.
So I agree with the GP here; why do we need to have a definition of severity that does not align with the things that we care about when it comes to actually fixing things?
> basically, you decided that crash or losing settings equals severe and cosmetic stuff equals not severe. Why?
Because you, as a developer, are clueless regarding business needs, and thus are unaware of why "cosmetic stuff" might be far more important than the risk of deleting someone's user profile. For example, perhaps the color scheme clashes with the one used by your client's main competitor and therefore might leave him vulnerable to lawsuits.
> Because you, as a developer, are clueless regarding business needs
Then that's the problem we should fix. Instead of creating an extra database field to capture somebody's incorrect opinion, the people who understand priority should be helping developers know enough to have useful opinions.
> Then that's the problem we should fix. Instead of creating an extra database field to capture somebody's incorrect opinion, the people who understand priority should be helping developers know enough to have useful opinions.
The priority database field is how you communicate this factor, but there is a point where reasonable people can disagree and the organization needs a way to make decisions clear.
To draw out the example even more: you could have a $600K invoice riding on customer acceptance, when the person at the customer site who signs off won't do so until the customer's logo color is correct. While that crasher when you enter a non-numeric character in a numeric field of a new feature? "We accept the functionality meets the milestone so will sign off but we don't plan to roll it out until next quarter after we have begun training personnel on the new feature."
Sure, every good organization should want everybody, not just developers, to understand the customer's business and such but sometimes you just need to get it shippable.
Why? What is the business value of having severity as essentially a protest field logged by people who don't understand business impact? You are basically in all your points explaining the business value of priority which nobody ever disagreed with and then going "and that's why we need two fields".
> Why? What is the business value of having severity as essentially a protest field logged by people who don't understand business impact?
I don't understand where you could possibly get the "protest field" idea. Severity is an objective statement regarding the known impact of a bug as verified by a developer. It summarizes the technical impact of a software defect. Stating that bug X is high-severity because it crashes is not a protest, and just because the PM decides to give priority to other more pressing issues it doesn't mean you should throw a tantrum.
What is the 'technical impact' of a defect, and how can you divorce it from the user impact? How can it be stated objectively?
Crash bugs aren't bad because crashes are inherently bad, they're bad because they have negative user impact - if the program crashes and loses user context, or data, or takes time to restart... those are bad things. If it crashes a little untidily when it receives a shutdown event from the operating system... maybe not so much.
Same goes for performance issues, scalability problems, security flaws, even badly structured code - they don't have technical impact unconnected to their user (or business, at least) impact.
> What is the 'technical impact' of a defect, and how can you divorce it from the user impact?
TFA provides a concrete definition and also the method to classify bugs based on severity.
Severity does not divorce a bug from "the user impact". There is, however, also the problem of low-severity bugs or even tasks having low user impact but high business impact.
But that's a contradiction. Unless the users aren't important (and the business is another entity, e.g, a CxO that has clout and demand a fix for a thing that users don't care about).
It could be useful if the folk prioritizing things are dealing with non-specific complaints about the software being unreliable or not working correctly.
Databases are a very bad communications medium. So if that's the major way devs and and product people are conversing about issues, it's no wonder the devs lack sufficient understanding of business context to understand what the real priorities are.
I do get that people have all sorts of adaptations to dysfunctional working conditions. So if a severity field is one of them, fine. But I don't want people to mistake that for healthy collaboration.
Are they? That's how majority (all?) systems that are asynchronous work. The data to be communicated has to be persisted. I think asynchronous is a good communication method.
There's nothing to fix. Developers assess severity but the project manager defines the priority. As the PM calls the shots, they pick which issue should be addressed first, and obviously it's expected that some low-severity issues should be given priority over some high-severity issues.
In fact, the only thing that needs fixing is any potential misunderstanding on behalf is some developers on how low-severity issues shall have priority over high-severity issues.
Because a crash is different than a button rendering with the wrong color, and although priority assessment might change with time, a crash is still a crash.
It seems that a recurrent theme in this thread is that some developers have a hard time understanding that PMs might have solid reasons to prioritize low-severity issues over high-severity issues. It's like these developers are stuck with a mindset where the forest doesn't exist and only the trees they personally have direct contact with should.be worth any consideration.
Why set a severity if you're not going to use it? A crash is still a crash if you don't set the severity and just write that it's a crash in the bug description.
I know some people look at me like I have three heads every time I say this. But if a project is dropping so many balls that it's hard to keep track of them all, I think the real solution is to work smaller and close feedback loops faster, so the sheer number of bugs is not overwhelming.
> Why set a severity if you're not going to use it?
Your question misses the point entirely.
The point is that severity is an element that's used to classify an issue with regards to priority. Severity does not exist in a vacuum, and priority is given depending on context. Severity is an attribute, other attributes, that's used to determine the priority.
It's only used to determine it in a way that's divorced from the business context. If everybody understands the business context, that's no longer useful. Ditto if people are collaborating with actual discussion, rather than trying to mediate communications via a database.
> It's only used to determine it in a way that's divorced from the business context. If everybody understands the business context, that's no longer useful.
That's the point, everyone does not understand the business context. Nor are they expected to. That's the job of the PM it's his main responsibility, and it's the reason PMs are hired to manage teams of developers.
I understand some organizations work that way. I'm saying it's bad.
The point of developers is to make software that people use. So if we want to do our jobs well, we have to understand how we are creating value. Product managers may manage that information flow, and they may get the final say in decisions. But if they are specifying software in enough detail that developers can be 100% ignorant, then developers can (and should!) be automated out of that work.
I think this thread is interesting and kind of funny, because it reminds me of work where I maintain some systems that keep track of projects for PMs, and I originally thought my job was to make everything consistent. But there are a whole slew of ways to express the "closedness" or "openness" of a project, and the PMs have evolved conventions where they want to be inconsistent and resist all efforts to make it all make sense. You have a project status which may be in progress or closed or something else. You have a closeout milestone which may be in progress or closed or something else. And you have a time entry code which may be open or closed. But it turns out there is no simple way to make these consistent, because people use inconsistent combinations to express things...but it's hard to tell what.
You guys are missing the corollary that low-severity bugs being escalated to high-priority is the edge case.
The point is that severity is not ignored; it does inform the priority. Most of the time there may even be a direct correlation between severity and priority.
But other (real-world business) factors also inform the priority; while severity will never change in the absence of new information about the bug, those other factors may change frequently. It doesn't make sense for a PM to reread every single ticket and reassess each one's severity when adjusting priorities, when the developer can just determine that once and record it in the ticket from the start.
Complexity sounds to me like more of an implementation-level concern.
e.g. A bug might be critical severity if it wipes the entire production database, but low complexity if the fix is to delete one line of code. And maybe its priority is P1 instead of P0 because the customer said they'll remember how to avoid triggering the behavior but they really need that logo color changed asap for an important demo.
The point i was trying to make is that the severity hardly changed the priority, if its user impact is low. But then that means the severity isn't high either!
Where are you getting "hardly" from? In this example, it normally would have been P0 (release-blocker) but was downgraded to P1 (still the second-highest priority) because of a special consideration on the customer's end.
The point of severity is that it's an objective metric determined in isolation based on specific technical guidelines that any developer on the team can follow (such as https://www.chromium.org/developers/severity-guidelines). Whereas priority is a higher-level determination that isn't purely technical.
It's like the difference between body fat percentage and attractiveness. Any "engineer" (doctor with a DEXA scanner) can tell you your BF%, and attractiveness will typically correlate with BF%, but ultimately your "customers" (romantic partners and targets) decide how attractive you are. Not a perfect analogy (priority is still something you'd decide internally, not your customers directly), but hope that clarifies things.
Hiring good programmers who are not too far out on Asperger/ introvert scale is an issue. So you can fix it by letting programmers only worry about the tech part and letting PM prioritise things.
I think motivation will not be as high but it is a way to get things shipped and profitable.
Speaking as someone with Asperger's who considers himself both a good programmer and capable of navigating / leading cross-functional prioritization discussions, and who likes knowing the context behind his work: maybe you should re-evaluate your assumptions about neuroatypical people.
(...and if you're indeed in a position where you're responsible for hiring decisions or performance reviews: strike "maybe" from the preceding sentence.)
Agreed. Non-neurotypical people may or may not need to approach understanding the context differently than neurotypicals. But it's not like we're incapable of understanding the context!
I work in a niche field. Not so niche that there isn't a lot of money in it, but niche enough that we need to explain to all of our new hires what we do as a company.
For us, it is absolutely 100% necessary to hire domain experts to prioritize bugs and features. It's not a question of incompetent or dense developers, it's a question of things that are not obvious to someone who doesn't have tons of experience in the field.
It's a problem that I imagine developers working on Chrome, Call of Duty, iTunes, or Outlook don't have. You can hire recent college grads and expect them to understand what the software does, have reasonably good instincts how to prioritize bugs and put together the right user experience even if the description in the feature request is sparse on details.
By the way, I heartily recommend working for such a company. My company works very, very hard to retain people. Someone who's spent ten years getting used to the weird stuff our customers expect is far more valuable than someone with half the pay who needs someone to hold their hand through every single issue. Everyone has their own office, management is extremely permissive about the shit that doesn't matter, there's never deadlines or crunch time, everyone chooses their own work/life balance. (We're hourly, and the expectation is that you work more than forty hours a week and need manager approval if you want to work more than 60 for more than three pay periods in a row) If we want more vacation, we can bank hours and spend it on supplemental vacation. Everything's great.
Playing devil's advocate: Why does the developer need to know why one bug is more important than another, if the priorities are clearly defined? I.e., if the backlog manager sets the priorities according to the customer's/business' needs, then the developer just needs to know that the cosmetic bug has a higher priority than the crash bug, but they don't need to know why the priorities are ordered that way to accomplish their tasks. And if they do want to know for their own knowledge, they can just ask someone who understands the needs, without the need for a more complex set of bug report attributes.
I agree with your overall point I think, but I find that people in general just work better when they have some context for their task and its priority/relevance. Absent that, they sometimes -- consciously or not -- decide "this is stupid" and either rush to just check it off or slow-walk it by allowing themselves to be distracted by other things.
The person you are responding to is saying, yes, "cosmetic stuff" might be far more important. So it's more important! Why have another dimension of assessment where we label it less important? Why not only have the dimension of assessment that actually matches the clients' needs?
Like I said in my other comment, because it makes the difference between controlled and uncontrolled.
Eg "the color is wrong because we specified it wrong" and "the color is wrong because the app doesn't respect what we ask it to display" both ends up with the same bug (wrong color), same priority, but not the same severity because the second case is uncontrolled.
Severity is a dev tool, priority is a business tool.
So instead of a severity rating, you are saying severity is encoded in the language of the description? Using descriptors a potentially non-technical PM can understand unambiguously?
I'm not saying this is the wrong approach by the way, it's just interesting how people approach this differently.
It's not another dimension. It's a classification. Some issues matter more from one perspective but might not justify allocating resources to address them from other perspectives. To be able to do an adequate job prioritizing issues, you need to take in consideration their classification. You're arguing about a desired outcome without paying any attention to the process that leads you to that outcome.
Because there is expressed client needs and real needs. They say they care about this cosmetic thing now so it better be fixed. However you well know that if you don't get this other thing fixed soon internal politics at the client will mean they throw you out. Thus you fix the thing they demand be fixed now (it may only take a few minutes), but you ensure the other things are fixed in the next release even though the client doesn't know they care yet.
Sure, but you don't need separate priority and severity scales to do that: it's just one priority scale but you just assign the priority not entirely on the clients expressed needs but rather also factoring in your own assessment of their needs.
You don't need that, but you are not everybody. When you have a large organization having a simple way to capture this type of thing and make it clear what you are talking about matters.
Of course it does add complexity. It is the call of each organization which things are important enough to be worth the extra complexity and which are not. Maybe for yours it isn't worth the extra cost - there is nothing wrong with that - but other people have different needs and so they will have different answers.
In short, you are wrong in thinking there is a universal right answer.
Yes but what is the point of severity _and_ priority? Why not one field that's first estimated by QA and then updated by project manager when the clients needs are known?
So that they can be tracked independently and reviewed later. As I explain in a comment elsewhere, my company uses an app to generate severity, and it may not be adjusted outside of that. We can then track the number of low or high severity bugs in a delivery regardless of how the customer perceives the impacts of the bugs, using a more-or-less objective measure. We can compare that to the customer's view of the quality of the delivery by using the number of low or high priority bugs.
Makes total sense and my team does this as well. I think calling the value completely perpendicular to fix priority is hyperbolic. Fix priority should be some combination of severity, frequency, effort and stakeholder desire.
We have dozens of customers worldwide for our software packages, and each package is highly customised for each customer's business. The severity measure lets us compare release quality across customers using an objective measurement defined and managed by us. The priority measure lets us refine that comparison per customised packaged. Generally, a release with a lot of high-severity issues will have a lot of high-priority issues (since by a default an S1 is a P1, an S2 is a P2, etc.), but some customers have different requirements, different custom features, and some are just more fussy.
If a base release that was customised for multiple customers has an expected number of P1, P2, P3, P4 issues for most customers but a high number of P1 and P2 issues for one particular customer, off of the same number of issues in the base release as measured by severity, then that will stand out in our measurements and we'll dive deeper into that customisation to see what's going on.
(Edited mid-stream... accidentally submitted too early.)
FTA, severity reflects "a bug’s impact to the functionality of the system", while priority reflect the
"client’s business and product requirements".
The point of this system is that high-severity bugs might have lower priority than low-severity bugs if you take business requirements into consideration. Yet, this does not mean that severity should be ignored.
You nailed it. Stick with priority and the right stuff will get fixed. Severity just encourages more debate about what needs to be fixed vs. deferred. Inevitably you will end up with a list of defects which will never be fixed.
that's a strong argument for having a single priority... Even (generously, IMO) assuming severity represents a tangible thing like a threat to code quality or system stability/debugging, a developer should not be the one trying to balance those internal demands against a customers priorities. The important thing here is that devs know what needs to be done. Distributing that arbitrarily across two fields obscures that.
They're both measures of importance, but evaluated at different times by different parties. In the article's formulation, the QA engineer sets the severity during the initial investigation based on technical criteria. This is then one of the datapoints that the PM uses during triage to set the priority, which is the controlling value from that point in the process onward.
I guess the underlying source of confusion is that they're both measures of severity, but priority is severity from business viewpoint while severity is severity from the affected user's viewpoint.
A high priority bug that isn't fixed presumably has severe consequences for business (or it was prioritized wrong).
A high severity bug that isn't prioritized presumably has low consequences (severity) for business, but it probably really sucks for the unfortunate user who is affected.
Here in pivotal-tracker-land the product manager controls priority by ordering stories in the backlog. There is no need for a separate 'priority' field because that is implicit.
FWIW, there's also no explicit 'severity' field; the PM is expected to understand all the factors that make issues important and order the backlog appropriately. If you need more categorization, you can apply arbitrary labels.
Devs inform a ticket with a severity as a counter balance to bug fixing being purely complaint driven. The business benefits from having someone with a technical eye look at a bug and determine if it really is just a cosmetic issue or may allow system misuse - then the PM should take that severity into consideration along with client complaints when handing out priority.
If the color is indeed the one set by the dev/designer/... and it's juste not the right one, that's a controlled thing, it's not severe. The app is doing what you want it to, you just gave it the wrong instructions.
The crash on the other hand is uncontrolled. That's severe, because it's a case where the app is not doing what you want it to.
Severity is something for the dev team internally, priority is something for the product manager and business people that the dev team follows.
When you start mixing the two in a single thing, the dev specific needs are always what goes out the window first.
I think it's helpful to think of how bad a cosmetic issue could be. For example, imagine that it's a big launch of a product people will be using in stores worldwide and the cosmetic issue is that the logo now consistently looks like a sex act because of missing letter or development art left inside the release. Also imagine that that act is relevant to a recent revelation in the news about the CEO's personal life.
Which do you think the CEO cares about more, the logo or the fact that it crashes sometimes on an infrequently accessed menu?
> For example, a crash that nobody cares about is not severe.
That's a bad example, something not severe is not severe... can't argue with that. A crash nobody cares about can be quite severe though, there's quite a bit of crash that has been used to make some of the most important security vulnerabilities. When I tried to reproduce the Dirty CoW vulnerability over a VM in a school setting made the teacher VM crash multiple time, he didn't care about the crash either ;), yet that single vulnerability allowed me to skip 90% of the vulnerabilities he wanted us to try to find.
I think your comment point something important that you miss too, different client may have different priority, and you too may. I think severity is closer to YOUR own priority. The severity is the impact this MAY have on your business if it wasn't fixed. Like you said, a small "cosmetic" slip up may be important for our brand, thus severe for us. That "cosmetic" slip up could still not be important for all of our client, like over a one page made to sell, yet still be on top in both priority and severity, because it matters for the next client or the sale team, which does matter to you.
> So I agree with the GP here; why do we need to have a definition of severity that does not align with the things that we care about when it comes to actually fixing things?
You could say that about many information in a ticket. It is still informative though, maybe it's no longer needed, but at one point it did was relevant and in many case, still is.
A ticket with an higher severity is something that you need to put more time on to decide whether it deserve an higher priority. I'll go back to your crash example, why would you works on a crash that nobody care about, why would you see "crash" in the ticket with so many steps to reproduce and believe that this deserve an high priority when it never going to happen? In an ideal world sure you would know 100% of the system and know 100% of the tickets and can understands 100% of the impact of this kind of crash and be able to fit it well in the priority, but let be honest, that ideal world doesn't exist. You have a limited time that you can put on each ticket to decide their priority, you have a limited capacity to understands the impact and everything, you have hundreds of tickets to go through, but something that you can do is see that the high severity set by the developer means that you may need to put more time on that ticket to choose its right priority.
Sure once the priority has been set, does the severity matter? That's a much bigger question that I won't try to answer, probably not, but did it matter at one point? It did. Would you remove any information from a ticket because it no longer matter? I sure hope not.
> Your job is not to build the perfect/best product, it's to build the one your client want and is willing to pay money for.
So why have severity and priority as two distinct dimensions then? If customer impact is the only relevant metric for your process, why would a bug about which the client says "that's fine for me" ever be considered severe or critical?
If the only metric you care about is customer impact, why track two of them?
Because severity is usually a general measure (eg, to use an example from application security: XSS is usually a sev:med), but priority can be subject to a bunch of forces not known to the person who found the bug. You might find eg an exploitable dangerouslySetInnerHTML, but if that’s on a separate domain (in the authn and the origin sense), in some back-office page somewhere, it may not be particularly exploitable, the impact of exploitation may not be particularly high, and so may justify a lower priority. Meanwhile, lack of CSRF protection on an endpoint that half the internet is engaged with may be a higher priority, even though CSRF is typically lower sev.
Maybe the page with the bad bug is getting shut down next month anyway and we’re just accelerating that instead. Doesn’t make the bug any less bad: definitely makes fixing it less of a priority.
(In hindsight I should have used SSRF or SQLi or RCE as examples of the former style bug and XSS as the lower severity of the two to emphasize the difference.)
Well, good, but if customer impact is the only metric that counts, what relevance does severity have?
If you find a RCE vulnerability that never affects your customer, and how much it affects your customer is the only metric by which bugs are prioritized, what relevant information does severity add?
Either it affects your customer, in which case you prioritize a bug's fix by its severity (i.e. simply by how much it affects your customer), or it doesn't affect your customer, in which case you prioritize is by... how much it affects your customer. Either way, you end up prioritizing bugs just by customer impact, which is either measured by the bug's severity (if it affects a customer), or zero (if it doesn't, regardless of severity).
Just to be clear, I am absolutely not arguing that severity shouldn't be tracked, I'm arguing against the idea that purely functional customer requirements should be the only criteria used for prioritizing bugfixes and development tasks.
It's certainly true that "your job is not to build the perfect/best product, it's to build the one your client want and is willing to pay money for", as the parent mentioned. But it's also your responsibility to do the building in a sustainable manner.
Decoupling severity from priority is a great way to run a codebase into the ground. RCEs that are unexploitable today will be expoitable five years from now, after sufficient requirement changes accrue -- and at some point in the future, you're going to have to deal with five years' worth of previously unexploitable RCEs that you have to fix yesterday.
Edit: I've worked for several years on two codebases where customer impact has been the primary metric used for bug prioritization for 15+ years and 7+ years respectively. Both were absolute catastrophes. You could literally crash devices by blowing hot air on them (literally, there was an out-of-bounds access bug in the code that handled overheating alerts and they crashed after a few alerts). An entire team was there just to put out fires, virtually all of which were known bugs that had at one point been deferred because they had "no customer impact" -- until they did.
It's been a very useful experience. Disentangling these things is a very useful skill to have for an expensive, and highly sought-after on the market.
You need both because severity and priority aren't necessarily set by the same people who have the same information. They're (of course) not completely orthogonal: generally there's a 1:1 map between severities and priorities. I think we're in violent agreement: severity is also a close proxy for how much I think you should care about something. (We don't assign severities to hardening tasks, since they're, well, hardening, not bugs--but there are plenty of hardening tasks that I think are more important than vulnerabilities and should be prioritized sooner. E.g. "unencrypted EBS volume" is a finding I will totally let you ignore, but I will also be on your case every day if you're not using aws-vault.)
So, to rephrase: you need both precisely because that's often the way you can even start having the prioritization conversation.
We're in perfectly peaceful agreement, it's not even violent :-). As I mentioned in my post above:
> Just to be clear, I am absolutely not arguing that severity shouldn't be tracked, I'm arguing against the idea that purely functional customer requirements should be the only criteria used for prioritizing bugfixes and development tasks.
On the other hand, when you have multiple different users..
One of them sometimes suffers a crash that also loses all their work, and it really pisses them off. That's got to be bad, right? I'd like to call it severe based on its impact to that user.
But then the vast majority of your users have never seen that bug, and the bug is damn difficult to track down, so maybe you can't treat it like a stopper.
Then you are back classifying it by impact x probability... What is a different set of dimensions and make much more sense to use basically because filling those 2 dimensions tends to be easier than filling a severity dimension.
But that says nothing about why one would want 2 unrelated priority dimensions.
The severity is a technical assessment of the issue that the PM, who doesn't necessarily have a technical background, weighs against business needs to establish the priority. Once there is a priority, the severity has already served its purpose.
Why would you weigh the technical "severity" ("what happened?") against the actual severity ("business needs")? The severity, by hypothesis, has no influence on whether you're going to work on the bug or not. You want to weigh the business needs against how easy the bug is to fix, not against what happened as a result of the bug. "What happened as a result of the bug" is purely within the domain of business needs.
I'm really confused by all this arguing against simply having more information.
Is the information about the criticality of the bug (whether or not the app crashes) relevant to the business decision?
Yes, of course: "When I press the 'more' button in the developer's bio the app crashes" is a more important bug than "When I press the 'more' button in the developer's bio I don't see more text."
Might the criticality of the bug not be relevant to the business decision? Again, yes: "When I press the 'more' button in the developer's bio -- ok, stop, I don't care about this bug."
In some cases the criticality is relevant to the business decisions, in others it may not be. It probably usually will be. The app shouldn't crash, because even if it only happens to 1% of people, they're the ones writing reviews on the app store.
So the criticality is a big red "Hey! This should probably be high-priority!"-flag, but it's not the only piece of information the PM is going to use.
> I'm really confused by all this arguing against simply having more information.
As with all information gathering, you need to weigh the cost of gathering and storing vs the gains of having the information.
People are saying there are no gains from having this information; which, if true means if there's any cost, the cost is too high.
People are saying it is hard to determine, which means the cost to gather it is high, it may include concensus building.
In my personal professional experience, there hasn't been a whole lot of retrospective analysis on the bugtracker data. So information gathering that doesn't immediately help with fixing the bugs, or communicating which bugs need to be fixed sooner is time wasted. Use of non-critical fields in the tracker is spotty and inconsistent, so analysis would require an expensive data cleaning phase.
Sure, you can say I didn't work in healthy organizations if they didn't do this sort of analysis. But that's the root cause of most of the bugs --- there were never any formal specifications and so many things didn't meet the specs; plus occasionally we wrote crashy code.
> I'm really confused by all this arguing against simply having more information.
What people are pointing out is that you're not arguing for having more information. The "severity" is a marginal zero information on top of the priority. There is no context where you would want to use "severity" instead of priority.
So...
GIVEN that priority trumps "severity" 100% of the time,
AND GIVEN that priority is already being recorded,
THEREFORE "severity" has no use, and shouldn't be recorded.
What's happening in the post is that someone wondered why there were two fields, and made up a justification for the second field instead of realizing it was a mistake to have two.
You can see the mistake happening whenever people choose a word that ordinarily refers to the concept of importance as their label for "severity", while simultaneously acknowledging that "severity" doesn't affect the importance of the issue.
They're saying that severity informs the decision of how to set priority. I'm not sure why that's necessary since severity is implicit in the description of the bug.
> In some cases the criticality is relevant to the business decisions, in others it may not be. It probably usually will be. The app shouldn't crash, because even if it only happens to 1% of people, they're the ones writing reviews on the app store.
> So the criticality is a big red "Hey! This should probably be high-priority!"-flag, but it's not the only piece of information the PM is going to use.
The point is that PM may be inundated with lots of bugs and new features and other stories that all need prioritization before the next release. If they had a perfect understanding of all of them, they wouldn't need any metadata at all (not even "bug" vs "feature" -- just read the long-form description!), but metadata helps people make decisions.
High-severity bugs should generally be prioritized higher, except in cases where PMs have gained a deep understanding of the situation and realize it's not business-critical, and override that.
At the point that they set the priority, the criticality is no longer important, just as the "bug" vs "feature" tag is no longer that important once it's been prioritized and assigned.
> I'm really confused by all this arguing against simply having more information.
I think the argument is that it's not actually more information; it's just smearing out the same information in a way that makes communication and decision making harder. (Note I'm not completely convinced by either side here, just trying to explain.)
> Yes, of course: "When I press the 'more' button in the developer's bio the app crashes" is a more important bug than "When I press the 'more' button in the developer's bio I don't see more text."
Of course? Really?
I mean, the upshot for the user is the same - they wanted to see more info about the developer, they clicked the button, they didn't see more info.
An app crash on most devices isn't actually a particularly dangerous problem. Start the app up again.
Arbitrarily deciding that a particular bug is worse because it causes more harm to the software rather than because it causes more harm to the user is precisely the problem. The importance of fixing those two bugs is probably exactly the same (unless the app crashing causes data loss or corruption or is exploitable or... you get it. Of course, the same could be true of the 'no more info appears' bug - behind the scenes it might be starting an infinitely retrying string of HTTP requests to download the developer bio that gradually leak memory and drain the user's battery. Maybe that's really the more serious bug?)
There isn't some separate measure of 'severity' that independently makes the crash bug magically worse than the functional failure. There's only the actual consequences of the bug. For the user, their data, their time their resources, and your business goals.
Your job is not to build the perfect/best product, it's to build the one your client want and is willing to pay money for.
If you tell the client that they're wrong and that you should fix the critical bug first how they react is a really good test of whether or not they're going to be a good client. If they insist on fixing the lower priority bug first then you should try to stop working with them, because at some point in the future they're going to demand more unreasonable and stupid things that blow up in their face, and you'll lose that client anyway. Save yourself the stress of working with bad clients and get rid of them at the earliest possible opportunity.
If they accept that you know what you're talking about and give you agency to fix things in the order that you think they should be fixed, do your absolute best to keep that client happy because they'll be a joy to collaborate with for years.
> If they insist on fixing the lower priority bug first then you should try to stop working with them
I'm all for firing bad clients. Fire the crazy ones. Fire the ones that don't pay. Fire the ones who make ridiculous demands at the last moment.
But the client wants you to deploy a fix to the logo ASAP, you do it. That's the client's priority. As long as they accept the disruption to the rest of the work, it's no problem.
The two values are good to separate if they come from different people/groups.
A developer or tester might be good at specifying how severe a bug is: impact (does everyone get it, or just some users) multiplied by frequency (does it happen in commonly used functionality?) multiplied by a category factor (is it a crash, cosmetic, dataloss...) will give you the severity.
Some completely different person can come along and give this a high priority because of business reasons, and to prioritize fixing the bug compared to other functionality. Two bugs with the same assigned severity might impact two different markets where the business knows one is critical for the survival of the business, and one is already a safe market. Then it's a priority to fix it.
A second reason might be due to ROI (easier severe bugs are better to prioritize than difficult bugs of the same severity).
Yes exactly -- it's two different sources of information that one party is not likely to have access to. Severity from a technical-only perspective is important to have. The sense of whether something is easy to fix (which should be size) is also important.
Ultimately you could have a low-severity bug, but it impacts a really important customer who cares about it, so it's high priority. This is the most common case I've seen.
Severity also can impact whether something gets prioritized at all. If it's low-severity and the business side isn't there either, maybe it just doesn't get done. Whereas something with high severity but low business impact still probably needs to be done regardless or it will lead to bigger problems.
Priority is useful for... Prioritization. One can autoassign a priority for an enum classification.
I've never seen impact/urgency/severity as being anything other than different ways to say priority or overcomplicated ways to derive priority. Then again I've never wrangled an absolutely massive bug/ticket queue.
> I don't see the benefit in having severity and priority as 2 distinct dimensions
Working directly with PO's both "Technical POs" and "Product POs" I've found that creating a strong distinction between the two as well as what the two dimensions define has a significant impact on the quality of the product over time.
In the original post we see that both Priority and Severity are two different dimensions each representing a scale. This is what I think is causing a lot of the discussion in this thread.
Because both of these values are a scale we are creating a foundation over which we can get confused and we find ourselves needing to differentiate these two in some way. This comment is a good example of the confused conversations I would often see. All of variations on the same flavour:
> If the goal is to better communicate what "type" of bug it is, using an enum classification (eg, cosmetic vs crash vs data-corruption) would be more appropriate than a numeric scale.
To get around this in Agile contracts I've always advocated the following:
"Severity" scale. Severity of the issue on the user from 1-5. (For Jira users imagine this as your 'Priority' field renamed to severity.
1: The user is unable to complete the journey or part of the journey
2: The user is unable to perform an action, but can complete the journey
3: The user is able to perform an action but...
4: The user is able to perform an action but...
5: The user is able to perform an action but...
Of course, you can put in your own definitions. The important thing being that this is defined from the 'users' point of view rather than attempting to define the nature of the issue (crash, freeze, etc...). The nature of the issue can be discussed within the ticket itself.
The second dimension 'Priority' instead being thought of as 'Ranking' and is not a field which is added to the ticket. This defines the order in which the issues must be dealt with. Thus allowing a PO to appreciate the severity of an issue and also negotiate with the team it's relative important through ranking.
I agree. My experience with the issue is that, while the distinction may exist academically, the reality is that most people don't grok it. The result being that they are so frequently misused that, in practice, they are indistinguishable dimensions.
Basically, if what you are trying to do is too complex/nuanced for everybody to understand - don't do it.
I agree. To assign a priority, the PM has to know exactly what happens in context. "Severity" doesn't help with that or with any other stage of reporting, tracking, fixing, and testing.
I could see where a severity dimension could be useful retrospectively if paired with classification. For example, you could look back and say, why did we have a big spike in severity 1 security bugs in Q4? However, I think I'd rather not have the dimension available than be forced to set it on every single bug on the off chance it will help with analysis later. "I bet we can do cool stuff with this someday" is never a good reason to add process overhead. If somebody retrospectively categorizes and summarizes tickets by some dimension and proves that it yields useful information, that could be grounds for asking engineers to track it in the future.
As explained, the severity is the impact on the system. For example, a null pointer that causes the system to crash will have a critical severity.
But maybe this bug occurs very rarely and in very specific circumstances.
On the other hand you have also found another bug that makes your app use bright green Comic Sans. It is not severe as it is only cosmetic.
But for some reason, on your last release this happens every time the app is started.
It seems likely that the second issue will have a much higher priority and will have to be fixed immediately, while the null pointer can wait.
Severity is purely technical. Priority is a project/commercial decision (which is informed by severity but also other considerations). They generally do not align.
Usually the other orthogonal assessment in addition to severity is frequency. If the bug is unlikely to be seen during regular use then even if severity is high the priority might be lowered. This is usually how risk is managed in my experience.
Agreed. 1 dimension can represent the impact to an individual user, and the other dimension represents the pervasiveness (frequency) in the user base/market.
We have applied the same metric to feature request prioritization. Severity can be thought of as necessity for an individual customer (must have, should have, nice to have, could have, won’t have) and frequency represents an estimation of demand in the market.
We even created our own Jira plugin to calculate the product of these and use that number as a self prioritizing index.
I think ultimately it boils down to what you want to glean from the data. Not all crashes are mustfix, and some cosmetic issues really are mustfix. If you're trying to assess the priority of two bugs, both are p1, but in your world are marked crash and cosmetic, you would choose the crash. But it might be that the cosmetic but is related to a sponsor/third party/branding/legal issue, which warrants fixing immediately. Having a "cosmetic" tag on that doesn't help you, but a severity of 1 vs a severity of 3 on an edge case crash on shutdown _does_ tell you.
It's always a scrummaster type who gets back from some training and wants to add a ton of JIRA overhead like separating severity and priority. I've ignored it for 15 years now and it hasn't bit me once.
At my work, crashes and even data loss situations are often given low priority and high severity. That's because we just tell the user the workaround and he can keep doing his job.
Low severity and high priority can be:
- it breaks the software for just one person, but that person is on the critical path.
- if breaks users workflow. For example a keyboard shortcut that is used every second stops working.
- the bug is an easy, high value fix, so why wait?
On top of what others have said, one case where I've seen value in them being separate is to workaround problems in long gaps between deployments.
A bug might be Major or Minor but there is a high risk to fixing it. You might want to do it early in the development process, ahead of higher severity issues, so you get the most runtime on the fix before the next deployment.
Priority/Severity is a good, useful way of describing things.
Buying clothes detergent because you ran out and have no clean clothes is high priority, low severity. Booking cancer screening sometime this year because you're reaching that age and there's a history of cancer in your family is low priority, high severity.
Severity can give you a prioritization within priority categories. Fix the severe high-priority bugs first, then the less severe ones, then on to medium-priority.
Of course, scheduling is often dominated by other factors, but it gives you a default at least.
Lets say you have 10 bugs and they are prioritized in order from 1 to 10. Bugs with the priority 2, 3, and 6 have been given a severity of P0 (a ship-stopper, as the original post described it). Bugs 1, 4, 5, and 7-10 are not considered P0 bugs.
Given those priorities and their relative severity, that means the team is committing to deliver, at least, bugs 1-6 prior to the next release. That makes severity a pretty useful dimension to consider on its own and not just as consideration when determining priority.
I've never really liked the priority or severity fields on bugs.
"Priority" feels to me like a band-aid on a UI deficiency in many electronic issue tracking systems: No way to manually sort the bug queue. If I can manually sort, it's easy for me to quickly find a spot where it's directly between something I'd rather have fixed sooner, and something I'd rather have fixed later. Which is ultimately what prioritizing really is - anything else is just beating around the bush.
And I don't believe that "severity", conceptualized as a linear scale, is meaningful. Whenever I see "severity X", my very next question is "why?" The severity score itself is, at best, not particularly actionable, and, at worst, something other stakeholders will learn to use as a vehicle for squeaky wheel prioritization. Or an agent for semantic rot, as someone decides to try and make the severity scale meaningful by imposing a strict hierarchy on all the possible kinds of things that might go wrong. (Or, rather, all the ones they thought of at the time.) I'd rather see tags like "data loss" or "known workaround exists". They paint a clearer picture, and they're harder to game.
- Risk of too much discussion on exact ordering. I think to my uses of graphical programming languages vs text and having infinite places to put wires made me waste more time on perfecting it than the coarseness of text.
- Sometimes there is not clear priority between multiple tasks
- Sometimes you want to think locally. I might be looking at a ticket and I'd like to know the general bucket it is in without having to look at it in context of the entire table. I might also want to triage it and put it in a bucket without trying worrying about the exact order.
I think a potential is to combine the two. You define discrete priorities and have dividing lines in the UX. The UX can let you drag between buckets or even order within a bucket.
This is one of those spots where I think that Scrum gets things right. Having the ticket priority ultimately belong to a single person (the product owner) sets up a pretty decent incentive structure. It makes it easy for that person to figure out on their own exactly how precise they need to be, and where the precision is needed, in order to get their job done.
When I see over-polishing happening in the wild, it tends to be in response to "too many cooks in the kitchen" type situations. Well, that or procrastination.
I've considered whether a hot or not type system could be used to rate the priority of bugs / tickets. It would present a summary of two tickets and make implications based on which one the user said was more important to order the whole backlog. I'm sure there is a fundamental flaw somewhere but I like the concept.
My main issue with allowing people to define the ordering is that, in my experience, enevitably you have some stuff at the top, some stuff in the middle and a big mostly unordered heap in the middle.
For the record, Microsoft's TFS/VSTS/AzureDevOps/whatevername has manual sorting of the bug queue. So we can drag-and-drop prioritize our work queue. However, the feature is supported very inconsisently, at least on the version we're using on our system. Many important searches and views don't sort by priority.
Yeah, all issues in the backlog should be sorted by priority. But a separate priority flag is still useful to indicate critical bugs. A high priority issue means you should drop everything and deploy a hotfix ASAP. This is not communicated by just moving the issue to the top of the backlog. But no need for multiple levels.
- "severity" as level of importance the tester assigned it when they found the bug and
- "priority" as level of importance a developer assigned to it after triage.
That first piece of information is important when you need to determine which bugs to look at first for triage, but how important is it after that point? Can it not just be replaced after the developer's judgement?
In other words, could you not have a single priority field? The tester uses a heuristic to assign an initial priority (e.g., crashes are P0, cosmetic are P4). The dev uses this to prioritize which bugs to triage first, and once they've determined a new priority based on customer experience combined with app behaviour, they replace the old one.
If you really need to go back and check what the tester assigned, then I assume you can just use the "history" or "revision" feature in your bug tracking app.
Additionally, as suggested in a different comment, you can add a label for the bug's type if you feel that's important (crashing, lagging, cosmetic, etc.).
Perhaps the message here is that the app's behaviour in a vacuum is not the sole determinant of its priority. But then that should be the message, rather than claiming there is another metric which needs to be separately tracked when evaluating bugs.
I think the formula should be - severity * (how widespread is this) - ease of workaround = priority. So if any of those measures change (e.g. an easy workaround is discovered, it's determined that the bio page that is crashing is also almost never viewed) then the priority should be adjusted. Having just severity without a measure of 'how many people does this impact?' and 'just how badly does this impact them?' seems like it's missing part of the picture.
But I think it should be similar to risk calculation, multiply by the "impact" or "damage". A typo might offend a small number of people but have a big publicity impact
> when you need to determine which bugs to look at first for triage
If you can't at least do a rough triage of all the new bugs in one sitting, you're either not allocating enough time for triage, or letting yourself get way too bogged down with ancillary conversation during the triage meeting, or it's just time to scrap the product and go home.
If you can stay on top of your triage, then there's not really any need to worry about what order you do them in.
Honestly, I've found that testers are poor judges of priority. They get emotionally involved in their bug and don't really make a good judgement of the priority to the business.
That's why the split between priority and severity exists: The tester is telling you the subjective view of the bug as if a user encountered it; and the triage process is all about judging the priority of fixing the bug in this particular release cycle.
Testers are not supposed to judge priority, that's a business decision (that could be based on technical know-how) and as such should be decided by business people aka PMs
> Effective bug triage is an essential hygiene and success factor for anyone managing a software release.
I could not agree more. To some extend, it applies to all change requests. That said, there is no need to be too specific in your change request specification.
Where I work, we have setup only a few parameters to help us triaging several hundreds of CR:
- Priority: Low, Medium, High
- Type: Add (new feature), Enhancement (small change on an existing feature) and bug
- env: Production, Acceptance, Development, ...
In 99% of the case, this is completely sufficient. As said by others, adding severity does not add useful information on a bug triage (it may provide information on the bug itself). If there is a data loss involved but the client is fine with it, that would never lead to a high priority bug. If the client care about its data, then be sure the priority will be high. It means that bug triage really is priority assignment (no matter the severity).
In our workflow, only high priority bugs (usually hitting the production) are subject to hot-fixes. All other tickets go into carefully crafted milestones (a package to deliver). As all CRs belonging to the same milestone are delivered together, they actually all share the same priority which becomes the priority of the milestone compared to other milestones.
Google has this concept. P0 S0 is the most urgent. P2 S2 is “standard”. P3/4 S3/4 is pretty much code for “this will never happen”. 99% of the time Pn = Sn. When it doesn’t it’s almost always off by 1. So practically I never saw the point in this distinction.
It's not a bug report if there's no one consuming the backlog. It's a wish list.
In my experience P3's capture tech debt, cut features, and design flaws. Those should be addressed for what they are, not shoehorned into a bug prioritization system that will never even glance at them.
There is certainly a difference beween the two, but to say that they are completely different things is going too far, as can be seen by some of the examples used to justify that claim - e.g.:
Something that is Critical severity doesn’t necessarily mean its a P0. That crashing bug in your bio screen? Face it, nobody really cares to read it and few will ever click the button, so its not something that should be a P0 ship-stopper.
In what way is that an issue of 'critical severity'? The quoted text contains an explanation of why it is not.
The real difference is that severity is a measure of the harm done, while priority is a plan for to responding to it. Clearly they are different, but clearly they will likely be correlated, at least when the severity is high.
> In what way is that an issue of 'critical severity'? The quoted text contains an explanation of why it is not.
The only reason would be this: given a large enough enterprise you have one person categorizing a bug as a crash and another making the call on whether that crash is actually worth prioritizing, a decision made on another day at another pay grade in another time zone. In this case it might be clear, but in the next case it might be "the configuration page crashes if the region is France". I as a developer don't know and I don't care that we haven't even launched the product in France so I just want to mark the issue as a very severe crash. Somewhere above my head the can call this low prio because the product launch in France is still a year out. This is enterprise backlog massage for enterprise workflows.
This is flirting with redefining 'severe' as something it is not. Severity is about impact on the user.
In your scenario, you don't know whether it is severe or not, so you just want to provisionally mark it as severe, and have someone with the appropriate domain knowledge make the final call. Until that happens, decisions made on the basis of that provisional call could be bad decisions.
And what might those decisions be? Priority, for one! To say that the severity should stay at 'severe', and the adjustment should be made only in the priority, is to confuse the two distinct concepts. I see a lot of that going on wherever data are used.
If you want to characterize bugs by the sort of technical outcome they produce, then do just that, instead of trying to bend 'severity' to try and encompass it, while actually hiding the issue that you wanted to characterize on. Again, while technical outcome is an aspect of severity, they are not the same thing (and, FWIW, crashing is rarely the worst thing that could happen (unless the domain is vehicle control.))
No I have the Domain knowledge and know that it’s a crash that occurs on odd weeks in France. I noticed it while developing and I can confirm it in the code. This to me is exactly as severe as the same crash in the US (as far as I’m aware). I must assign a high severity.
However I have little to no knowledge of the business (markets, release schedule). I’m not even aware that we have zero users in this region. A manager I have never met assigns a low priority, two months later.
Domain and business overlaps, but for example if one doesn’t mark this “severe” because the business hasn’t launched for affected customers, then a year later when it is launched in France you have a low severity bug that should be high.
The priority can be constantly re-evaluated against business needs while the severity is fixed.
Yes this is a simple technical characterization. It’s not useless to separate the business-stopping bugs from the cosmetic. And importantly: It’s not my call to say what is important.
Someone might say that the font size is more important than a deadlock. I don’t care and I don’t want to care. But I’m rather sure that those that do make those decisions want to be able to sort the bugs I wrote by a severity.
There is no "severe to you" as a dev in this sense. Severe to you is when your IDE is not working and you can't work on the priority tasks.
The crash has no severity to you. It is either severe or not to the client. If you don't know whether it is severe to them, sure you can guess that a crash in france is severe, but that's an incorrect guess, not an independent dimension of severity.
There is no "techical measure of severity", everything is goal oriented. It's not severe unless it's important to the client, or is impeding something that is important to the client.
In that case, I don't see this as very relevant as a counter-example to my original point, though you haven't made it clear what your organization's concept of severity is, and why this example counts as such. Does your definition convey any information, to the person setting the priority, that is not redundant with respect to the other information about the bug that they must necessarily take into account when making that decision?
Do you call a bug that corrupts data more or less severe than a bug that crashes a program? In whichever order you put them, is the severity always necessarily in that order?
>I’m rather sure that those that do make those decisions want to be able to sort the bugs I wrote by a severity.
Why would they do that, if, as you say, "It’s not my call to say what is important"? In what way is this not you signalling that you suspect, provisionally, this has a severe impact, as I suggested in my previous post? I think you are being a bit inconsistent here.
The > and < comparisons have no meaning for complex numbers. If you don't have a way to determine which (P + iS) is bigger, then you can't sort your bugs.
In other words, by adding a severity dimension, you lost your ability to order your bugs in any ascending or descending order. That defeats the whole point of having a priority field in the bug in the first place. Now you need to have a person decide if your [P1 S2] bug should be solved before your [P2 S1] bug by some arbitrary mechanism. And once you've made that decision, you'll communicate it with "Do this first" - which is essentially a priority field that now exists outside your system of [P S] classification.
Get rid of severity - or treat it like an enum that will help inform you what your priority of each bug is and bring sanity to your system.
I think this is a feature more than a bug. For the most part, sorting by priority then severity seems like the best way to approach things, unless something is extremely obviously a quick fix.
That said, the P-spectrum may have some crossover, especially at P1-P2, depending on the release cycle. For example, crashes should be focussed on a little more for always-online software, whereas for more glacial, manually-updated software, prioritising incorrect behaviour or functional issues may be more important.
This is confusing. It amounts to saying ‘bugs have an inherent property called ‘severity’. We explicitly track it in our system. It has no effect on how we prioritize work’
If there are people on the team who insist on objectively classifying bugs with no reference to customer impact, you don’t need to indulge them by giving them a meaningless drop down to play with in Jira, you need to get them up to speed with understanding your product and release priorities.
My company's measure of severity is measured by impact to the end user as well as to the customer (retail enterprises). However, the customer's priority may vary from the severity based on a number of factors.
> we use the JIRA field called ‘Severity’ to represent a (somewhat) objective assessment of a bug’s impact to the functionality of the system
> crashing bug in your bio screen
The examples makes it clear that they use "severity" to map types of bug to severity level in a completely opaque way. Regardless of the impact of the bug, "crash" -> "high" and "typo" -> "low". A crash is high-severity even if it has low-impact in every category ("functionality of the system" included), but a typo is low-severity even if it is high-impact in an important category? Why?
> a misplaced comma might seem like nothing worthy of emergent attention, but what if it completely re-purposes your brand statement
This just sounds like a high-severity bug: it may be low "functionality of the system" severity, but it is high "brand identity/loyalty" severity.
I think it becomes obvious that there is not much benefit of a special field for "functionality of the system" severity or even the underlying bug "type" that it seems to be representing. All severity is used here as a triage-heuristic anyways. Great. Now you can freely forget about it and find a better triage-heuristic for your QA team so that they can assign the priority more accurately.
Unless I'm missing something no one has mentioned the effort to fix a problem? I've seen cosmetic issues (using the wrong masculine / feminine form in French place names for example) which would have taken a huge amount of time to fix correctly and internal server errors which where the result of someone accidentally including a comma after a value in python, turning it into a tuple where one was not expected. At some point that should be factored into the order of priority.
The effort may indeed affect the priority, and it also is potentially a factor in severity. If a customer knows that a high-severity issue will take a long time to fix, they may decide on a mitigation for a particular delivery and lower the priority so that the fix comes in a patch or later delivery. The severity measure still counts in determining the overall quality of the release, though.
I work on an embedded system. We had found a major bug and labeled it as "stop ship" because we didn't want anything more getting out. It is better for the factory to build nothing (paying all the assembly line workers to stand around if required - though more likely build a different product) than to have one more customer see this product and encounter this issue. I don't know the details on this issue, but there have been software issues that could kill people in the worst case.
Anyway, the developer got this stop ship the day before Thanksgiving, so he worked all weekend and Monday morning turned in a fix. At which time the factory manager told him "thanks, but one of the parts to make this machine is on back order for 4 months so we weren't planing to make that product anyway".
I think the issue here is that "Severity", as used here, isn't a good label. It sounds like they're mapping the kind of effect the bug has on the system to "Severity" values. E.g, a crash and hang maps to Severity "critical" and typos map to "minor".
The problem is that "Severity" is being used for something that doesn't really match its intuitive meaning.
Maybe call the field "kind of impact" (or something similar) and have enumerated values or tags like "crash", "hang", "typo", etc. Now the field is intuitive and you don't need to retrain (or write blog posts) to get people to understand your new meaning for the word "severity".
Severity is how the reporter sees the problem: if it interferes with their particular use case, it's of the highest severity. Almost all bugs end up being of the highest severity.
Priority is how the developer (or, more likely, the project management team) sees the bug. Highest priority bugs are the ones in the project manager's spreadsheet and ended up there because of the most vocal complainers (paying customers, an inconvenienced CEO, etc). The rest will never get fixed.
It's impractical to use the severity of the bug to rank the priority, because they're not strongly correlated.
My gut reaction to the headline and the article was that it's a mistake to make such a distinction. However, if you look at QA/Support owning severity, and PM owning priority, it potentially reduces conflict--what do you mean this crash isn't critical?!
I think they are highly correlated, but they occasionally diverge.
I've preferred Urgency and Impact as my terminology, rather than Priority and Severity, respectively. But yes, I've been frustrated that these concepts are often combined when they shouldn't be.
This may be a controversial opinion but I strongly disagree with using bug priority/severity/urgency/impact etc to do work prioritization.
I think there are two separate things about tickets: one is the about technical merit of a ticket: documentation, evidence, reproducer, quick info about context. Basically, a ticket as a tool to gather and sort "technical discussion". This is fine and encouraged.
Then there is the "priority" - I don't think is a feature of a ticket. It's a property of context - do users complain _now_? Is the fix risky? Is the boss shouting? Do we lose money? Does the PM want this work done for personal reasons? How much time will it take for me to do a context switch to start the work?
I don't think it's possible to add this "priority" (as defined by social interactions) to a ticket. I always found I prioritize tickets that I understand the social context of. A colleague nagging me by walking to my desk is 100X the priority for me than "P1 critical" written by some PM on the other side of the planet.
The point: I try not to fight with the social aspect. If you want me to do a ticket. Good. Come over. Do a hangout and explain the context. Setting a mindless value in some field on some random badly described ticket is not going to make me de-prioritize other things.
My take is not to split priority and severity, but priority and tags. For example P0 typo,UI and P2 crash,UI ...the crash might be P2 if if's not severe for the product itself, as in the example, if it is on bio or whatever page.
Tags/label have been working quite well for me. Priority can be replaced with milestones, but you have to create them for each releases to ensure P0 are quickly enough.
Google's internal bug tracker has both priority and severity, and it's kind of a running joke that severity is meaningless (because it isn't meaningfully different from priority). On my team for example we leave it at its default value and ignore it completely.
Severity and Priority are dependent variables. You can try to treat them as independent but eventually it will catch up with you.
Where you see companies get into trouble is when an unlikely high severity bug actually happens (or becomes likely) and it comes out that they've known about it for a while. It's also the sort of thing that's a source for employee burnout.
My old standby for Severity is a 5 point scale based on data loss. It's a little fuzzy and everyone has to agree to definitions, but as a first proposal I'd say 5 is unrecoverable loss of pre-existing data. 4 is losing data entered now, 3 is loss that the user can work around (eg, enter the data via this other workflow and it's not lost), and honestly what constitutes 1 and 2 is nitpicky, because most places can't even keep up with all of their 3's, so arguing that your scenario is a 2 instead of a 1 is a bad expenditure of energy.
Save that energy for the fact that the business is going to rate 90% of all work as priority 1 or priority 5, sometimes re-ranking them after the fact. Especially for perf or security issues - always lowest priority until we don't have it, and then blame the engineer in the grand tradition of, "Why did you let me eat this whole box of chocolates?"
If people's first impression of what you want your UI to mean is different often enough to cause problems (for example see the amount of debate here), that is the bug, not the people misinterpreting it. The use of these terms in Jira is one of my pet peeves, since different teams often use them very differently and can be real fun when sharing Jira boards with multiple opinionated companies.
The colloquial usage of 'severity' and 'priority' just has too much overlap. Something like pairing 'likelihood' and 'severity' assessments, as is standard in safety, would still be general and make it immediately clear why something could be severe yet low priority, especially since people commonly mean both likely and severe when ranking a problem as severe. Keeping with the author's definition of severe, renaming severity as 'System Impact' at least immediately narrows down what is severe, but still carries the possible implication of 'frequent'.
I think fighting for a given interpretation here without using different terms is akin to insisting that people simply understand that a 'significant difference' reported in science means 'statistically different' as opposed to an 'important difference'.
High priority, low severity: Your program at one point displays an uninitialized value, which consistently shows "BadDon". You are conducting a demo tomorrow that is make-or-break for your startup. You're pitching to a guy named "Don".
Low priority, high severity: Your ICBM will experience an uncommanded launch when Feb 29 does not exist in a year divisible by four.
Severity is subjective: those users who are not affected by a bug find it to have low severity (possibly zero). Those who are affected by it, and have no workaround find it a showstopper (high severity).
The consequences add to the severity. Not being able to save a document versus having your identity stolen versus an aircraft crashing are different severities. Severity fields that have four or five values can be difficult to use due to the orders of magnitude differences in consequences.
Since severity is experienced differently by different users, if only a single severity field is attached to a bug, it is somewhat tricky to use. A guiding principle should be that the bug reporter should imagine the circumstances of the "worst-case user": the user who is most likely impacted by the bug, with the worst consequences, and set the severity accordingly. It can feel wrong to do this, though, when it seems that such a user is purely imaginary; there is no actual user who is affected in that way or to that extent.
Priority is nearly the same thing as severity, but from the perspective of the user being the project itself. A bug has high priority if the project is severely affected. Priority usually drives what gets fixed now, versus later.
The project cares about the users, so if a large number of users experience a high severity, that bug will likely be treated with priority. The project may also care about the largest users who bring in the most business, and other things like pride in its good reputation. It might give more priority to a bug that is more embarrassing. Or it might give priority to a bug that is more widely publicized, such that there is attention on the project's handling of it. A bug that creates a blocking situation in the project's own development will typically be treated as high priority, even if it hasn't been released to any users.
I've always understood that severity was supposed to be objective and defined based on the needs of the organization/project.
For example:
Crashing bugs and data loss are highest severity.
UI bugs that prevent the user from doing something are medium severity.
UI bugs that don't match the design are the lowest severity.
This should obviously change for Boeing where software bugs have deadly consequences.
When the bugs are filed, they're assigned a priority based on the set criteria. When the bugs are triaged, they're prioritized based on the needs of the organization/project. A typo doesn't actually make the software work less, but it's not something most organizations are okay to ship with.
I don't agree that severity is different if you aren't affected by it. A design flaw in a bridge that could lead to collapse isn't low severity just because I may never cross it.
> UI bugs that prevent the user from doing something are medium severity.
But who is "the user"?
Bugs are high severity based on the principle that if we put ourselves into the shoes of a user who is affected, we understand that the user is affected badly.
However, a crashing bug can be such that only a very specific, rare way of using the program makes it happen. Only a tiny minority of the users who are doing something very specific might be affected.
Quite possibly, "the user" might even be actually "the", as in the one and only one who is running into it.
Furthermore, the severity of a crash bug can be mitigated by whether the affected users have a workaround: can they alter their use of the program in some reasonably small way so that their workflow can continue.
> A design flaw in a bridge that could lead to collapse isn't low severity just because I may never cross it.
That's only so because you've made it a requirement that there is a single parameter called "severity" that may be associated with the bridge, which is a reasonable thing to do. But then we have to imagine it from the point of view of the worst case user: the user of the bridge who is on top of it when it collapses. The severity estimate comes from "how bad could it possibly be".
A bridge I've never heard of in some country where I will never set foot that is on the brink of collapse is certainly not of any severity to me.
Furthermore, a bridge that is in such a state can be cordoned off so that nobody may go over or under it. If nobody may use it (other than trespassers who go past the danger signs which inform them that they do so at their own risk) then there is no "worst-case user" (that we rationally care about). In that situation, the bridge's stability issue is no longer of the same severity, even though it is in the same state as before.
I suppose we're disagreeing on the term severity. I don't see any reason it has to to with the likelihood of encountering the bug. In my opinion severity is based on the result of something happening. Not the likelihood of it happening.
> A bridge I've never heard of in some country where I will never set foot that is on the brink of collapse is certainly not of any severity to me.
That's why you would set the priority low. You don't care if it's fixed, but that doesn't change the fact that it could kill someone and is quite a severe bug.
I've always used severity as a way of objectively classifying bugs in order to help speed up a bug triage. It works well for that.
We use three fields to triage a bug. When the bug is create the PO/support team/Dev tries to the best of his knowledge to assess:
1. Severity - roughly defined as in the article
2. Occurrence - How probable will this bug happen in the field? How many users would be impacted? How much support will be needed for this issue?
Low - Only some very special workflows or mainly dev work impacted.
Medium - Some regular users.
High - Many regular users.
3. Reproducibility - How often will the bug occur when we follow the steps.
Low - Single occurrence or difficult to reproduce
Medium - Erratic behavior, thread problems
High - It will almost always happen.
Then a PO can set a prio and give the ticket the status "will not fix" or "to be analyzed". The dev team then takes the bug and does a time boxed (1 day max) first analysis and comes back with an effort estimate of how big the issue it's too fix. Then a PO can reprioritize and set the status to "to be fixed" or not.
The post doesn't answer the biggest question, which is "What would I do with this additional information"? In concrete terms, what actions would your team take for a P2S0 that would be different from what they would do for a P2S3?
If anyone has a workflow where "severity" is important for your decision-making, I'm curious to learn about it.
Low severity bugs that are hard to fix sometimes get closed as "won't fix". It is unlikely any customer will hit the area where we discovered to breaks and after several weeks we still don't have a clue - why spend more money fixing this issue?
I wish I could get my organization to explore this a little more. We tend to use a single priority value. But it doesn't feel like prioritization to me. If you have 100 issues that all have the same priority level, that means you don't care what order they're dealt with in - which is never really the case. There's invariably another level of "prioritization" that has to happen within each level. But instead of being codified, it gets left to casual discussion, or worse, it never gets discussed.
This has led me to think that a hierarchical priority structure would be useful. So we could say that these 100 Level 3 issues are generally of the same priority, but there's a ranking within that. But even the highest among them is still less than the lowest Level 2 issue.
That's not to say that it shouldn't also be 2-dimensional though.
In my area of software, we provide customised solutions to large enterprises. We have an app that takes in a lot of dimensions about each issue report and spits out an "objective" severity. This app has been honed over 20 years of reviewing issues. That's the only severity that's allowed into JIRA. By default, the priority is set the same as the severity. When we review the issue list with the customers, however, they are free to adjust the priority according to their unique business needs, and developers work on issues in priority order only, not severity. It's a good system, and it ensures that we're working on what the customer needs and not our isolated view of severity. Granted, we will consult with the customer during triage and give them a steer based upon our experience, but they have the final say.
From a developer perspective, I've had a hard time whenever I see the two labels Priority and Severity used together. I find I need to apply some sort of mental kung-fu every time to remind myself what this article is stating. (e.g. Low Priority, High Severity) The reality is, it IS confusing. Kudos to Blue Label Labs for effectively managing it well.
I have worked in a development team that had both priority and severity. The main difficulty I encountered was that people would silently factor one into the other.
Some would factor severity into priority (This is a crashing-type bug, so we should prioritize it), or priority into severity (This bug only causes minor harm, so it's not a high priority for us). But this was done silently and unconsciously, and different people had different directions for how they would do it. Due to this experience, I think one metric is best.
Given a list of items one can purchase (here the currency is engineering resources), given each one's cost and benefit (assuming each item is mostly independent), the optimal strategy is approximately to sort them in decreasing order of benefit/cost and work from the top of the list.
In the terminology I use:
Severity = total impact of the bug = (impact to each customer that experiences the bug) * (estimated rate at which customers experience the bug)
Priority level = severity / cost, rounded to fit in one of n buckets
Sometimes I would almost advocate a little confusion because who is to say you've correctly identified the priority. Who's to say the bug is correctly triaged?
They provide some distinction between the impact and how soon something is to be dealt with. But in many cases, they tend to go together in a proportional manner, at least for the top values of severity and priority. And in all the cases I've seen, people understand them as directly linked together and never try to assign different values (that seem to crisscross if you put them in a two column table).
I disagree. If a lot of people know about the bug, and actually take time to reach out and tell you, it must be more severe than you thought.
People these days are used to bugs. And bugs that arent as severe often go completely unreported; Its so common we shrug it be off. People tend to report bugs when they see no other option.
On some level it’s sort of weird that priority is a field at all, when you also presumably have an overlapping concept of “order we will do these things in”. Priority literally means the degree to which something needs to happen prior to another thing.
We switched to P1, P2, P3. Why? A company that bought us used P1, P2, and P3, and they just couldn't comprehend that most of our "P1" bugs were really P2 bugs under their system.
Misunderstanding of priority and severity fields has been going on for decades and suggests that they're not really useful terms.
Priority in particular is quite subjective and also changes depending on what else is going on (other tickets perhaps?)
Having said that, the bug tracker we use provides the fields by default and we try to use them intuitively. We also try to discourage the "everything is P0" behaviour that happens when people use an arbitrary numerical scale. (What even is P0? I see it pop up when people discover that all their current tickets are P1.
Anyway, here's ours. They're kind of ordered but mostly try to be descriptive and sympathetic to the idea that the judgement is subjective:
Priorities
* "If time permits"
* "Next"
* "Do not defer"
* "Blocks dependants"
* "Release blocker"
* "Don't go home"
The default is "If time permits".
Note how all of them are pretty "important" sounding except the "lowest" which instead admits that sometimes things just can't be done because of other factors rather than because they're not important.
"Next" means "do this next" or "in the next phase of work / sprint / release" and things can climb up to this priority as we take decisions.
"Do not defer" means that it must be done in the milestone it's allocated to. It can't be deferred to another milestone. i.e. we've made decisions that mean it can't be deferred to the next phase. This is for things that we'll need for the next phase or will become blockers but aren't yet. It's not for things where we have generally decided that Now Is The Time. There must be a specific reason it can't be deferred. We use the milestone features to actually decide what's in or out of a scope of work.
"Blocks dependants" is pretty high up and as such needs to be explicit about what it's used for. If your bug blocks something else or has something that depends on a fix then the schedule is going to get messy if it's not done. It's hard for someone with authority to wedge something in here because they feel like it.
"Release blocker" is also pretty high up. This is where people get to jump up and down and explain why they think their bug is so important (and why it's doesn't fit into "do not defer"). This is for stuff that means we can't actually ship the release rather than things we'd just like to be in it. For example "we can't build the software until this is fixed". Hopefully reasonably rare stuff. You need to explain why this blocks a specific release rather than the release otherwise being enough better than the previous one to deliver value to users.
"Don't go home". Sometimes things really are that bad and need to be fixed. At least the person assigning this knows what they're asking, what the implications are and how bad it looks if everything always ends up in this bucket.
Severity works a bit differently. These things are hints from the reporter about the impact of this bug. They really are very subjective.
* Don't care
* Embarrassing
* Minor peril
* Major shitstorm
* QA blockers
* Showstopper
"Don't care" is the default. We want to capture everything we know about our software and encourage people to not feel too "whiny" about filing the notes. This is often for things that end up as "enhancements" or "tasks" rather than "defects".
"Embarrassing". This is something that a user would notice and be left with an impression that we have less attention to detail than the best in the industry.
"Minor peril". This is a bug that's going to cause friction or trouble and give the user an uneasy sense or stop them from building confidence in the software.
"Major shitstorm". I'm never profane in my software but someone suggested this and lots of people liked it and had a good idea of how it should be used. This is for stuff that causes problems when the feature is used. Stuff that leaves a persistent mess that needs to be cleaned up by hand or corrupts some state or something.
"QA blockers". Without fixing these bugs then we're unable to test our software properly. This means we might attract regressions and we probably shouldn't be releasing stuff to users. For example, if you can't save a record that contains the new features then you can't test the corresponding load functionality.
"Showstoppers". These are bugs that cause the whole software to break. Crash bugs, things that destroy user data, etc.
Most of our open bugs mostly sit at "don't care", "embarrassing", "if time permits", "next" and occasionally "do not defer". If stuff starts creeping up the ladder then we know we have a problem coming. If stuff starts to appear straight at the top of the ladder then we get those things fixed ASAPs: they don't hang around as "open" for very long!
A lot of the "higher" values are things that help us deal with interactions. Problems on their own are often tractable. When things start affecting other things and causing knock-on effects then trouble occurs. We try to head that off by knowing about it when it comes in and fixing those things quickly.
> It’s important to note that a bug’s priority shouldn’t be confused with its severity, which exists as an entirely different dimension... ‘Severity’ to represent a (somewhat) objective assessment of a bug’s impact to the functionality of the system. From Critical representing major crashes or hangs, to Minor functional issues or purely Cosmetic blemishes. While a QA engineer will classify a bug’s Severity at creation time, it’s the PM who assigns the Priority at the time of triage based on their knowledge of the client’s business and product requirements
I don't see the benefit in having severity and priority as 2 distinct dimensions. Given a specific priority, what additional benefit do you get from having a severity label as well? Based on the author's description, I can't think of any.
If the goal is to better communicate what "type" of bug it is, using an enum classification (eg, cosmetic vs crash vs data-corruption) would be more appropriate than a numeric scale.