The politics of bug fixing

PM Clinic: Week 29 discussion summary

Compiled: 5/9/2005

The Situation:

My team has finished major development work and is now turning the corner into bug fix mode. I'm the group manager, and I sat down with my dev managers and dev leads to discuss how we wanted the team to prioritize work. We quickly fell into quicksand. I felt we should be prioritizing around importance - the kinds of bugs that we needed to make sure we fixed.

Most of the development managers/leads made arguments in favor of complexity - they wanted us to focus on how difficult bugs would be to fix. The case in point were two examples:

1. Trivial to fix bug that made a pri 1 feature look *awful*
2. Difficult to fix bug that prevented a pri 2 feature from working properly all the time

My argument was that A should come before B, since Pri 1 features come before Pri 2. But they're argument was that B should come before A. Since they bet we'd have to do it anyway (Executives wouldn't let it ship), and since it was a big work item we should take it on first.

I quickly realized that my team wasn't prioritizing for itself - but was prioritizing bugs around management and other perceptions of how other people in the organization would *force* us to prioritize bugs. So in essence my development team wanted to make pre-emptive political choices when it came to bug fixing. Any advice for making this process less frustrating?

- Lost in defect politics

Common bug fixing criteria

Just for reference, here's some common criteria for evaluating bugs. Priority and Severity are two related but different criteria that all bugs should have.

Severity of this item.

1. Bug causes system crash or data loss.
2. Bug causes major functionality or other severe problems; product crashes in obscure cases.
3. Bug causes minor functionality problems, may affect 'fit and finish'.
4. Bug contains typos, unclear wording or error messages in low visibility

Priority of this item.

1. Must Fix within 24-48 hours. Blocking build testing.
2. Must fix as soon as possible. Bug is blocking further progress in this area.
3. Should fix soon, before product release.
4. Fix if time; somewhat trivial. May be postponed.
   

Common project timelines that impact bug/defect decisions

Many projects use milestone to indicate overall progress. This basic timeline can be mapped to most methodologies. If defined right these help frame how bugs should be managed at any given time.

• Spec complete: The designs of features are written and reviewed by the team. Work estimates for all work items are in, and leaders add/cut features by this date. The clock of scheduled work begins.
• Code Complete: This is the date when all work items have been finished. Incomplete features should be cut or negotiated with other work. The team switches from creating features to refinement and quality work. All remaining work of any kind should be tracked in the bug database.
• Triage Starts: All bugs must be triaged by a bug committee at this point. Start punting lower severity bugs and ensuring that high-severity bugs get fixed. Battles about priority and severity, such as this week's situation, should be fought here, if not earlier. There should be daily or weekly goals (and reports) for how quickly bugs should be triaged, how many should be fixed, etc.
• Visual/UI Freeze: All bugs that impact the UI (even if they’re trivial) must be fixed or punted by this date. Make this date early enough so that technical writers and localization folks have time to get their work done before you ship.
• ZBB (Zero bug bounce): This means you hit zero bugs at least once. This gives you the opportunity to settle into a final period where you will fix only the most severe bugs that will seriously impact customer quality. The only bugs considered during ZBB should be new bugs found that were not part of the earlier triage(s).
• RCs (Release Candidate): This should be the first build that seriously a candidate for release. Criteria should be set for what kind of newly discovered defect will cause a second (third, or Nth) candidate.

Managing general bug politics

Early on in the project someone has to define the criteria for evaluating bugs. Even if it's just the list above (Priority and Severity) it will reduce the number of arguments people have, and elevate the quality of those remaining arguments.

The project goals should serve as the source for decision making. The goals should be interpreted against the current situation (bug count, remaining schedule, etc.) by team leaders and specific instructions should be presented to the eam. If the project goals need to be changed to reflect the current situation, then team leaders need to be very public about this.

So if you find yourself constantly fighting the same battles over bugs, talk to your manager or other team leads. Ask them for advice on how to avoid repeating the same arguments over and over again. if you are a team leader, it's your job to step up and provide the team with some guidance.

Common ways bugs are manipulated

Here's a partial list of things to watch out for:

• Inflated priority or severity.
• Feature masquerading as a bug.
• Hot potato bugs. (Bugs assigned away because no one wants to be responsible for them).
• Bugs described in misrepresentative ways to either help get them killed or help get them fixed.
• Bugs that are fixed outside of the team triage, but on the company's time.
• Raw bug counts used to track competitive progress (10 trivial fixes is not as much work as 1 complex fix).

Managing bug manipulators

If you're dealing with people that are playing games with bugs like in this week's situation, here's some approaches:

• Persuasion: get the one main bug manipulator to have coffee with you, and explain your case. Ask him to try working with bugs differently, even if just for a few days, and give you a chance to demonstrate the better way.
• Force the issue: get key players in a room and discuss this exact difference of philosophy. Outline the 3 or 4 common tactics you see being used and suggest alternative ways to deal with those situations. See if anyone else even cares - if they do, see if there are other bug manipulation issues that need to be discussed.
• Work around the bug manipulators. Stay ahead of the curve and triage bugs early, avoiding the involvement of other folks.
• Get someone with more power/influence than you to use persuasion or force the issue.
• Go to the dark side. Become a better bug manipulator than they are.

The real costs of bug fixing

Several folks commented that bug fixes are often evaluated heavily on their programmers work estimates. This is shotgun thinking: typically the testing time is the bigger hit, especially late in a project. Another important criteria is the probability of fix related faults (bugs caused by the fix to the previous bug).

It's important for team leaders to hold fast to their goals: if the priorities are really the priorities, then the amount of time it takes to fix a bug isn't the most important criteria (unless two bugs are of otherwise equal value to the project).

Contributors

Mark Colburn, Shep McKee, Karen Barrett, Steven Levy, Faisal Jawdat, David Gorbet, Andrew Stellman, Scott Berkun (editor)

About PM-Clinic

The pm-clinic is a friendly, wise, open forum for discussing how to lead and manage teams of people. Each week a new situation is sent to the list and we share advice, ideas and stories. Anyone can join as long as they follow the simple rules.