CODEWITHSUNDEEP
metrics
Hugo
Hugo

Reputation: 6454

Bug Metrics -- What can I get from my Bug Database?

As part of a internal research project, we are trying to collect some metrics from a Bugzilla database; we already have found a tool to help us collect some metrics from it (BugzillaMetrics) but we are now asking ourself what metrics should we collect?

Now, that is why I would like to ask you:

**

What Kind of Metrics about Bugs do you Collect?

**

In our office, the teams are small (2 to 5 developers), we have thought in metrics like bugs per developer, per development sprint, bug per category (GUI, business logic, database) but we would like to hear some other ideas.

Thanks in advance =)

Upvotes: 4

Views: 2934

Answers (4)

Susheel
Susheel

Reputation: 11

Following are the more useful ones:

  1. Ratio of CRITICAL and MAJOR bugs found per iteration, and avg time taken to resolve them. e.g. could be targeted in hours for CRITICAL and few days for MAJOR, targets could be revised based on historical numbers to be realisitcally challenging.

  2. No. of MAJOR Bugs remaining in product at the time of Release. It may be acceptable to release with 3 or 5 or 7 MAJOR, depending on product/ industry/ customer. {{ Assuming CRITICAL Bugs = 0 i.e. not acceptable. }}.

  3. High priority lifetime ratio : Ratio of P1 resolution time compared to AVERAGE resolution time of all Priorities.

  4. Reopened Rate : CRs Percentage of reopened cases out of those Fixed in an iteration.

  5. CRs with no Comments/ answers within 2 days : Ratio of cases having no response from R&D in 2 days after creation.

  6. Priority of severe bugs Average Priority of blocker and critical CRs

  7. Ratio of resolved cases with resolution INVALID | or DUPLICATE.

Susheel Jalali

Upvotes: 1

Mark Kofman
Mark Kofman

Reputation: 563

I suggest following list of metrics:

  • Number of currently open defects in whole product.
  • Metrics for iteration burn-down chart: Number of open bugs/tasks, number of resolved bugs/tasks planned for given iteration
  • Defect detection percentage for each product version. This metric shows the ratio between defects detected during development and QA compared to defects found after QA when version was already release

Upvotes: 1

Cătălin Pitiș
Cătălin Pitiș

Reputation: 14341

One of the relevant metric is the number of defects discovered on a time unit (e.g. week, testing iteration, etc.). This might be a good indicator for when it is acceptable to stop testing and fixing. Of course, this metric can consider also the priority of the bugs as well (you might be less interested if there are 10 trivial bugs reported per week than if there are 1-2 major defects per week).

Another metric you might find useful is the mean-time to fix a defect (the time between reporting and fixing/closing the bug).

Upvotes: 4

RS Conley
RS Conley

Reputation: 7196

Bugs per category definitely. I would also do time estimates versus actual time spent. The point of which to give the developer a tool to learn how to make accurate estimates. Estimating time is a notoriously fuzzy process and your best source is experience. With this metric you can gain confidence in the estimates given by everybody.

However mind you still won't be able to just say that Bug X should take Y time because it is similar to Z Bug. But you will able to let Developer Baker look at it and when "It will take 2 days to fix" you have something to judge how accurate he is.

Upvotes: 2

Related Questions