Kiran and Vivek said that they chose this topic for presentation because they wanted to share some pieces of their upcoming book. The book now has a working title ‘Metrics for Agile Tech Teams’. They also mentioned about how they, along with Guru Thimmapuram, are hoping to help organizations in setting meaningful goals.

CONTEXT

The speakers set the context by defining Engineering Excellence as ‘building a product that you can be proud of.’

The speakers laid an outline that they will be discussing three pairs of metrics. 

Each pair would have two metrics.

1. Metric 1: Something that is generally considered useful but gives bad results
2. Metric 2: Something that can be used in the place of Metric 1

PAIR-1

Code Coverage: Code coverage is a popular metric in agile engineering teams.  However, it is not a good measurement for quality.  Code coverage tells us whether there are tests.  It does not tell us if the tests are of good quality.

Mutation Coverage:  Mutation testing is a practice where we intentionally add bugs to the code and check if the test cases catch those bugs.  This tells us whether tests are of good quality or not.

PAIR-2

Build Failure Count: Since build failure is treated as a negative thing, many organizations measure the build failure count and try to reduce it over a period of time.  However, this creates a culture of blame and prevents people from taking risks.  Also, red builds are not a bad thing – they are the Return-on-Investment of Continuous integration system.

Total Build Red Time: Instead of measuring the build failure count, teams can measure the total time for which the build was red in a day, week or a month and try to reduce the time progressively.  Here, we are focusing on fixing rather than blaming.  This makes people motivated to fix the build as soon as possible, thereby reducing the development downtime.

PAIR-3

Bugs per Developer: Organizations commonly measure defects per developer or defects per team with an aim to reduce the number over a period of time.  Again, like the Build Failure Count above, this creates a culture of blame and prevents people from collaborating towards a common goal, as they see one person’s victory as another person’s failure.

Support Case Density:  For each feature that they deliver, the teams can wait for a defined period and measure how many support cases were created related to that feature.  They can then divide this number by the size of the feature in story points to get the support case density. Over a period of time, they aim to reduce this number. This is a better metric for quality because this focuses on usability in addition to defect-prevention.

Questions

Here are some of the questions that were asked in the webinar.

1. What programming languages do mutation testing tools support?
2. Is there a leading indicator to prevent getting a bad Support case density?
3. How is mutation testing different from negative testing?