Measure it or it didn't happen

30-minute Talk

In order to improve quality, we have to find the right KPIs to measure progress

Timetable

11:55 a.m. – 12:25 p.m. Tuesday 13th

Room

Room F2 - Track 2: Talks

Audience

Testers

Key-Learning

  • Test passed is irrelevant in terms of user value or tech sanity
  • Choosing the right KPIs can help us focus and develop
  • The wrong KPIs can create false goals desire to improve KPIs rather than quality
  • Too many of the wrong KPIs can demotivate the team

What I learned about quality KPIs over the last 5 years in testing

In order to improve quality, we have to find a way to measure progress. Also managers love KPIs. My experience started counting the tests executed vs the test failed. Since these numbers are far away both from the user value and from the health state of the product itself, I deemed them as irrelevant.

Next I started counting the bugs opened and closed per sprint. It had couple of downsides: its too simplistic and its’ easy to manipulate. 

I wanted less subjectivity and a more granular approach. From the tech side, I tried the mean between bug creation on release on live, downtimes, count of post mortems. From the user side, the number of ’non-issues’ was quite interesting. 

Let me take you on a KPI journey throught the metrics I tried, that worked and didn’t work for us and how I assessed them. I hope  some of them will help you as well.

Related Sessions