Measure it or it didn't happen

In order to improve quality, we have to find a way to measure progress. Also managers love KPIs. My experience started counting the tests executed vs the test failed. Since these numbers are far away both from the user value and from the health state of the product itself, I deemed them as irrelevant.

Next I started counting the bugs opened and closed per sprint. It had couple of downsides: its too simplistic and its’ easy to manipulate. 

I wanted less subjectivity and a more granular approach. From the tech side, I tried the mean between bug creation on release on live, downtimes, count of post mortems. From the user side, the number of ’non-issues’ was quite interesting. 

Let me take you on a KPI journey throught the metrics I tried, that worked and didn’t work for us and how I assessed them. I hope  some of them will help you as well.