A journey through an adapted zero-bug policy

For a long time, we deferred the non-priority bugs in our backlog, to fix them later. However, we rarely went back to actually do it. Our backlog increased over the years, making its cleaning more laborious, painful, and with subsequent tech debt (thus with a higher cost), as old bugs take more time to review, investigate and reproduce. Moreover, these bugs were kept low in the list as new features and bugs would make their way in. We decided to implement a new Bug Treatment Strategy, with the goal of putting Quality and the Customer at the center of our product. It's a change of paradigm around product quality and accountability, that we already see having a massive impact on our mindset and quality culture. This Strategy is inspired by the Zero-Bug Policy concept (https://bit.ly/2OZLID8), applied to our context.

Whenever a new bug is open, the Product Owner of the Team is responsible to decide whether the bug needs to be fixed, or if it must be closed. New bugs that are kept must be fixed within 30 days after the bug creation. The motto is ‘be honest, be accountable’. Be honest. Never defer the bug. If you don't fix it right now, chances are that you never will. If you have decided not to fix it now, it means that you estimate that there are more important things to do. Be accountable.

Making a decision right away increases accountability, because the reporter will receive either an explanation why the issue won't be fixed or that the issue will be actually fixed in a reasonable time. Also, the decision-maker will have to justify their decisions if too many bugs are closed and if the quality of the product decreases. We expected the following benefits: easier prioritization, team morale improving, increase reporter satisfaction, and reduce tech debts The risks to mitigate are velocity slow-down, no explanation closing bugs, people not raising bugs. Before the actual implementation, we held necessary though fruitful conversations at management/executive levels and involved key members across the organization, to assess the impact and the challenges of such change. We set up an initial rollout of about one month, to let the teams the time to fully understand what was needed of them, and coach them on how they could implement the strategy on a daily basis.

Also, we promoted the strategy by training teams outside product and engineering around bugs and quality, as everyone in the company has a role to play and can contribute to quality improvements. To measure the results of the new strategy, we defined 5 metrics. These metrics are being used to follow teams' progress, and adjust the framework if needed. As a QA, we started communicating the metrics' progress during reviews, contributing to place bugs and quality at the center of teams' daily work, with the precious help of Project Managers and Product Owners.

Within a quarter, we observed positive trends in all the metrics, and in the capacity of delivery net new work. By the time of this conference, we will have data for three quarters. I will update the trends and the problems we encountered, and how we adjusted the framework to tackle them.