Skip to main content

How Much Money is Our Test Improvement Initiative Worth?

30-minute Vendor Talk

We know our test processes best. It is thus on us to make demands for more resources in a language our managers understand, which is often a return on investments estimation. Here is how we did this.

Virtual Pass session

Timetable

10:45 a.m. – 11:30 a.m. Wednesday 26th

Room

Room E1 - Track 4: Talks

Other

Audience

Test Managers. Testers. Everybody who needs to ask management for more resources for their tests

Key-Learnings

  • Use domain specific data to link software bugs to monetary costs. In our case: Penalty payments caused by production loss of wind turbines.
  • Test Gap Analysis helps to prevent untested changes that cause many bugs in practice.
  • Keep testing your hypotheses and updating your estimates as you implement your test initiative and get fresh data.

How we estimated the monetary value of our test improvement initiative to make our management see the value, instead of only the cost of testing. And how maybe you can, too.

The challenge of securing resources for testing—often called the quality dilemma—stems from the imbalance between cost and perceived value. While testing costs are easy to quantify, the benefits are harder to express in monetary terms—the metric decision-makers care about. How do we price a bug our tests successfully prevented?

At Vestas, we found a way! Our wind turbines rely on complex software, and bugs can cause malfunctions, impacting energy production. If downtime exceeds limits, Vestas incurs penalties based on lost energy. These penalties are traced back to specific components, including software, driving accountability and improvement.

By attaching a price tag to bugs, we can estimate the benefits of improving test processes if they prevent (some of) these issues.

This talk outlines Vestas’ test process for wind turbine control software, the Test Gap analysis initiative we launched a year ago, and our cost-benefit analysis of it. Finally, we compare our estimates with real-world data after Test Gap analysis went live.

While not every domain has a direct link between bugs and costs, we hope this talk provides a template for quantifying the benefits of testing.

Related Sessions

Mon, Nov 24 • 8:30 a.m. – 4:30 p.m.
F-,E- & D-Rooms

Full-Day Tutorial (6 hours)

Artificial Intelligence (AI) Other

Wed, Nov 26 • 2:45 p.m. – 4:45 p.m.
Room D6 - Track 9: Workshops

120-minute Workshop

Other Testability

Wed, Nov 26 • 10:45 a.m. – 12:30 p.m.
Room D6 - Track 9: Workshops

105-minute Workshop

Agile Methodologies Other