Room
Room E2+E3 - Track 5: Test Automation Deep Dive
Audience
testers, test automation engineers, production chain managers, infra-administrators,
Key-Learning
- How to leverage historical data to build best strategies to reduce flakiness
- How to use smart and automatic processes to identify the defects worth fighting for
- How to build consensus and commitment within your organization to address this challenge
- How to use gamification to add a fun factor to fixing flakiness
Sharing my experience and the experience of my organization with treating flakiness at-scale and in an industrialized fashion.
Before attending this version of the ATD conference, you probably have read numerous articles and watched numerous talks discussing test flakiness, the reasons behind it, and the ways to reduce it.
For this talk I would like to share my experience and the experience of my organization with treating flakiness at-scale and in an industrialized fashion. Some backstory: Our software factory has been producing and continuously evolving a single software product for more than 35 years, growing to be one of the biggest market leaders in the fintech industry, trusted by major banks and institutions to keep their business running and their numbers straight. This could not have happened without a strong attention to quality and huge investment in testing.
I and other QA consultants in my organization, we pride ourselves with the testing culture that we have built over the years and with the test repository we have developed, reaching today an arsenal of more than 1200 online tests that fully deploy and test our software, with a frequency of 20,000 instances of tests executed daily amounting to 20,000 hours of test runtime.
So, you can imagine on this scale how flakiness becomes a fact we cannot ignore, the impact of which can disrupt our production chain and frustrate our teams. A solution has been put in place by the QA department and is being continuously enhanced to tackle this problem, and I am eager to share with you the steps we took to reach our current model:
• How we introduced short term patches while investing in long term resolutions
• How we leveraged historical data to build our strategies
• How we used smart and automatic processes to identify the battles worth fighting for
• How we gathered consensus and commitment within the organization to address this challenge
• How we used gamification and mock awards as a team building activities adding a fun factor to our recipe
• How our model can be further enhanced to cover more aspects of test health
• And finally, how it can be tailored to fit your organization