Test Impact Analysis.

Are you struggling with the number of WebDriver tests you have? Do you run them continuously on every change? Have you tried all the things already been suggested? You run tests in batches, on multiple nodes, against light weight web application stripped of any animations? Well this is a presentation for you, as I was facing similar problem a while ago. Let me start with a bit strange question, have you ever considered generating test coverage report for your code, and either got discouraged by colleagues, or found out yourself, how uneasy to read and understand such report can be? What if I told you there is a hidden value in such report, and that this value is Test Impact Analysis. While test coverage can’t really tell you whether your tests are sufficient or not, it can tell you what tests are relevant for particular code, particular change. Automated tests are vital in guarding us from introducing regressions into our products. Tests aren’t just sugar and cream. They come with a cost. Execution time. Tests Impact Analysis determines how relevant tests are. Help you reduce execution time to minimum focusing only on the tests that matter. I will talk about our journey of adopting the technique in a product safety guarded by over 50k automated tests. I will share with you the challenges we faced and how we solved them. What kind of improvements we were hoping for and how we reduced execution time of all tests, including WebDriver by 30%. Also what we learnt from the experiment and what are our plans for the future. To summarise, I will show you how you can have Test Impact Analysis in your project using open source tools and what kind of strategy you’d follow to have it safely running in your pipeline. You will learn how we built our operational muscles as we were introducing Test Impact Analysis to pipeline, and how we treat all internal services affecting pipeline as production services, so that we can assure smooth delivery of critical fixes to our users.