Test Impact Analysis.

30-minute Talk

Learn how to use code coverage data in your advantage. Forget about setting virtual thresholds. Learn from the data to improve pipelines.

Timetable

11:10 a.m. – 11:40 a.m. Thursday 7th

Room

Room F2 - Track 2: Talks

Audience

Testers, Developers, DevOps Engineers

Key-Learning

  • Learn new types of tests helping you better balance test execution time and accuracy.
  • Learn how to continuously test in feature branching model.
  • Learn how to configure open source coverage analysis tools to generate per test coverage.
  • Learn how to release test impact analysis solution safely. What it means to TestOps really. Treating test infrastructure as production.
  • Learn what kind of benefits you can expect from introducing Test Impact Analysis.

Smarten up your pipeline and reduce test execution time.

Are you struggling with the number of WebDriver tests you have? Do you run them continuously on every change? Have you tried all the things already been suggested? You run tests in batches, on multiple nodes, against light weight web application stripped of any animations? Well this is a presentation for you, as I was facing similar problem a while ago. Let me start with a bit strange question, have you ever considered generating test coverage report for your code, and either got discouraged by colleagues, or found out yourself, how uneasy to read and understand such report can be? What if I told you there is a hidden value in such report, and that this value is Test Impact Analysis. While test coverage can’t really tell you whether your tests are sufficient or not, it can tell you what tests are relevant for particular code, particular change. Automated tests are vital in guarding us from introducing regressions into our products. Tests aren’t just sugar and cream. They come with a cost. Execution time. Tests Impact Analysis determines how relevant tests are. Help you reduce execution time to minimum focusing only on the tests that matter. I will talk about our journey of adopting the technique in a product safety guarded by over 50k automated tests. I will share with you the challenges we faced and how we solved them. What kind of improvements we were hoping for and how we reduced execution time of all tests, including WebDriver by 30%. Also what we learnt from the experiment and what are our plans for the future. To summarise, I will show you how you can have Test Impact Analysis in your project using open source tools and what kind of strategy you’d follow to have it safely running in your pipeline. You will learn how we built our operational muscles as we were introducing Test Impact Analysis to pipeline, and how we treat all internal services affecting pipeline as production services, so that we can assure smooth delivery of critical fixes to our users.

Related Sessions