Continuous performance testing through the users’ eyes

One day, part of my team went to visit one of our customers and returned a bit concerned. They went to learn more about how users interact with our system, but also noticed some performance improvements we could work on.

This specific customer used our product intensively, producing not only large amounts of data but also complex relationships between records in the data-model. The team quickly addressed these never-seen-before scenarios and greatly improved performance. But how were we going to make sure that performance would not degrade over time for our customers, as we developed more features?

At first this seemed like a common scenario, but we found out that performance testing tends to focus on simulating lots of users rather than data-intensive scenarios with few users. Also, a lot of the tools that are commonly used focus on requests to the server, but part of the performance issues we had were happening on the browser-side.

In this talk I will present how we built a performance testing solution integrated with our continuous integration system. I’ll be covering the tools we chose for the task, including one we built ourselves, leveraging standard APIs in modern browsers to obtain metrics that are truthful to what the user experiences. You’ll see what we learned along the way, what worked best for us and also what didn’t.