Testing on the beat

Last year I decided to take on a job as a Quality Engineer for music production software. For me, music was a complex new domain, in which I didn’t have prior expertise.

I was excited to learn more about making music, but at the same time I was not an aspiring musician. I didn’t mind my music to sound off the beat, but I definitely did not want my testing to be the equivalent of that. So it was important to not miss important workflows while focusing on irrelevant flows.

The big questions were how much depth of domain knowledge would I need to test effectively, and how could I transfer to this new context what I already know about gathering information and organizing my exploration?

This talk will cover what I learned from my experience after more than a year, as well as how ideas from Domain-Driven Design and other resources have inspired me to do new experiments. I will also describe how I re-evaluated my understanding about the importance of local and situated knowledge, and what that kind of knowledge encompasses.

I will tackle questions like:

- How desirable is it to become a domain expert for the project you are testing on?

- How much effort should you invest into that kind of knowledge, given that as testers we constantly need to make trade-offs in terms of the range of specialization areas in such a complex, fast-growing domain as software?

- How do we, as tester experts, effectively gain enough domain knowledge to be able to provide valuable information in a team?

We will also look into some of the strategies that were useful in my case:

- Use analogies to domains where I had more knowledge(like video editing software)

- Get close to the implementation level – it is software in the end, so the code and the product’s architecture reflects the functionalities it has.

- Discuss with UX designers with curiosity, and prepared to change my mind

- Include every team member’s ideas in the test planning process by providing a structure to come up with useful question. For example, I used Elisabeth Hendrickson’s testing heuristics cheatsheet, as well as an Outside-In risk analysis structure from James Bach

- Model a feature in multiple ways to understand it better

Join me in making some beats and learning about how to test that process!