automating mobile payment tests using the aid of a robot
In the past few years the mobile market experienced a large inflation. This trend subsequently lead to more powerful devices with a wide range of functionalities. Ensuring the quality of mobile applications is vital and many frameworks provide excellent support in quality assurance. However, the fast growing market, and emerging fields of use present testers with new challenges.
Apps do no longer just interact with the mobile phones' hardware itself, but also use wireless technologies, e.g. NFC, to exchange data. The additional hardware makes testing a tedious job. This presentation addresses an instance of this problem, in particular testing an NFC-payment system. Merchants use a payment terminals or a mobile devices, running Android, to process contactless payments. The payment system distinguishes between a few states of NFC-smartcards e.g. default, locked, expired and blacklisted.
Although the test cases for the software itself can be automated, the entire process of testing the system against all NFC-smartcards requires continuous human interaction. Thus testing gets tiresome and facilitates failure due to human error. Test automation is a substantial issue and therefore a robot, which takes care of managing the set of NFC-smartcards, is designed and integrated into the testing process. The robot's motions are highly precise and can be customized to fit different NFC-reading devices. The main benefit of using a robot is its lack of exhaustion and more reliable reproducible test results. Furthermore, the automated tests are less error-prone due to the robot's accuracy and error handling features. Without the robot, payment tests are restricted to trained personnel, who would execute all test cases by hand. The automated robot-assisted tests can be triggered by anyone and enable a publishing of test results to a testmanagement system with some additional configuration. The presented robot exposes a REST-API for external access. Currently, all payment tests are integrated into GitLab CI, and are executed immediately after a successfully built new software. The execution can also be triggered manually.
In my presentation I will share some of the experiences I have made during developing the robot-assisted test setup, present obstacles, challenges, and solutions to them. The focus will not just be on the robot itself, but also on how its application improved the whole testing environment, from test automation to test reporting. The presentation concludes with a live demonstration or a video.