Mobile test automation - 5 mistakes to avoid

Welcome to Part 3/5 in our “Scalable Mobile Test Automation” series.

So far, we’ve seen how testing strategies must adapt to meet the rapid rise of mobile adoption, and why the mixed success of automated testing so far calls for a new approach to test automation creation. Otherwise, test automation history shows us that automation rates will struggle to move beyond the 15-20% achieved so far [1].

Part 2 analysed this growing demand for automation, quantifying the vast “combinatorial explosion” created by mobile use. It explored how combining devices, operating systems, browsers and more can quickly create trillions of new permutations that might require testing.

To meet this demand, mobile test automation must learn from the decades-long history of test automation. It must avoid the mistakes of “manual” test automation, which has succeeded in automating just a fraction of tests to date.

Want to read all five parts of the series now? Download Curiosity’s latest eBook, How to Scale Mobile Test Automation.

Mobile test automatioN: 5 test creation mistakes to avoid

Manually creating repetitive test scripts has proven too slow and brittle to match the historical demand for test automation, while tending to create low-coverage test suites that leave systems exposed to bugs. Key lessons learned from this history include:

1. Manual scripting is too slow and repetitive. Copying and pasting boilerplate code will not match the “combinatorial explosion” created by mobile testing.
2. Unsystematic test creation will leave complex systems exposed to bugs. Faced with trillions of potential combinations, manually prioritising test cases will focus on a narrow range of low-coverage scenarios.
3. Manually scripted tests are too brittle to change. This creates impossible maintenance requirements, piling up testing bottlenecks and technical debt sprint-over-sprint.
4. Unavailable, inconsistent and erroneously matched data destabilises automation. Inaccurate or constrained test data creates frustrating test failures and delays, undermining the scalability of automated testing.
5. Repetitive scripts are dense and do not explain clearly what is being automated. This hinders collaboration with other stakeholders involved in development, including non-coders who should provide critical feedback for testing early and avoiding rework.

If manually creating automation has not worked for web and desktop testing, adding the additional strain of testing on mobile will not fix the approach. A new approach is needed, and Part 5 in this series will propose an approach to generating and scaling mobile tests. But next, Part 4 will identify four principles to follow when implementing your mobile test automation strategy.

Want all five parts of the series in one place?  Download How to Scale Mobile Test Automation.

References:

[1] Capgemini, Sogeti (2022), The World Quality Report 2021-22, P. 23. Retrieved from https://www.capgemini.com/insights/research-library/world-quality-report-wqr-2021-22/ on December 12^th 2022.