How to Scale Mobile Test Generation

Welcome to Part 5/5 in our “Scalable Mobile Test Automation" series!

Part 4 identified four principles for scalable mobile test automation, concluding that rigorous mobile test automation cannot be exhaustive or manual, but must automatically and collaboratively generate targeted tests from among the vast “combinatorial explosion” created by mobile tests and devices.

Let’s now consider techniques for putting these principles into practice, considering how model-based test generation creates rigorous tests at scale, creating and targeting automated tests continuously for mobile.

Want to read all five parts of the series? Download our eBook, How to Scale Mobile Test Automation.

Generate and scale mobile tests using visual models

Model-based test generation from Test Modeller offers a proven technique for implementing the principles of scalable mobile testing identified in Part 4 of this series.

In this approach, collaborative flowcharts map the logic of each mobile component, auto-generating targeted tests and data for mobile.

The visual flows specify both the test logic and reusable automation needed to test each step, as well as the devices, platforms and browsers to test against. Combining the flows then authors end-to-end scenarios at speed, applying automated coverage algorithms to generate targeted tests.

As mobile systems change, you then only need to make one-off changes to the reusable automation actions and flowcharts, regenerating rigorous and targeted mobile tests.

Combining reusable flowcharts and automation libraries generates and maintains optimised tests and data for integrated mobile components.

Let’s now look in more detail at this approach, considering how it sidesteps repetitive scripting and optimises in-sprint test coverage, enabling rapid and rigorous mobile testing at scale.

Rapid and simplified Mobile test design

The scalable test generation starts with the creation of visual flowcharts that chart your mobile application logic. The models map a series of “if this/then this” statements, forming journeys through the mobile website or app. To maximise reusability, each step only needs defining once to feature in multiple user journeys:

An easy-to-use flowchart models defines user journeys through a mobile web store.

Each model is likewise reusable, enabling lightning-fast test authorship for end-to-end scenarios. Combining vast mobile test logic becomes as quick-and simple as dragging-and-dropping blocks to master flows, connecting the dots between the subprocesses:

Reusable flowcharts are quick-to-combine, rapidly modelling end-to-end journeys through mobile systems.

Reusable mobile test automation

The reusability does not stop with the models. In Test Modeller, automation actions are likewise centralised and reusable from shared libraries.

Automating test steps is in turn as quick and easy as dragging-and-dropping reusable actions from central repositories, providing a locator from your choice of mobile app inspector:

Automating the modelled logic is as quick and simple as dragging and dropping reusable automation from a central library.

Test Modeller comes equipped with a library of reusable mobile automation actions, while additional code can be synchronised from scripted frameworks. The actions are all customisable using code templates, ensuring the extensibility and flexibility of mobile test automation:

Reusable automation actions are all configurable at the code level, ensuring that mobile test generation can “cover” even the most complex logic.

Testing across devices and platforms

The reusable automation library further includes actions for spinning up different devices, operating systems, browsers and platforms. This is performed in your choice of device lab, emulator or simulator, as well as visual testing grids.

You can accordingly reuse test logic and scenarios defined once in reusable flowcharts to test at scale across platforms and devices. This simply uses a decision block to pick between devices and platforms, using optimisation algorithms to force test generation down particular paths in the end-to-end flowchart:

A path has been auto-generated to spin up an Android device and app in Sauce Labs’ US Server.

Targeted test script generation:

The logical precision of Test Modeller’s flowchart models enables this automated test generation.

The flowchart model provides a “directed graph”, to which automated “graph analysis” algorithms can be applied. These algorithms act like a GPS finding routes through a city map, and are capable of identifying every possible path through end-to-end models:

Automated test generation identifies an optimal set of paths through a modelled mobile application.

With the combinatorial explosion created by mobile testing, you will rarely want to test exhaustively. Test Modeller’s coverage algorithms therefore allow you to reduce the total test volume, without compromising test coverage.

The test generation systematically generates the smallest set of test cases needed to “cover” every logically distinct path through the flowchart, while coverage profiles can focus on at-risk logic. This doesn’t just include targeting test logic; coverage profiles can additionally focus on particular devices, browsers, and platforms, achieving the optimal mix of devices and test logic in sprint.

Automated maintenance and continuous testing

This approach creates a 1:1 mapping between mobile system component and flowchart model, as well as a 1:1 mapping between the automation code and each reusable automation action. This makes test maintenance as quick as making one-off changes in centralised automation libraries and flowcharts, recombining the componentised logic and automation to generate up-to-date tests.

The automated test generation refactors previously generated test cases and scripts, which are all traceable back to the flowcharts. This avoids manual test maintenance and the risk of growing technical debt, regenerating targeted tests “just in time” during CI/CD.

Deliver quality experiences to every user and device

As app and website traffic fast moves to mobile, testing strategies must likewise shift to ensure that every user, on every device, enjoys a seamless experience. This calls for new approaches in test automation, avoiding the “manual” test automation that has historically only managed to automate 15-20% of tests [1].

Mobile test automation cannot furthermore focus on creating more tests, faster. It must also target and optimise testing, prioritising from the trillions of permutations created by the combination of different device types, browsers, operating systems, mobile logic and beyond.

Model-based test generation offers a proven technique for generating rigorous tests at scale, automating the prioritisation and creation of automated tests.

Generating targeted mobile tests from visual flowcharts accordingly fulfils the four principles of scalable mobile test automation identified in this series, rigorously testing fast-changing apps for use across an optimal mix of devices and platforms:

Reusability

The model-based test generation creates automated tests from central models and reusable automation libraries. This enables rapid test creation and maintenance, as you only need to define logic and automation code once. Making one-off changes to models and automation libraries then reflects changes in mobile code and logic, rapidly refactoring tests.

Automation

The flowchart models automatically generate mobile test cases and scripts, avoiding the bottlenecks and technical debt associated with repetitive test scripting and brittle test maintenance. In its fullest implementation, this test generation becomes continuous, importing artifacts to central flowcharts and leveraging traceability analysis to regenerate tests and models based on changes from across the SDLC.

Optimisation

The automated test generation further creates the smallest set of tests needed to “cover” the whole model, while the optimisation algorithms can also be used to target particular logic and combinations based on time and risk. This overcomes mobile’s “combinatorial explosion”, testing the right blend of devices and logic in-sprint.

Communication

The intuitive, visual flowcharts additionally provide a central point of collaboration and communication. They are readily accessible to system designers, developers and testers, who can all provide feedback to avoid rework and leverage the easy-to-use models to generate “shift left” tests.

Read all five parts of this series in Curiosity’s latest eBook, How to Scale Mobile Test Automation.

References:

[1] Capgemini, Sogeti (2022), The World Quality Report 2021-22, P. 23. Retrieved from https://www.capgemini.com/insights/research-library/world-quality-report-wqr-2021-22/ on December 12^th 2022.