IT change remains a persistent struggle for most organisations today. Software teams are aware of the need to move faster and be more agile; yet, they are dealing with growing complexity and the weight of unknowns within the ecosystem of their current IT architecture estate. The misinterpretation of Agile principles has seen a culture where documentation (of which test design is a part) has fallen by the wayside. Fortunately, for teams who appreciate that software engineering is a complex, emergent discipline, there are techniques for turning this situation around.
Testing is a key part of this solution. Testers can help uncover and formally document knowledge needed to:
Model-Based Testing (MBT) is becoming more popular with high-performing teams as an approach to deliver the outcomes that the business demands from Testing teams.
This article considers two different approaches to Model-Based Testing, arguing that the right approach can achieve the quality at speed originally sought by “Agile” methodologies.
Like many popular capabilities within Testing, the definitions of Model Based Testing (MBT) has been blurred by the different approaches given the same name. This is similar to the many deviations of TDD and BDD.
This article aims to explain the difference between the two interpretations of Model Based Testing (MBT) that I am aware of, though I’m sure there are or will be deviations in future as MBT becomes more broadly used.
The two broad interpretations of MBT are:
Let’s now consider the scope and value of both approaches to Model-Based Testing.
The first understanding of model-based testing aims to design/visualize the system being developed. The models are used subsequently to derive test cases and automation. These tests are based upon system rules that are embedded in a living specification:
A model visualizes and identifies routes by which data and users can flow throw a system. These “paths” are equivalent to auto-generated test cases.
This approach offers a range of benefits for software delivery:
In this way, it helps resolve some of the core barriers to delivering increasingly complex systems at speed.
The second approach to model-based testing starts later in the development lifecycle and has a narrower focus.
It aims primarily to create test automation modules, which can then be pieced together and executed against a system under test. In the narrowest applications of this approach, system logic and requirements are not modelled, and nor are equivalence partitions. Instead, models effectively do the work of copy/pasting code into scripts. They chain together reusable automation libraries, with a model representing one or more automated test case.
Overall, this approach offers several benefits for scaling test automation:
However, this second, narrower approach does not offer the full benefits of model-based testing.
It is not rooted directly in the system logic and so does not guarantee test optimization or coverage. It further starts too late to improve software requirements, nor does it help retain knowledge of complex systems, improve collaboration, and remove silos in software delivery.
In fact, creating “low” or “no” code style models solely to generate automation code can inherit many of the issues of traditional test automation approaches:
Can Artificial Intelligence help with these challenges? AI can broaden the amount of testing done. However, it isn’t a short cut to resolve these problems. On its own, the increased test volume can create an analysis overload, while many of the findings turn out to be superficial observations.
To unlock the full benefits of model-based, sufficient thought must instead be put into test design and test approach.
Modelling system requirements and logic helps remove challenges in test automation, while offering benefits across software design and development.
The system models collaboratively refine requirements, while linking test design to the requirements and code. The generated tests can therefore be optimized for test coverage, while test data and environments can additionally be spun up from the models:
This collaborative, “shift left” approach to modelling starts far earlier in the development cycle. It captures data and knowledge from across tools and teams, exposing it in a way that avoids technical debt, builds accurate requirements, and efficiently manages growing complexity.
At the same time, the act of documentation drives accurate development and continuous test generation. The documentation therefore unlocks the very “Agile” methods that have historical lead organisations to ditch documentation in the first place.
Modelling provides a “one input, many output” approach, in which the act of modelling generates, maintains and links the different artifacts needed for rapid development and testing.
Many organizations will seek guidance when it comes to test tools. This might be out of choice, or due to organizational purchasing polices. You are likely familiar with different types of vendor comparisons.
However, test design is rarely a category by which tools are compared and recommended, with the exception of some commendable research. Reviews, analysis and advise instead tends to focus on automation execution tools, test management tools and service providers.
This is a problem, as test Design is a cornerstone of the success of any organization’s success in software testing, and for software delivery overall. It is therefore surprising that it lacks the guidance and support that you might expect.
When you are reviewing your test approach, ensure that test design is front and center. Hopefully the points in this article can go some way to help you navigate what kind of problems you are trying to solve, and will motivate you to consider certain model-based testing tools that can solve them.
Want to explore how model-based testing can unblock your software testing a delivery? Book a meeting to speak with a member of the Curiosity team.