Model-Based Testing (MBT) itself is not new, but Model-Based Test Automation is experiencing a resurgence in adoption. Model-Based Testing is the automation technique with the greatest current business interest according to the 2018 World Quality Report, with 61% of respondents stating that they can foresee their organisation adopting it in the coming year.[1]
Technologies like the Test Modeller have significantly reduced the time and technical knowledge needed to model complex systems. Organisations can now enjoy all the benefits of Model-Based techniques within short iterations, whereas previously modelling had been reserved for only the most high-stake projects, such as lengthy Waterfall projects in aerospace and defence.
This five-part blog series considers the benefits of introducing Model-Based Techniques to QA, discussing five reasons for adopting modelling. Today’s article begins with the very beginning of the delivery lifecycle, setting out how modelling can uproot defects earlier, where they require far less time and resources to fix.
Historic data from both industry and academic research has consistently shown that the errors introduced earliest in the software delivery lifecycle account for the greatest number of defects:
In 2001, research from The IT University, Denmark, found that 59% of defects are traceable to the design phase;[2]
In 2009, Bender RBT suggest that 59% stem from requirements;[3]
By 2012, the Hyderabad Business School found that 64% of bugs emerge in the requirements analysis and design phase.[4]
The same research has established that it is far more costly and time-consuming to remedy these defects after the design phase. In fact, requirements defects can account for as much as 82% of total defect remediation efforts,[5] while a bug identified during testing costs on average 15 times more to fix than one discovered during the design phase.[6]
This is the textbook argument for beginning QA earlier: it allows you to detect the majority of defects as they emerge, rather than after they have been perpetuated throughout the code. Detecting defects during the requirements analysis and design phase therefore gives you a greater chance of delivering quality software on time and within budget.
This raises an important question: what does it mean to perform QA during the design phase? To understand this, we must first consider why so many defects are introduced during requirements analysis and system design.
The quality of the requirements themselves remains the major cause of defects. The formats and methods used to capture software designs are ill-matched to reflect complex system logic. They tend to be ambiguous and incomplete, leading to miscommunications. These misunderstandings then require rectification through costly and time-consuming rework.
Organizations still frequently capture user and business needs in a range of disparate formats. These include written user stories, written documents and behaviour-driven specifications, as well as email or chat requests and business process diagrams. Such requirements are rarely connected formally and rely heavily on natural language.
Natural language is naturally ambiguous and is far removed from the precise system logic that needs to be converted into code. Written requirements therefore frequently allow for multiple interpretations of how the system should function, risking a drift in the vision held by BAs, developers, and testers.
A Gherkin Feature File: the natural language is ambiguous and the feature file focuses only on a handful of possible logical scenarios. A multitude of these feature files are needed to document a system. However, the files are rarely formally connected to one another.
The same requirements also tend to be incomplete. Unsystematically piling up unconnected requirements inevitably leave holes in the design of vastly complex systems, and requirements often focus near-exclusively on a limited number of expected scenarios. Developers must fill the remaining holes with their imagination, and defects arise when their vision deviates from the desired system.
Finally, the unconnected and informal nature of requirements formats prevents formal dependency mapping. Developers must then imagine how the mass of components in a complex system should integrate, again increasing the likelihood of bugs. Numerous high-profile outages that have followed supposedly routine updates point to the real danger this presents.
Requirements are a prime place to improve the speed and reliability of software delivery, given the prevalence of design defects. Formally modelling requirements offers one way to achieve this “shift left” QA. It is now furthermore now well-suited to short delivery lifecycles.
Flowchart modelling offers one such way to formally model a system’s logic. The blocks of a flowchart break a system down into its core cause-and-effect, with the process and decision blocks forming a directed series of “if this, then this” statements. These are consolidated clearly and graphically, mapping the logical journeys through a system that could be taken, and the data required to traverse that path:
A visual flowchart clearly depicts the validation rules of a new user registration page.
The act of modelling new or existing requirements works to eradicate ambiguity. If there are multiple ways of parsing a requirement, the singular and correct understanding must be established to build the model. Testers can thereby act as critical modellers, working with BAs from day one to crystallise the desired system logic. This “shift left” QA roots out ambiguity in the design phase.
Modelling further works to eradicate incompleteness. Any missing logic is far easier to spot in consolidated visual models when compared to unconnected disparate written requirements and diagrams.
Flowchart modelling offers the significant advantage that it is already frequently used by the BAs who gather requirements, many of whom already create business process. Flowcharts therefore enable testers to work collaboratively with BAs, to eradicate ambiguity and incompleteness from day one.
Test Modeller further provides a range of connectors for building flowcharts rapidly. This includes importers for existing requirements and test cases, as well as for recordings and scans of already built systems. This accelerates the modelling process, introducing all the advantages of modelling to short iterations. The choice between clear and complete specifications and rapid delivery is no necessary.
The Gherkin importer converts specification files automatically to unambiguous flowchart
models in Test Modeller.
Not only does explicitly modelling requirements to eradicate defects before they enter code, but the same requirements models can be used to generate the functional and performance tests needed to validate the code once it has been developed. This enables greater testing agility, and is the subject of the next article in this series.
[1] Capgemini, Micro Focus, Sogeti (2019), World Quality Report, 27-8. Retrieved from https://www.sogeti.com/globalassets/global/wqr-201819/wqr-2018-19_secured.pdf on 30-May-2019.
[2] Soren Lausen and Otto Vinter (2001), “Preventing Requirement Defects: An Experiment in ProcessImprovement”, in Requirements Engineering (2001, 6:37-50), 38. Retrieved from http://www.itu.dk/people/slauesen/Papers/PrevDefectsREJ.pdf on 30-May-2019.
[3] Bender RBT (2009), Requirements Based Testing Process Overview, 2, 16. Retrieved from http://benderrbt.com/Bender-Requirements%20Based%20Testing%20Process%20Overview.pdf on 30-May-2019.
[4] P Mohan, A Udaya Shankar, K JayaSriDevi (2012), “Quality Flaws: Issues and Challenges in Software Development”, in Computer Engineering and Intelligent Systems (2012, 3:12:40-48), 44. Retrieved from www.iiste.org/Journals/index.php/CEIS/article/viewFile/3533/3581 on 30-May-2019.
[5] James Martin, cited from Bender RBT, Requirements Based Testing Process Overview 2.
[6] P Mohan, A Udaya Shankar, K JayaSriDevi (2012), “Quality Flaws: Issues and Challenges in Software Development”, 44.
[Image: Pixabay]