Introducing “Functional Performance Testing” Part 3

This is Part 3/3 of “Introducing “Functional Performance Testing”, a series of articles considering how to test automatically across multi-tier architecture, and across the testing pyramid. 

INTRODUCING “FUNCTIONAL PERFORMANCE TESTING” PART 3: COMPLETE TEST AUTOMATION

Complete test automation: Generating performance and functional tests from the same models

This is Part 3/3 of “Introducing “Functional Performance Testing”, a series of articles considering how to test automatically across multi-tier architecture, and across the testing pyramid. Read Parts One and Two, or download the whole series as a paper.

Part One of this Series set out the myriad of factors that effective functional testing must account for when testing multi-tier architecture, and the additional complicating factors involved in Load testing. Part Two then set out an approach to test case design and data creation that is capable of generating tests that “cover” all these factors. This instalment of the series now sets out how these test cases and data can be automated, using the same central models to generate either functional or Load tests.

The VIP Test Modeller enables “single pane of glass” automation, generating automated tests that can be executed across a range of frameworks from a single tool. This means that both functional API tests can be created for tools like REST Assured, alongside performance tests for tools like Taurus and JMeter. Automated UI tests can likewise be generated, for web testing in Selenium, mobile testing in Appium, and more.

These tests can be generated from the same models, significantly reducing test creation and maintenance time. This “single pane of glass” approach is set out below, focusing for brevity on generating performance tests for Taurus.

Automating the Performance Tests to Simulate Real-World Conditions

Building the message data: a simple, pick-and-choose approach

Executing the test data generated from the models described in Part One as Load tests first requires messages with which to inject the data. The VIP Test Modeller makes the creation of complex message data simple. It enables test data values defined at the model level to be quickly rolled up into JSON messages, for instance, using a visual message builder.

This uses a pick-and-choose approach to build JSON messages, selecting from data variables defined in the flowchart models. This specifies which dynamic data values will be included in the JSON message:

A simple, pick-and-choose approach: Building a JSON message to test a log-in screen.

The simplicity of this approach does not undermine its flexibility. It can be used to create more complex messages, for instance selecting from variables defined at the model level to create a JSON Payload with embedded structures and arrays.

The messages are furthermore defined at the node level, using the same method shown above for defining test data values and variables. The JSON messages are in turn compiled and created during test case generation, creating the message data needed for rigorous Load testing across UIs and APIs.

Defining automated tests to exercise the JSON Messages

Using this approach, the JSON messages that will be produced during automated test generation will contain the full range of data defined at the model level. They will therefore test every logically distinct route through the system’s API and UI. The messages are now therefore ready to be executed, creating automated tests execute them.

The process for generating automated tests in The VIP Test Modeller is again a visual, fill-in-the-blanks approach. Automation modules are built in the start and end-points of the flowchart models, picking from drop-down lists of actions and object.

These include a standardized set of Selenium actions, in addition to custom code that can be imported from existing automation frameworks. These frameworks include a range of performance testing frameworks, such as JMeter. Similarly, it is just as easy to define automated tests for Taurus, building an action to Create a Header:

“Low Code” Test Automation is as simple as picking actions and objects from
drop-down lists.

The same “low code” approach is used to specify the parameters needed to simulate real-world performance conditions. Parameters can be entered for concurrency, hold time, and ramp up time, rapidly building sophisticated performance tests that cover the full range of functionality involved in multi-tier architecture:

Rapidly parameterising tests using a “low code” approach.

The tests generated in this approach accordingly satisfy the four requirements listed in Part One for covering the functionality involved in multi-tier architecture. They further satisfy the additional requirement for performance testing: the tests are parameterized to simulate the range of Load and stress that a system might be subjected to in production.

Automatically generating and executing the test code

Generating automation test code using The VIP Test Modeller is a one-click, pre-defined process. This produces compiled scripts, that are ready to execute using existing frameworks, and which contain all the data needed for testing rigour.

This execution can be performed as part of the same automated process, or separately. Both test generation and execution processes are performed by VIP RPA, Curiosity’s fully connected Robotic Process Automation framework.

Using VIP RPA not only means that tests can be executed across a range of frameworks: standardized workflows from VIP RPA will additionally update test cases and run results within the technologies used across a DevOps pipeline. The automated processes will simultaneously synchronise the same run results in The VIP Test Modeller.

The time-savings in this approach to automated test design and execution are again significant. Testers no longer need to manually write repetitious and brittle test scripts for vastly complex systems, but simply update central models. They further do not need to update fields in Continuous Delivery or ALM tooling, with high-speed bots performing these “TestOps” tasks for them.

Single Pane of Glass Automation: combine the simple test design with full test flexibility

The automation actions and objects available in this “low code”, fill-in-the-blank approach are imported from existing test automation frameworks. VIP RPA again provides an automated process for this, parsing existing automation code. This makes objects and actions available in The VIP Test Modeller, regardless of whether they come from open source, commercial, or home-grown frameworks.

This enables “Single Pane of Glass” automation, where tests are generated from central models for a range of frameworks. Prior to defining JSON Messages and Taurus performance tests, for instance, the above functional models could have been used to generate functional API and UI tests. These might be executed in an existing REST Assured framework, for instance, quickly building automated tests from actions and objects imported from the framework into Test Modeller.

Single Pane of Glass Automation generates tests for a range of automation frameworks,
and to cover the whole testing pyramid. The rigorous tests are executed across multi-tier
architecture.

At a time when skilled automation engineers are in short supply, anyone can use Test Modeller to build tests using drop-down menus, or by chaining together subflows containing pre-defined automation logic.

Single Pane of Glass Automation generates tests for a range of automation frameworks,
and to cover the whole testing pyramid. The rigorous tests are executed across multi-tier
architecture.

Two-pronged test automation, with business analysts and testers able to request clearly
defined automation logic from engineers. The re-usable logic then feeds a central
repository, enabling flexible but simple-to-build test automation.

The approach set out in this series is capable of generating rigorous tests that cover multi-tier architecture and the whole testing pyramid. The tests are maintained automatically as the central flowchart models update, enabling the rigorous testing of complex systems to occur within an iteration. Download the whole series here, and please get in touch with any feedback or questions.

Or does this all sound too good to be true? Watch the demo or Speak with a Curiosity expert to get started!

[Image: Pixabay]