Skip to the main content.

Curiosity Modeller

Design Complex Systems, Create Visual Models, Collaborate on Requirements, Eradicate Bugs and Deliver Quality! 

Product Overview Solutions
Success Stories Integrations
Book a Demo Release Notes
Free Trial Brochure

Enterprise Test Data

Stream Complete and Compliant Test Data On-Demand, Removing Bottlenecks and Boosting Coverage!

Explore Curiosity's Solutions

Our innovative solutions help you deliver quality software earlier, and at less cost!

robot-excited copy-1              AI Accelerated Quality              Scalable AI accelerated test creation for improved quality and faster software delivery.

palette copy-1                      Test Case Design                Generate the smallest set of test cases needed to test complex systems.

database-arrow-right copy-3          Data Subsetting & Cloning      Extract the smallest data sets needed for referential integrity and coverage.

cloud-cog copy                  API Test Automation              Make complex API testing simple, using a visual approach to generate rigorous API tests.

plus-box-multiple copy-1         Synthetic Data Generation             Generate complete and compliant synthetic data on-demand for every scenario.

file-find copy-1                                     Data Allocation                  Automatically find and make data for every possible test, testing continuously and in parallel.

sitemap copy-1                Requirements Modelling          Model complex systems and requirements as complete flowcharts in-sprint.

lock copy-1                                 Data Masking                            Identify and mask sensitive information across databases and files.

database-sync copy-2                   Legacy TDM Replacement        Move to a modern test data solution with cutting-edge capabilities.

Explore Curiosity's Resources

See how we empower customer success, watch our latest webinars, read our newest eBooks and more.

video-vintage copy                                      Webinars                                Register for upcoming events, and watch our latest on-demand webinars.

radio copy                                   Podcasts                                  Listen to the latest episode of the Why Didn't You Test That? Podcast and more.

notebook copy                                           eBooks                                Download our latest research papers and solutions briefs.

calendar copy                                       Events                                          Join the Curiosity team in person or virtually at our upcoming events and conferences.

book-open-page-variant copy                                          Blog                                        Discover software quality trends and thought leadership brought to you by the Curiosity team.

face-agent copy                               Help & Support                            Find a solution, request expert support and contact Curiosity. 

bookmark-check copy                            Success Stories                            Learn how our customers found success with Curiosity's Modeller and Enterprise Test Data.

file-document-multiple (1) copy                                 Documentation                            Get started with the Curiosity Platform, discover our learning portal and find solutions. 

connection copy                                  Integrations                              Explore Modeller's wide range of connections and integrations.

Better Software, Faster Delivery!

Curiosity are your partners for designing and building complex systems in short sprints!

account-supervisor copy                            Meet Our Team                          Meet our team of world leading experts in software quality and test data.

calendar-month copy                                         Our History                                Explore Curiosity's long history of creating market-defining solutions and success.

check-decagram copy                                       Our Mission                                Discover how we aim to revolutionize the quality and speed of software delivery.

handshake copy                            Our Partners                            Learn about our partners and how we can help you solve your software delivery challenges.

account-tie-woman copy                                        Careers                                    Join our growing team of industry veterans, experts, innovators and specialists. 

typewriter copy                             Press Releases                          Read the latest Curiosity news and company updates.

bookmark-check copy                            Success Stories                          Learn how our customers found success with Curiosity's Modeller and Enterprise Test Data.

book-open-page-variant copy                                                  Blog                                                Discover software quality trends and thought leadership brought to you by the Curiosity team.

phone-classic copy                                      Contact Us                                           Get in touch with a Curiosity expert or leave us a message.

6 min read

Test Data is make or break for parallel testing and development

Test Data is make or break for parallel testing and development

Today, there is a greater-than-ever need for parallelisation in testing and development. “Agile” and iterative delivery practices hinge on teams working in parallel, as otherwise iterations become mini-waterfalls as one team waits for another to finish.

DevOps and CI/CD likewise rely on parallel processes, while part of the speed gains offered by test automation lie in its ability to run parallelised tests. Certain types of testing, like load testing, are furthermore impossible if high volumes of tests cannot be run in parallel.

Too often, test data undermines parallelisation

In spite of its importance, the parallelisation of tests and teams is frequently undermined by a lack of autonomy (or independence) in non-production environments and data. This is reflected in perennial frustrations that testers and developers experience, like:

  1. “My automated test failed because another test in the suite used up or edited the data it needs”.
  2. “I need a rare customer type with this history to run this high-priority test. However, another tester has deleted the customer from the database and a refresh is not due for weeks.”
  3. “We cannot run our tests as we are waiting for another team to finish using a shared environment”.
  4. “We cannot run our tests as an integrated component has not yet been developed fully, and we do not have access to it in our environment.”

Given that a tester can spend as much as 44% of their time on test data-related activity [1], the above frustrations are clearly a problem worth solving. Overall, providing data and environments to work in parallel is imperative for achieving the pace and quality aimed for in “Agile” ways of working and CI/CD.

Achieving Test Autonomy

Test autonomy is key to parallelisation. It can be split into two categories: The independence tests need to run in parallel, and the independence testers need to work side-by-side. Let’s call the latter “tester autonomy”.

Considering first test autonomy, automated tests each require unique data combinations to run in parallel. Otherwise, they might fail when one test uses up another’s data. This challenge is discussed in an excellent video by Sauce Labs’ Titus Fortner, which inspired the language of “test autonomy” used in this article.

The challenge of achieving test autonomy today is that an increasing number of parallelised tests require increasingly-complex data, and these tests will change constantly to support rapid software development. Manually matching unique data combinations to tests is simply too slow and brittle, risking bottlenecks and errors. Instead, an on-the-fly and “just in time” approach is needed.

Automated test data “Find and Makes”

Curiosity’s Test Data Automation offers test data “find and makes” that hunt for data from across interrelated data sources. The data “finds” are based on exact test criteria, which are passed automatically as parameters by tests. This can be achieved, for example, by firing an API call to a list of reusable data lookups, with a simple function embedded in automated test scripts.

But what if data cannot be found in available data sources, or if multiple tests require the same combinations? To ensure testing agility, the test data “finds” are integrated with automated data “makes”. These “makes” produce new data when needed, ensuring that every test has the data combination it needs on-the-fly.

A range of techniques can be used to make new data as needed, as shown in the below video. For example, Test Data Automation might look for similar data and modify it, automatically parsing a data query to understand the type of data needed. Alternatively, existing data combinations might be cloned and assigned to different tests, with each data clone assigned unique identifiers. Synthetic test data generation might additionally create new data from scratch.

Automated test data “find and makes” ensure that parallelised tests runs on-the-fly, with on demand test data. Resolving data “just in time” ensures that each test has accurately matched data, even as tests change. This in turn resolves the time lost finding, making, or waiting for test data to become available, helping to solve one of the greatest bottlenecks in testing and development.

Test Data Allocation

Test Data Allocation further supports test autonomy, and integrates seamlessly into Test Data Automation’s “find and makes”.

Allocation matches data combinations to tests within a shared environment. Data within a database, file, API or message can then be locked for use by an individual test, or can be set to have "read" privileges if the test does not need to edit it. Read privileges allow other tests to access the data without editing it.

Allocating data to tests ensures that every test is equipped with the data it needs, without other tests editing or using that data up. At the same time, parallel tests still enjoy all the data they need, by virtue of integrated test data “find and makes”.

Achieving Tester Autonomy

We’ve now looked at techniques for ensuring that every parallel test in a suite comes equipped with the data it needs on-the-fly. Let’s now look at techniques for ensuring that every tester and developer can access the data they need in parallel.

Database Virtualisation and Orchestration

One of the greatest barriers to providing parallelised test data lies in the time and expense associated with copying data.

Organisations today still frequently make physical copies of non-production data. Ops teams run through a complex set of manual or semi-automated processes, striving to create copies of data that resemble complex production relationships. This risks massive bottlenecks and runaway costs, as ever-growing data copies increasingly consume physical infrastructure.

Today, data refreshes can take longer than an iteration, meaning that data is always out-of-date by the time it reaches testers and developers. Teams are furthermore often forced to share environments, as physically copying data cannot scale to meet the demands of parallelisation. They then face the frustration of useful data being edited or deleted by another team.

Database virtualisation instead allows organisations to make lightweight, virtual copies of data at a fraction of the time and cost of making physical copies. It provides a cost effective, scalable solution to parallel data access needs, where as many as 30 databases can run on a single 8 core server [2].

Database Virtualistion and Orchestration for Test ParallelisationDatabase virtualisation allows parallel teams and frameworks to access affordable, lightweight data copies in parallel and on demand.

Spinning up virtual databases can also take less than a minute, with full version control over the data. With integrated database orchestration and a self-service test data portal, virtualisation allows parallel teams and frameworks to spin up the exact databases they need, exactly when they need them. 

The following video shows virtualisation incorporated into a broader Test Data Automation solution, providing complete and compliant virtual test data on demand:

Data Cloning

If teams are sharing a database, it’s imperative that they can access data combinations in parallel. Automated data cloning supports tester parallelisation, just as it ensures that automated tests can access equivalent data combinations in parallel.

If two testers require the same data combination, data can be cloned and each clone assigned unique identifiers. Both testers can then access and update data that is functionally equivalent, without impacting one another. This avoids the frustration caused by data cannibalisation, when one tester or team uses up, edits, or deletes otherwise useful data.

Self-serve data and reusability

To support true parallelisation, test data processes like cloning and virtualisation should be reusable on demand. Otherwise, testers and developers will remain dependent on an Ops team to create the data they need, creating a dependency that undermines parallel ways of working. Ideally, every requisite test data process should be reusable on-the-fly, including masking, generation, subsetting, and beyond.

Automating test data utilities and making them parameterisable on demand alleviates test data bottlenecks, as testers and automated tests can trigger the test data jobs they need, exactly when they need them.

With Test Data Automation, each configured test data job becomes reusable from a central catalogue. The jobs can then be triggered on-the-fly, passing in parameters “just in time” to create and provision the right data at the right time.

Automated and Reusable Test Data Generation and Masking Jobs Support Test ParallelisationReusable data masks and generation jobs can be triggered on-the-fly, providing the right data to tests and teams in parallel.

Automated tests can pass parameters into reusable jobs via API calls, while orchestration engines, schedulers and batch processing can likewise trigger jobs. Parallel testers and developers can similarly trigger test data processes on-the-fly. For example, they can fill out configurable forms to self-provision data, selecting from drop-downs and inputs to pass parameters into the reusable test data jobs:

Self-Service Test Data Forms

Self-service forms configured in intuitive business language allow testers and developers to request the exact data they need on demand.

If no data can be found, automated data “find and makes” plug the gaps, ensuring that every tester receives the parallel data they need. Each test data request made by testers can additionally feed a central list of reusable test data queries. The lookups can then be triggered by test automation or CI/CD frameworks, or re-used to refresh virtualised environments with a comprehensive test data:

Automated Test Data Find and Makes Cropped

Lists of reusable data requests refresh environments with comprehensive test data on demand, while allowing automation frameworks and CI/CD pipelines to self-provision data on-the-fly.

Exposing parameterizable and automated jobs to testers and tests ensures that both receive the data they need on-the-fly. Resolving a comprehensive set of test data utilities “just in time” provides complete and compliant data-as-a-service, even as the exact data needed in testing and development changes:Test Data as a ServiceTriggering a set of automated and parameterisable utilities “just in time” allows parallel testers and tests to access the exact data they need, when and where they need it.

Test Data for Test autonomy

This article has now discussed a range of techniques for providing data in parallel to both tests and teams. Providing this autonomy to tests and testers is imperative to removing test data blockers in automated testing, while providing the parallelisation needed for iterative delivery and CI/CD. To learn more about how Test Data Automation provides data “just in time” to parallel teams and frameworks, check out Curiosity’s Test Data-as-a-Service solution brief.

Download the Solution Brief


[1] Capgemini, Sogeti (2020), The Continuous Testing Report 2020. Retrieved from on 08/07/2022.

[2] Windocks, "The Business Case for Containers and Clones". Retrieved from on 08/07/2022.

We Need to Talk About Test Data “Strategy”

We Need to Talk About Test Data “Strategy”

For many organisations, test data “best practices” start and end with compliance. This reflects a tendency to focus on the problem immediately in...

Read More
8 Criteria for a Modern Test Data Solution

8 Criteria for a Modern Test Data Solution

In 2023, (test) data availability, quality, and compliance risks remain a major headache for software development.

Read More
5 Test Data Challenges That Every CTO Should Know About

5 Test Data Challenges That Every CTO Should Know About

At Curiosity, we talk about test data extensively, because we believe test data is repeatedly neglected in testing and development discussions....

Read More
The Democratisation of (Test) Data

The Democratisation of (Test) Data

A glance at industry research from recent years shows that test data remains one of the major bottlenecks to fix in DevOps and CI/CD:

Read More
If testing was a race, data would win every time

If testing was a race, data would win every time

Okay, so that title doesn’t make complete sense. However, if you read to the end of this article, all will become clear. I’m first going to discuss...

Read More
Quality Testing Requires Quality Data

Quality Testing Requires Quality Data

My two most recent blogs have made the case for a new TDM paradigm called “Test Data Automation”. The first article considered how a logistical...

Read More
Is test data the engineering problem to solve in 2024?

Is test data the engineering problem to solve in 2024?

It’s 2024 and the risks associated with poor test data practices show no signs of abating.

Read More
Curiosity and Windocks announce Containerised Test Data Automation

Curiosity and Windocks announce Containerised Test Data Automation

Curiosity Software Ireland, creators of Test Data Automation, and Windocks, on demand database specialists, today announced a joint solution for...

Read More
GDPR and testing: Are you a sceptic or a gambler?

GDPR and testing: Are you a sceptic or a gambler?

Last week, we published a blog making the case for the next generation in TDM “best practice”. We considered why the logistical approach of “mask,...

Read More