5 Test Data Challenges That Every CTO Should Know About
At Curiosity, we talk about test data extensively, because we believe test data is repeatedly neglected in testing and development discussions....
Design Complex Systems, Create Visual Models, Collaborate on Requirements, Eradicate Bugs and Deliver Quality!
Product Overview | Solutions |
Success Stories | Integrations |
Book a Demo | Release Notes |
Free Trial | Brochure |
Pricing |
Our innovative solutions help you deliver quality software earlier, and at less cost!
AI Accelerated Quality Scalable AI accelerated test creation for improved quality and faster software delivery.
Test Case Design Generate the smallest set of test cases needed to test complex systems.
Data Subsetting & Cloning Extract the smallest data sets needed for referential integrity and coverage.
API Test Automation Make complex API testing simple, using a visual approach to generate rigorous API tests.
Synthetic Data Generation Generate complete and compliant synthetic data on-demand for every scenario.
Data Allocation Automatically find and make data for every possible test, testing continuously and in parallel.
Requirements Modelling Model complex systems and requirements as complete flowcharts in-sprint.
Data Masking Identify and mask sensitive information across databases and files.
Legacy TDM Replacement Move to a modern test data solution with cutting-edge capabilities.
See how we empower customer success, watch our latest webinars, read our newest eBooks and more.
Events Join the Curiosity team in person or virtually at our upcoming events and conferences.
Blog Discover software quality trends and thought leadership brought to you by the Curiosity team.
Help & Support Find a solution, request expert support and contact Curiosity.
Success Stories Learn how our customers found success with Curiosity's Modeller and Enterprise Test Data.
Documentation Get started with the Curiosity Platform, discover our learning portal and find solutions.
Integrations Explore Modeller's wide range of connections and integrations.
Curiosity are your partners for designing and building complex systems in short sprints!
Meet Our Team Meet our team of world leading experts in software quality and test data.
Our History Explore Curiosity's long history of creating market-defining solutions and success.
Our Mission Discover how we aim to revolutionize the quality and speed of software delivery.
Our Partners Learn about our partners and how we can help you solve your software delivery challenges.
Careers Join our growing team of industry veterans, experts, innovators and specialists.
Press Releases Read the latest Curiosity news and company updates.
Success Stories Learn how our customers found success with Curiosity's Modeller and Enterprise Test Data.
Blog Discover software quality trends and thought leadership brought to you by the Curiosity team.
Contact Us Get in touch with a Curiosity expert or leave us a message.
4 min read
Thomas Pryce 29 March 2022 15:07:36 BST
It’s 2024 and the risks associated with poor test data practices show no signs of abating.
According to the latest World Quality Report, the use of potentially sensitive production data rose in 2021, despite the compliance risks associated with using raw data in less-secure test environments.
This risk of costly legislative non-compliance is compounded by the fact that testers at 45% of organisations do not always comply with security and privacy regulations for test data, making the distribution of sensitive information to test environments an even more startling concern.
In addition to compliance risks, test data continues to undermine both testing speed and quality, with around half of organisations reporting that they do not have sufficient data for all of their testing. Half again report that they do not have timely access to the right test environments, undermining testing agility as well as coverage [i].
These challenges are not new, and their persistence highlights how test data “best” practices have fallen behind evolutions in “agile” delivery methods, DevOps, automation, and CI/CD:
Perennial test data challenges reflect how far test data "best" practices have fallen behind "agile", DevOps, automation and CI/CD. Original Image: Khanargy, Wikimedia Commons, published under CC BY-SA 4.0 license.
This blog will explore some of the risks to software delivery associated with outdated test data practices. It then identifies current trends that threaten to make test data problems worse if not addressed proactively. Fortunately, these same trends often hold the key to solving test data challenges, offering techniques for automated, on-the-fly and “just in time” Test Data Automation.
To see how organisations today can transition from existing test data management practices to Test Data Automation, watch Curiosity's and Sogeti's webinar, The state of test data in 2022: New challenges, opportunities, and the role of “AI”.
The risks associated with outdated test data practices touch upon delivery speed, costs, and legislative compliance.
These wide-reaching challenges undermine goals that commonly motivate an organisation’s drive to become more “agile”. In other words, test data frequently conflicts with a drive to deliver increasingly complex software, faster, and without incurring prohibitive delivery costs.
Some of the risks associated with test data are seen in the following stats drawn from recently available research:
Test data challenges accordingly risk the efficiency and quality of software delivery, while posing a threat to legislative compliance and an organisation’s bottom line. TDM is clearly a challenge worth solving today. Yet, it’s also a problem that is continuously becoming more complex to “fix”.
A range of trends in recent years have added to the complexity of finding, making, and provisioning fit-for-purpose test data. These same trends have contributed to the demand for data across the SDLC, and in turn risk exacerbating bottlenecks associated with test data. These recent trends include:
Test data today must span an intricate combination of new and existing technologies.
Unless test data tools and techniques match this growth in the complexity and demand for test data, test data challenges will persist and grow.
Fortunately, many of the factors that are today adding to the demand for and complexity of test data also offer solutions to perennial test data challenges.
For instance, techniques used today in test automation can be leveraged to make data via UIs, enabling test scripts to self-provision data on-the-fly. Integrating parameterizable test data utilities as part of CI/CD pipelines can likewise reduce the need for manual intervention when allocating data to tests, reducing the risk of bottlenecks during continuous test execution.
And, of course, there’s AI, and its promise to automate many of the complex tasks associated with test data. As with any discussion of AI and testing, caution must be exercised today to identify what truly constitutes “AI”, and whether the value of incorporating AI is greater than using alternative approaches. Nonetheless, techniques today can support the on demand creation of complex data, such as using solving techniques to reverse-engineer data values needed to reach given expected results.
If you’d like to see how test can leverage valuable techniques found across DevOps, automation, and the emerging world of “AI”,watch Curiosity's and Sogeti's webinar, The state of test data in 2022: New challenges, opportunities, and the role of “AI”.
References:
[i] All the stats in the introduction are taken from Capgemini, Sogeti (2021), The World Quality Report 2021-22. Retrieved from https://www.capgemini.com/gb-en/research/world-quality-report-wqr-2021-22/ on 18/02/2022.
[ii] Capgemini, Sogeti (2020), The CONTINUOUS TESTING REPORT 2020, 21. Retrieved from https://www.sogeti.com/explore/reports/continuous-testing-report-2020/ on 22/03/2021.
[iii] ScopeMaster, Shift Left Testing. Retrieved from https://www.scopemaster.com/blog/shift-left-testing/ on 18/
[iv] See, for example, Appnovation (2021), The Digital Consumer: Expectations are higher than ever - Can your brand keep up? Retrieved from https://www.globenewswire.com/news-release/2021/02/17/2176967/0/en/The-Digital-Consumer-Expectations-are-higher-than-ever-Can-your-brand-keep-up.html on 18/02/2022.
[v] Cisco (2019), Consumer Privacy Report. Cited in Thomas C. Redman and Robert M. Waitman (2020), Do You Care About Privacy as Much as Your Customers Do? Retrieved from https://www.globenewswire.com/news-release/2021/02/17/2176967/0/en/The-Digital-Consumer-Expectations-are-higher-than-ever-Can-your-brand-keep-up.html on 18/02/2022.
At Curiosity, we talk about test data extensively, because we believe test data is repeatedly neglected in testing and development discussions....
Today, organisations utilise and adopt a range of technologies, both old and new, in service of enabling their “agile” delivery methodologies. Yet,...
Evolution and innovation in software delivery often focuses on automation, or on changing how teams collaborate and work together across the software...
A glance at industry research from recent years shows that test data remains one of the major bottlenecks to fix in DevOps and CI/CD:
Curiosity often discuss barriers to “in-sprint testing”, focusing on techniques for reliably releasing fast-changing systems. These solutions...
Test automation must be lightweight, re-usable and easy to apply, in order to help organisations, ease its implementation enterprise wide. Curiosity...
For many organisations, test data “best practices” start and end with compliance. This reflects a tendency to focus on the problem immediately in...
Low level, repetitious tasks underpin all business activity. However, these labor-intensive activities are still frequently performed manually, to...
Okay, so that title doesn’t make complete sense. However, if you read to the end of this article, all will become clear. I’m first going to discuss...