Time to migrate from your legacy test data (TDM) tools? Here’s how.
If you’re reading this, you’re probably already painfully familiar with the complaints that Curiosity hear from organisations seeking alternatives to...
Design Complex Systems, Create Visual Models, Collaborate on Requirements, Eradicate Bugs and Deliver Quality!
Product Overview | Solutions |
Success Stories | Integrations |
Book a Demo | Release Notes |
Free Trial | Brochure |
Pricing |
Our innovative solutions help you deliver quality software earlier, and at less cost!
AI Accelerated Quality Scalable AI accelerated test creation for improved quality and faster software delivery.
Test Case Design Generate the smallest set of test cases needed to test complex systems.
Data Subsetting & Cloning Extract the smallest data sets needed for referential integrity and coverage.
API Test Automation Make complex API testing simple, using a visual approach to generate rigorous API tests.
Synthetic Data Generation Generate complete and compliant synthetic data on-demand for every scenario.
Data Allocation Automatically find and make data for every possible test, testing continuously and in parallel.
Requirements Modelling Model complex systems and requirements as complete flowcharts in-sprint.
Data Masking Identify and mask sensitive information across databases and files.
Legacy TDM Replacement Move to a modern test data solution with cutting-edge capabilities.
See how we empower customer success, watch our latest webinars, read our newest eBooks and more.
Events Join the Curiosity team in person or virtually at our upcoming events and conferences.
Blog Discover software quality trends and thought leadership brought to you by the Curiosity team.
Help & Support Find a solution, request expert support and contact Curiosity.
Success Stories Learn how our customers found success with Curiosity's Modeller and Enterprise Test Data.
Documentation Get started with the Curiosity Platform, discover our learning portal and find solutions.
Integrations Explore Modeller's wide range of connections and integrations.
Curiosity are your partners for designing and building complex systems in short sprints!
Meet Our Team Meet our team of world leading experts in software quality and test data.
Our History Explore Curiosity's long history of creating market-defining solutions and success.
Our Mission Discover how we aim to revolutionize the quality and speed of software delivery.
Our Partners Learn about our partners and how we can help you solve your software delivery challenges.
Careers Join our growing team of industry veterans, experts, innovators and specialists.
Press Releases Read the latest Curiosity news and company updates.
Success Stories Learn how our customers found success with Curiosity's Modeller and Enterprise Test Data.
Blog Discover software quality trends and thought leadership brought to you by the Curiosity team.
Contact Us Get in touch with a Curiosity expert or leave us a message.
5 min read
Thomas Pryce 29 March 2022 14:30:00 BST
Data generation enables organisations to create data of the right variety, density, and volume for different testing and development scenarios, all while avoiding the compliance risks associated with using raw production data.
Creating data from scratch will be familiar to testers and developers, who often produce data when they find that the varied data they need is missing or unavailable. Today, further applications of data generation are emerging, such as the generation of training data sets for AI/ML.
Several factors might lead you to consider adopting a new data generation solution. These include evolving compliance requirements, growing data complexity, and an increasing demand for data across your organisation. In these instances, an automated, proven solution might promise to be more robust and scalable when compared to the ad hoc data creation that occurs manually during testing and development.
This article sets out 28 questions you should ask yourself when considering a new data generation solution. The questions aim to ensure that a data generation solution will be capable of creating fit-for-purpose data at scale. They aim to assess whether a solution will provide the data your teams need, without introducing unwanted complexity, bottlenecks, or overheads.
The types of data used at an enterprise are constantly shifting, as teams and systems implement new technologies. This leads to a proliferation in interrelated data types, as legacy components sit alongside new technologies like cloud-based applications and Big Data systems.
Data generation must be capable of creating data consistently across these new and legacy technologies. It should be equipped with an easily extensible range of connectors, as well as a range of techniques for generating data across systems. Otherwise, generation will not scale over time, and will not be future proofed as new technologies are adopted. You might find then be forced back to using copies of production data as your data generation tool cannot generate the data you need.
Synthetic data generation must be capable of meeting a proliferation in interrelated data types needed in testing, development and CI/CD.
When considering approaches to data generation, ask yourself:
A common concern with synthetic data generation is that it will be too time-consuming and unreliable when defining data generation for complex data. A data generation tool should therefore be intuitive and easy-to-use, while maximising reusability to boost efficiency.
When considering synthetic data generation, ask yourself:
In addition to supporting the full range of data types at an organisation, data generation should integrate seamlessly into both manual and automated ways of working. A lack of integration risks creating dependencies and bottlenecks, as data generation will require manual intervention within otherwise automated pipelines.
Synthetic data generation should furthermore be easily combinable with other technologies and techniques for creating and manipulating data. Otherwise, disparate processes for creating and provisioning data will require manual configuration and alignment. Running these complex, interrelated processes manually in turn risks creating inconsistent and misaligned data sets, undermining effective testing and development.
When deciding techniques for synthetic test data generation, ask yourself:
To ensure scalability and adoption, data generation should maximise automation and reusability, while minimising repetitive manual configuration.
Manual approaches and narrowly defined (non-event-driven) automation are typically limited in scope, requiring fresh configuration for each subtly different scenario. They further require rework and reconfiguration as data, scenarios, systems, and requests change. This continuous manual intervention is not sustainable, given the continuously increasing demand for data demand at enterprises today.
Data generation should form part of an automated service, capable of providing data for different requesters and scenarios on-the-fly. Learn more about automated "find and makes".
When creating a strategy for synthetic data generation, ask yourself:
Data generation does not just need to create data for numerous interrelated data types; the logic that must be reflected in that data can also be immensely sophisticated.
This includes the logic and structure of common types of data, such as the varying logic and syntax for creating different types of Social Security Number. At the same time, accurate data for testing and development must reflect and respect system logic, for instance reflecting temporal trends and sequences.
When deciding an approach for synthetic data generation, ask yourself:
Today, data is required for a wide range of different purposes and scenarios. Testing and development, for instance, requires data of different volumes, density, and variety. Functional testing, for instance, might require high-variety data that’s of lower volumes than the data needed in stress testing, while unit testing and smoke testing will require different data sets yet again.
Generating data of the wrong volume or contents will undermine the accuracy and rigour of testing and development, while introducing bottlenecks. Meanwhile, different data sets must be kept up-to-date, including aligning data for different versions of system components.
Data generation should be informed by granular analysis, such as data comparisons, coverage analysis, density analysis and kurtosis.
When deciding on an approach to synthetic data generation, ask yourself:
The types of data and integrated technologies used at enterprises continues to grow ever-more complex, while the constant need for varied data is growing more demanding across teams and frameworks. Meanwhile, compliance and data privacy requirements continue to evolve.
Automatically generating fictitious data for a range of different testing and data scenarios offers to remove data bottlenecks, while avoiding the use of raw production data in non-production environments.
Any such solution must be scalable and robust, capable of quickly generating data that is equal in complexity to your systems and development needs. The 28 questions set out above are intended to help you identify such a solution.
If you’re reading this, you’re probably already painfully familiar with the complaints that Curiosity hear from organisations seeking alternatives to...
Rigorous testing at the speed of today’s release cycles requires on demand access to “good” data. That means data combinations with which to execute...
For many organisations, test data “best practices” start and end with compliance. This reflects a tendency to focus on the problem immediately in...
A glance at industry research from recent years shows that test data remains one of the major bottlenecks to fix in DevOps and CI/CD:
Part one in this article series summarized the shockingly high failure rates for migration projects, identifying data migration as a key area of...
It’s 2024 and the risks associated with poor test data practices show no signs of abating.
I’ve been harping on about GDPR and other recent developments in compliance for years now, and it’s good to see QA organisations are now seriously...
Discover how Test Data Automation can help you automate your test data management by reading the infographic below! Curiosity's Test Data Automation...
Today, many organisations rely on rudimental tools and techniques for creating and managing their test data. These outdated techniques not only...