Most QA teams are still building test data the old way, manually every single time. They're typing data into the front end, creating Flat files, reserving numbers from spreadsheet, and refreshing entire databases when they need something new. It's slow. It's error prone. And it's burning through your teams' time like nothing else.
We worked with a national Cell Phone provider that was hand-building test data in Excel, and when I say hand-building, I mean someone was literally typing in data, twenty rows a day. That's it. They needed millions of records across twenty different end points to properly test their systems, and at that rate, they'd need over a century to build what they needed. The math is depressing, but it's also the reality for a lot of teams right now.
Here's the thing though: you probably already have most of the data you need sitting in your test environments right now. But instead of easily finding and allocating it, teams just create everything from scratch, making the same data over and over while ignoring what's already there. There's a better approach, and it changes everything. It's called find and make.
Why Manual Data Building Is Your Actual Bottleneck
When your QA team spends their days typing data instead of testing software, something fundamental has gone wrong. At the Cell Phone company, the problem wasn't just speed, although that was bad. The real issue was complexity. They had twenty interrelated business objects, and each device set could have up to seventeen linked end- points. Unique IDs, meter-point numbers, addresses, all enforced automatically. You can't do that by hand, and even if you somehow could, it would take forever and you'd make mistakes in the process.
The second problem was security. Their system blocked direct database access, which meant all data had to go through the front end. But the front-end loader had limits on how much data you could upload at once, so what actually happened was queues formed. Teams waited. They stalled while they manually prepared batches of test data, and meanwhile the rest of the testing pipeline just sat there.
Finally, there's the accuracy problem, which is less obvious but worse. When you're manually allocating numbers from spreadsheet pools with millions of options, duplicates happen. Ghost failures in testing become common as testers spend days chasing bugs that don't exist in the code. The reason for the failures are usually the data they're testing against has changed, does not match anymore or someone else has transacted against the data. It’s not the software it’s the process to manage your test data. It's incredibly wasteful.
The Answer Is Actually Pretty Simple: Find Before You Make
Instead of manually managing your data, try this: look at what you already have, find the test data that already exists and matches with what you need, then create only what's missing. If you can’t find it, make it automatically, that's the whole idea.
At the Cell Phone company, they modelled their entire twenty object environment and built a find-and-make pipeline. Here's how it actually works in practice. The "find" part searches existing data for users and roles that already match what you're testing. Instead of uploading new data through that slow front-end loader every time, you just reuse what's already there. Problem solved. No bottleneck. The "make" part is surgical. You only generate the specific records you actually need, and they're created with full business integrity supplementing the data you already have. You're not building a new world from scratch; you're just filling in the gaps.
The Results Speak for Themselves
Before this approach: twenty rows per day, two months to build two thousand records.
After: fifty thousand device sets per day, complete datasets in a single day.
Let that sink in for a moment. Each device set contains between nine and twenty rows, so they went from twenty rows a day to somewhere between half a million and a million rows a day. That's not an incremental improvement. That's a complete transformation of what's possible.
Suddenly they could run proper performance tests. They could test at scale. Their teams got their time back. No more Excel nightmares, no more endless queues waiting for data to be prepared. The reason this worked was because they stopped thinking of test data as something you build once and copy everywhere. Instead, they made finding and reusing existing data the default behaviour, and generation became the exception, something you only did when you actually needed something new.
It Integrates Naturally into Your Automation
The best part is that this fits naturally into your existing automation framework. When a test runs, it doesn't just hope the data exists. It asks a simple question: "Is there test data in the system that matches what I need?" If yes, use it. If no, generate it, insert it, and proceed. No waiting. No manual intervention. Your automation framework handles it all.
Stop Building Everything from Scratch
Here's what you learn from a project like this: manual test data preparation is killing your velocity. It's not just slow, it makes teams miss opportunities to actually test properly. If your QA team is trapped in that twenty-rows-a-day grind right now, it's time to change. Look at the data you already have, find what you can reuse, make only what's missing. You'll move faster, your tests will be more reliable, and your teams can finally focus on finding actual bugs instead of building data.
The goal isn't to make data. The goal is to test software. Find data before you make data, and you'll get there a lot faster.
