Skip to the main content.

Curiosity Modeller

Design Complex Systems, Create Visual Models, Collaborate on Requirements, Eradicate Bugs and Deliver Quality! 

Product Overview Solutions
Success Stories Integrations
Book a Demo Release Notes
Free Trial Brochure
Pricing  

Enterprise Test Data

Stream Complete and Compliant Test Data On-Demand, Removing Bottlenecks and Boosting Coverage!

Explore Curiosity's Solutions

Our innovative solutions help you deliver quality software earlier, and at less cost!

robot-excited copy-1              AI Accelerated Quality              Scalable AI accelerated test creation for improved quality and faster software delivery.

palette copy-1                      Test Case Design                Generate the smallest set of test cases needed to test complex systems.

database-arrow-right copy-3          Data Subsetting & Cloning      Extract the smallest data sets needed for referential integrity and coverage.

cloud-cog copy                  API Test Automation              Make complex API testing simple, using a visual approach to generate rigorous API tests.

plus-box-multiple copy-1         Synthetic Data Generation             Generate complete and compliant synthetic data on-demand for every scenario.

file-find copy-1                                     Data Allocation                  Automatically find and make data for every possible test, testing continuously and in parallel.

sitemap copy-1                Requirements Modelling          Model complex systems and requirements as complete flowcharts in-sprint.

lock copy-1                                 Data Masking                            Identify and mask sensitive information across databases and files.

database-sync copy-2                   Legacy TDM Replacement        Move to a modern test data solution with cutting-edge capabilities.

Explore Curiosity's Resources

See how we empower customer success, watch our latest webinars, read our newest eBooks and more.

video-vintage copy                                      Webinars                                Register for upcoming events, and watch our latest on-demand webinars.

radio copy                                   Podcasts                                  Listen to the latest episode of the Why Didn't You Test That? Podcast and more.

notebook copy                                           eBooks                                Download our latest research papers and solutions briefs.

calendar copy                                       Events                                          Join the Curiosity team in person or virtually at our upcoming events and conferences.

book-open-page-variant copy                                          Blog                                        Discover software quality trends and thought leadership brought to you by the Curiosity team.

face-agent copy                               Help & Support                            Find a solution, request expert support and contact Curiosity. 

bookmark-check copy                            Success Stories                            Learn how our customers found success with Curiosity's Modeller and Enterprise Test Data.

file-document-multiple (1) copy                                 Documentation                            Get started with the Curiosity Platform, discover our learning portal and find solutions. 

connection copy                                  Integrations                              Explore Modeller's wide range of connections and integrations.

Better Software, Faster Delivery!

Curiosity are your partners for designing and building complex systems in short sprints!

account-supervisor copy                            Meet Our Team                          Meet our team of world leading experts in software quality and test data.

calendar-month copy                                         Our History                                Explore Curiosity's long history of creating market-defining solutions and success.

check-decagram copy                                       Our Mission                                Discover how we aim to revolutionize the quality and speed of software delivery.

handshake copy                            Our Partners                            Learn about our partners and how we can help you solve your software delivery challenges.

account-tie-woman copy                                        Careers                                    Join our growing team of industry veterans, experts, innovators and specialists. 

typewriter copy                             Press Releases                          Read the latest Curiosity news and company updates.

bookmark-check copy                            Success Stories                          Learn how our customers found success with Curiosity's Modeller and Enterprise Test Data.

book-open-page-variant copy                                                  Blog                                                Discover software quality trends and thought leadership brought to you by the Curiosity team.

phone-classic copy                                      Contact Us                                           Get in touch with a Curiosity expert or leave us a message.

3 min read

Ensuring The Efficiency and Effectiveness of Software Testing Contracts

Ensuring The Efficiency and Effectiveness of Software Testing Contracts

Using Function Point Analysis and model-based testing to objectively measure services.

A perpetual challenge in managing software testing projects is gauging whether testing efforts are efficient, valuable and effective enough. Testing can consume 25-50% of total software delivery project costs, so optimizing the value delivered is crucial. Yet, efficiency and effectiveness have proven difficult to objectively quantify in testing contracts and execution.

This deficiency stems from the reliance on subjective metrics, like the number of test cases covered or defects found. Effective measurement and optimisation of services, by contrast, requires quantifiable size and complexity analysis.

Adopting function point analysis and model-based testing could offer an objective solution.

The Subjectivity of Testing Efficiency

Efficiency determines how quickly testing generates outcomes relative to the time and resources invested. Managers often assume that specifying tester hours in a contract provides efficiency accountability. Yet, raw hours reveal little about actual throughput.

Ten senior tester hours should provide more coverage than ten junior tester hours. Skill levels, existing test assets, and tooling all further sway efficiency. Hours get booked regardless of quality or progress. Even utilising story points or planned test cases offer limited insights into efficiency. Velocity varies across test case complexity, which subjective story point estimates fail to capture.

With so many variables obfuscating internal efficiency, managers typically resort to crude outcomes. They track metrics like the total number of test cases executed or defects found. However, these outcomes reveal nothing about the efficiency of the testing process itself. Testing could follow wasteful paths yet still log cases and bugs. These results-oriented metrics incentivize the wrong behaviours.

The Pitfalls of Measuring Test Effectiveness

Effectiveness determines if testing delivers business value and provides sufficient risk mitigation. However, effectiveness has traditionally proven even more nebulous to quantify than efficiency.

Test coverage and defect counts provide one-dimensional metrics of effectiveness at best. High test case coverage does not guarantee that meaningful scenarios have been exercised. More critically, a zero-defect output could wrongly suggest the system was rigorously tested, even if large swaths have been missed.

Testers themselves struggle to articulate objective criteria for effectiveness. Exploratory testing provides high value yet follows unpredictable paths. Defining completion requires subjective judgment calls on residual risk. This leaves managers wanting tangible evidence of effectiveness.

Asking stakeholders directly if they “feel” testing was effective introduces bias and unfair accountability. People often equate effectiveness with liking the testers or delivery pace, rather than objective technical insights. Yet unlike development and design, few measurable artefacts emerge from testing to judge independently.

The Perils of Outsourced Testing Contracts

The subjectivity around efficiency and effectiveness is amplified when testing gets outsourced. Without visibility into internal testing processes, managers craft contracts around outputs to create supplier accountability.

Contracts specify quantities like the number of test cases to design, hours to execute, or a maximum number of defects to find. While understandable, this incentivizes suppliers to game numbers, instead of delivering rigorous testing. For example, testers may author questionable cases to meet volume requirements or cap defect finds to avoid penalties.

Even well-intentioned suppliers struggle to convey the nuances of their expertise into contracted metrics. Searching for definitive requirements to prevent scope creep, managers end up with prescriptive contracts biased towards activity over aligning to testing needs. All the while, subjectivity still hinders determining true efficiency and effectiveness.

Quantifying Scope and Complexity Through FPA and Model-Based Testing

To inject objectivity into managing testing efficiency and effectiveness, testing contracts and execution should shift from subjective metrics to quantifying scope size and complexity. This enables comparative analysis across projects, calibrating optimal rates.

Function point analysis (FPA) offers standardized sizing of the functional scope to be tested, independent of technology choices. By quantifying size through weighted input, output, interaction, file, and interface components, FPA provides the "what" to test objectively.

Model-based testing (MBT) then addresses the "how" by generating test cases from system models:

A flowchart provides a clear representation of the “LBW” rule in cricket, and auto-generates paths (“tests”) through the logic.

A flowchart provides a clear representation of the “LBW” rule in cricket, and auto-generates paths (“tests”) through the logic.

Models codify complex real-world behaviour in a manner that’s more accurate than alternative forms of specification. MBT reveals testing needs unbiased by individual testers’ skills. Automated test case generation also improves execution efficiency predictably.

FPA and MBT integrate to provide:

  1. Test sizing baselines - FPA sizes functionality to test for apples-to-apples comparisons across projects.

  2. Test scoping guides – MBT’s models highlight testing gaps not covered in FPA sizing.

  3. Test efficiency rates – Comparing FPA sizes to MBT test cases executed over time provide throughput metrics.

  4. Test coverage transparency - MBT tests tie back to functional areas, showing coverage comprehensively.

  5. Continuous efficiency monitoring/alignment - MBT auto-generates new test cases as models update, keeping testing in sync efficiently.

MBT reveals testing needs unbiased by individual testers’ skills. Automated test case generation also improves execution efficiency predictably.

These quantified artefacts enable objective testing efficiency and effectiveness tracking. Managers gain data to calibrate optimal testing processes and contractor performance, grounded in evidence versus subjective judgment calls.

Objectivity to Navigate Complexity

Suboptimal software testing has broad consequences downstream, from delayed releases to reputational damage and high-profile defects. Poor testing burns significant time and budget, yet organisations often lacks the metrics to course correct.

By quantifying test scope, complexity, execution, and coverage through FPA and MBT, teams finally gain the objective insights needed to master efficiency and effectiveness. Testing transforms from a black box to an optimized, value-add driver of quality and business outcomes.

Speak to a Curiosity expert today to get started with your model-based testing journey! 

Schedule a Demo

Going lean on your testing approach

7 min read

Going lean on your testing approach

When teams are looking to transform, optimize, or cut costs in testing, where do they first look? More often than not, they follow the advice given...

Read More
Accidental Complexity is killing your testing efforts and IT budget

6 min read

Accidental Complexity is killing your testing efforts and IT budget

You’re working hard to transform your ways of working, with a range of different goals. Common aims of digital transformations include:

Read More
Model-Based Testing Can Lead the Way in IT Change

6 min read

Model-Based Testing Can Lead the Way in IT Change

IT change remains a persistent struggle for most organisations today. Software teams are aware of the need to move faster and be more agile; yet,...

Read More
Continuous Development: Building the thing right, to build the right thing

7 min read

Continuous Development: Building the thing right, to build the right thing

Test Automation is vital to any organisation wanting to adopt Agile or DevOps, or simply wanting to deliver IT change faster.

Read More
5 Reasons to Model During QA, Part 4/5: Faster QA Reaction Times

6 min read

5 Reasons to Model During QA, Part 4/5: Faster QA Reaction Times

Welcome to part 4/5 of 5 Reasons to Model During QA! If you have missed any previous instalments, use the following links to see how modelling can:

Read More
5 Reasons to Model During QA: “Shift Left” QA Uproots Design Defects

6 min read

5 Reasons to Model During QA: “Shift Left” QA Uproots Design Defects

Model-Based Testing (MBT) itself is not new, but Model-Based Test Automation is experiencing a resurgence in adoption. Model-Based Testing is the...

Read More
Q&A – Cypress.io UK Community Meetup – Model-Based Testing for Cypress

8 min read

Q&A – Cypress.io UK Community Meetup – Model-Based Testing for Cypress

Yesterday, I had the immense pleasure of presenting on the second Cypress.io UK Community Meetup. My talk was called “Introducing model-based testing...

Read More
5 Reasons to Model During QA, Part 3/5: Coverage Focuses QA

5 min read

5 Reasons to Model During QA, Part 3/5: Coverage Focuses QA

Welcome to part 3/5 of 5 Reasons to Model During QA! Part one of this series discussed how modelling enables “shift left” QA, eradicating potentially...

Read More
Evolving or Devolving? A Deep Dive into AI's Impact on Testing

7 min read

Evolving or Devolving? A Deep Dive into AI's Impact on Testing

Since the initial launch of ChatGPT, interest in AI has exploded across almost every industry sector. The unique ability to solve problems by...

Read More