Skip to the main content.

Curiosity Modeller

Design Complex Systems, Create Visual Models, Collaborate on Requirements, Eradicate Bugs and Deliver Quality! 

Product Overview Solutions
Success Stories Integrations
Book a Demo Release Notes
Free Trial Brochure

Enterprise Test Data

Stream Complete and Compliant Test Data On-Demand, Removing Bottlenecks and Boosting Coverage!

Explore Curiosity's Solutions

Our innovative solutions help you deliver quality software earlier, and at less cost!

robot-excited copy-1              AI Accelerated Quality              Scalable AI accelerated test creation for improved quality and faster software delivery.

palette copy-1                      Test Case Design                Generate the smallest set of test cases needed to test complex systems.

database-arrow-right copy-3          Data Subsetting & Cloning      Extract the smallest data sets needed for referential integrity and coverage.

cloud-cog copy                  API Test Automation              Make complex API testing simple, using a visual approach to generate rigorous API tests.

plus-box-multiple copy-1         Synthetic Data Generation             Generate complete and compliant synthetic data on-demand for every scenario.

file-find copy-1                                     Data Allocation                  Automatically find and make data for every possible test, testing continuously and in parallel.

sitemap copy-1                Requirements Modelling          Model complex systems and requirements as complete flowcharts in-sprint.

lock copy-1                                 Data Masking                            Identify and mask sensitive information across databases and files.

database-sync copy-2                   Legacy TDM Replacement        Move to a modern test data solution with cutting-edge capabilities.

Explore Curiosity's Resources

See how we empower customer success, watch our latest webinars, read our newest eBooks and more.

video-vintage copy                                      Webinars                                Register for upcoming events, and watch our latest on-demand webinars.

radio copy                                   Podcasts                                  Listen to the latest episode of the Why Didn't You Test That? Podcast and more.

notebook copy                                           eBooks                                Download our latest research papers and solutions briefs.

calendar copy                                       Events                                          Join the Curiosity team in person or virtually at our upcoming events and conferences.

book-open-page-variant copy                                          Blog                                        Discover software quality trends and thought leadership brought to you by the Curiosity team.

face-agent copy                               Help & Support                            Find a solution, request expert support and contact Curiosity. 

bookmark-check copy                            Success Stories                            Learn how our customers found success with Curiosity's Modeller and Enterprise Test Data.

file-document-multiple (1) copy                                 Documentation                            Get started with the Curiosity Platform, discover our learning portal and find solutions. 

connection copy                                  Integrations                              Explore Modeller's wide range of connections and integrations.

Better Software, Faster Delivery!

Curiosity are your partners for designing and building complex systems in short sprints!

account-supervisor copy                            Meet Our Team                          Meet our team of world leading experts in software quality and test data.

calendar-month copy                                         Our History                                Explore Curiosity's long history of creating market-defining solutions and success.

check-decagram copy                                       Our Mission                                Discover how we aim to revolutionize the quality and speed of software delivery.

handshake copy                            Our Partners                            Learn about our partners and how we can help you solve your software delivery challenges.

account-tie-woman copy                                        Careers                                    Join our growing team of industry veterans, experts, innovators and specialists. 

typewriter copy                             Press Releases                          Read the latest Curiosity news and company updates.

bookmark-check copy                            Success Stories                          Learn how our customers found success with Curiosity's Modeller and Enterprise Test Data.

book-open-page-variant copy                                                  Blog                                                Discover software quality trends and thought leadership brought to you by the Curiosity team.

phone-classic copy                                      Contact Us                                           Get in touch with a Curiosity expert or leave us a message.

4 min read

Experiments with GPT Vision for Modelling: A Journey from Screenshots to Whiteboards

Experiments with GPT Vision for Modelling: A Journey from Screenshots to Whiteboards

The landscape of artificial intelligence is rapidly evolving. The recent announcement of GPT-4 with vision capabilities by OpenAI stands as a groundbreaking development in multi-modal language models. Although this feature is still in its preview phase, the Curiosity team is always on the lookout for innovative solutions. We were immediately captivated and excited by the potential of integrating this advanced technology into our Test Modeller product.

Our journey with GPT-4 began with a simple, yet ambitious, goal: to integrate cutting-edge AI into model-based testing and quality assurance, realising the combined value of both. In this blog, we showcase some of our early experiments using GPT-4's vision capabilities within Test Modeller - a tool designed for collaborative, quality-focused development.

Preview #1: Screenshots to Models

Model-based design and testing provides a structured, collaborative development methodology, that maximises accuracy and efficiency across the whole development lifecycle. Collaborative models engineer better quality requirements to drive development, while simultaneously generating optimised tests to rigorously validate the development work.

However, it's rare to start a modelling project from scratch. More often, we find ourselves needing to model existing applications or business logic. This led us to an intriguing question: How effectively could OpenAI's GPT Vision handle the task of creating models from screenshots of existing applications?

To put this to the test, we chose a practical example: a registration form from an ecommerce system. Our goal was to convert this form into a model using GPT Vision, feeding the screenshot into the system. The co-pilot then efficiently analysed and translated the form image into a linear model.

This model served as a starting point but did not give full coverage of the system behaviour when responding to different inputs, and the underlying business logic for validations, which ultimately triggers error or success conditions. This is one area where the power of modelling comes into play, with the ability to further refine and develop the model with SME knowledge of negative scenarios and edge cases, rapidly creating complete system specifications that can then auto-generate tests.

Preview #2: Realising The Value of Siloed Information and Artifacts

Software teams often invest time, thought and effort formulating information in static, siloed artifacts. This includes ideation sessions on whiteboards, converted into photos at best, or wireframes with no downstream integrations. Similarly, BAs often formulate valuable requirements in “static” diagrams with limited integration or automated downstream applications.

These siloed artifacts quickly become out-of-date, as they do not react to changes from across the SDLC. The siloes additionally hinder communication and feedback, while downstream teams overlook valuable artifacts due to the time and manual effort associated with converting them into code and tests.

These limitations not only hamper the efficiency of workflow documentation and requirements gathering; they significantly increase manual effort, miscommunication and the potential for bugs in development and testing.

We are leveraging GPT Vision to un-silo this valuable information, converting “static” artifacts into “living documentation”. For instance, we can convert computer-generated flowcharts, wireframes and whiteboard images directly into Modeller’s flowcharts. These “active” integrated flows are then used to drive collaborative requirements engineering, accurate development and automated test generation.

For flowcharts, this process begins by creating visual representations of the application's processes, which are then fed into the system. The co-pilot intelligently analyses these visuals, interpreting and converting them into detailed, editable flowcharts within our modelling tool.

This approach is particularly effective for flowcharting applications that are visually rich, but lack the necessary export functionalities. Bypassing the need for manual data entry or complex integration solutions, we can swiftly convert static images into dynamic, interactive models that accurately reflect an application's intended behaviour. This process yields accurate results because the input images (flowcharts) are themselves computer-generated, and therefore lend themselves well to automated, AI-augmented analysis.

Preview #3: Wireframes to Flowcharts

Wireframes are the skeletal framework of a digital application, outlining its structure and layout, without delving into the finer details of design. The transition from design wireframes to comprehensive flowcharts is a crucial step in the model-based testing process. Our challenge was to convert these wireframes into detailed flowcharts that not only represent the structure, but also encapsulate the application's flow and functionality.

The system takes the wireframes and intelligently interprets them, identifying key elements like navigation menus, input fields, and user interaction points. From these elements, Test Modeller’s “modeller co-pilot” constructs a flowchart that maps out how these components interact and connect, turning a static layout into a dynamic flow of processes.

One unique challenge in this process is ensuring that the nuances of a wireframe are accurately captured in the flowchart. Wireframes are often high-level and may not include detailed information about every user interaction. Secondly, the wireframes rarely contain the business logic which sits behind a front-end design.

Preview #4: Whiteboard to Flowcharts

Whiteboarding is a key, collaborative approach to ideation and design for most enterprises today. Using GPT-4's vision capabilities, we capture these initial bursts of creativity and structure them into actionable models.

We achieve this by taking images of the whiteboard drawings and importing them into Test Modeller using the modeller co-pilot. Doing this allows us to convert these often chaotic and unstructured drawings into clear, organized flowcharts and models.

The process begins with a simple photograph of the whiteboard. Modeller co-pilot then analyses the content, deciphering text, diagrams, and even hastily drawn shapes. It intelligently recognizes the relationships and hierarchies within these drawings, transforming them into a digital format that serves as the starting point for more detailed models.

While this feature has opened new avenues in capturing and digitizing spontaneous ideas, it's important to acknowledge its current limitations. The accuracy of converting these drawings into models largely depends on the clarity of the whiteboard sketches. In instances where drawings are overly abstract or text is illegible, the system may face challenges in accurately interpreting the content.

Human-centric development, AI acceleration

In our exploration of GPT Vision through our co-pilot functionality in Modeller, we've made significant strides in enhancing the modelling processes with additional sources of data which often exist in images. From transforming application screenshots into detailed models and importing flowcharts from non-exportable applications, to converting wireframes and whiteboard sketches into structured flowcharts, each method has showcased the power of AI.

A key insight drawn from all of these examples is the inherent limitation of relying solely on image-based data. This underscores the critical role of having human expertise in the loop, to scrutinize AI-generated results and refine them with the knowledge of Subject Matter Experts (SMEs). With AI’s current capabilities and limitations, this is essential in ensuring high quality and accuracy of any AI generated content.

The synergy between AI capabilities and human expertise through modelling paves the way for a more accurate, and accelerated approach to software quality. To join Curiosity on our journey to create this synergy at enterprise scale, book a time to speak with one of our experts today:

Book a Demo

Shift Left Quality With Curiosity's Modeller

Shift Left Quality With Curiosity's Modeller

Software delivery teams across the industry have embraced agile delivery methods in order to promote collaboration between teams and deliver new...

Read More
How Curiosity Accelerate Quality Software Delivery - Infographic

How Curiosity Accelerate Quality Software Delivery - Infographic

Discover how Curiosity helps organisations delivery better quality software, faster. This infographic highlights key ways that Curiosity's tools,...

Read More
The Philosophy of Application Delivery:Was Plato a Model-Based Tester?

The Philosophy of Application Delivery:Was Plato a Model-Based Tester?

This is part one of three of the “Philosophy of Application Delivery” eBook. The eBook contains Curiosity Software Ireland’s musings on the nature of...

Read More
Parasoft and Curiosity partner!

Parasoft and Curiosity partner!

Parasoft, the global leader in automated software testing, today announced they have partnered with Curiosity Software Ireland, specialist provider...

Read More
Curiosity, Omnext and Specialisterren Announce code-less automation

Curiosity, Omnext and Specialisterren Announce code-less automation

Curiosity Software Ireland, specialist vendor in model-based testing, today announced its partnership with Omnext, experts in software quality...

Read More
Announcing Test Automation for Technologies Built with Apache Solr

Announcing Test Automation for Technologies Built with Apache Solr

Curiosity Software Ireland, a specialist vendor in test data management and model-based test automation, today announced support for Apache Solr. The...

Read More
Removing Barriers to In-Sprint Testing and Development - Infographic

Removing Barriers to In-Sprint Testing and Development - Infographic

Test automation must be lightweight, re-usable and easy to apply, in order to help organisations, ease its implementation enterprise wide. Curiosity...

Read More
Second decade of a new millennium: observations and predictions for QA

Second decade of a new millennium: observations and predictions for QA

The past decade has seen me sell one start-up, join a multi-national, and co-found my latest venture: Curiosity Software Ireland. I am grateful for...

Read More
Curiosity Software announce automated migration testing

Curiosity Software announce automated migration testing

Curiosity Software Ireland, specialist vendor of visual test automation, today announced its dedicated solution for mainframe migration testing. The...

Read More