In the world of software development, generative AI has established itself as a formidable ally, assisting developers in coding and detecting potential problems in their applications.
My own experience with ChatGPT has been akin to working with a junior programmer, but with the added benefit of near-instant solutions for review. The ability to delegate tasks to ChatGPT and then seamlessly integrate or resolve issues with the provided solution empowers developers to move at an incredible pace.
Recently, Paul Gerrard released a series of videos showcasing the application of ChatGPT for software testing, with a focus on test automation. The results have been highly impressive, demonstrating how quickly you can transitioning from zero automation to functional test automation. However, there are concerns that need to be addressed to ensure the industry moves in the right direction.
To discover how you can start reaping the benefits of generative AI for your software testing, watch me, Paul and Jonathon Wright have a live roundtable discussion, available on demand.
Merely generating tests without proper understanding of the application and requirements does not advance the industry. In fact, it has the potential to move software quality backwards.
The purpose of quality assurance is to provide confidence that what is being delivered meets the initial software requirements and functions as expected. Automation and manual testing are vehicles for demonstrating that confidence to stakeholders.
The exception to the rule is tech giants like Google who can largely manage with minimal automation, relying on well-designed A/B which roll back instantaneously on the presence of defects.
To effectively use AI in testing and provide confidence to stakeholders, it is crucial to comprehend how the AI is testing the application, and the underlying testing methodology which has been applied.
Furthermore, most enterprise applications possess business logic and rules that require thorough testing. These rules are often ingrained in the knowledge of subject matter experts (SMEs) and may not be readily available for training / inputting into generative AI models. To overcome this challenge, collaboration between SMEs and AI models is essential.
By incorporating the expertise of SMEs into AI-driven testing processes, a more accurate and comprehensive testing framework can be achieved.
Incorporating a feedback loop is crucial for enhancing the performance of generative AI in understanding business logic, as it allows the AI model to continuously learn and improve.
At the same time, visualization plays a vital role in offering stakeholders transparency and insight into the testing process, ensuring they are well-informed about the aspects of the application being tested and the methodologies employed.
Visualisations communicate what is being tested, how, and to what extent.
By combining these two elements, we can foster a more effective and efficient software testing ecosystem that not only maximizes the potential of AI integration, but also builds stakeholder confidence in the quality of the applications being delivered.
Paul Gerrard has presented an approach that features models as a collaboration hub for improving a tester's understanding of an application, enabling them to test more effectively. We foresee an extension of this model to include generative AI that comprises the following steps:
A “model in the middle” approach to generative AI in testing.
This model in the middle process moves away from the black-box approach of generative AI, encouraging collaboration between AI models and testers to create a more efficient and effective software testing ecosystem, while also providing confidence in the tests and software being delivered.
The following video shows one application of this approach, leveraging generative AI to generate and refine requirements models and tests:
By moving away from the black-box approach of generative AI and adopting a more collaborative process, we can achieve several benefits:
The integration of feedback loops and visualization into generative AI for software testing is a promising approach for enhancing the testing process. Through it, we can create a more efficient and effective software testing ecosystem. This can be achieved in three broad steps:
This approach not only ensures the quality of the tests generated but also maximizes the potential benefits of AI integration in the software testing industry.
To discover how you can start reaping the benefits of generative AI for your software testing, join Paul Gerrard, Jonathon Wright and me for a roundtable discussion, now available on demand.