A Journey toward Generative AI

Since their introduction, generative AI tools such as ChatGPT and GitHub Co-Pilot have promised great improvements in efficiency and quality across the entire software development lifecycle. GitHub has published impressive results, indicating a more than 50% gain in productivity, as well as substantial increases in employee satisfaction. These results are supported by similar findings from other sources. However, at the same time the number of publications observing potential negative effects of the tools is growing, as more data becomes available. Since AI-Tools make it easier to write code, they also make it easier to write code “that should never have been written in the first place”.

With that in mind when planning to introduce generative AI-Tools into a company, the question becomes: how can this integration harness the power of generative AI while effectively minimizing any potential negative impact?

In this talk I will share firsthand insight gained during our ongoing journey to introduce ChatGPT and GitHub Co-Pilot at ZEISS for Software Development and Testing. While focusing on testing use cases I will describe:

• How we started our journey with an initial assessment.
• How we continued the roll out of the tools to a wider user base at ZEISS.
• What metrics we are planning to use to continuously monitor the effectiveness of the tools.
• In what areas the tools showed less effectiveness.
• What initial hypotheses did not work as expected.

With this talk I hope to provide you with key ideas and takeaways that will help you on your own journey toward generative AI, whether you are just starting out or are already on the way.