Stop Guessing. Test the Bot.
AI chatbots challenge many of the assumptions behind traditional software testing. From a user point of view, the real question is simple: how do we build confidence in what people actually experience? Chatbots are conversational, context-sensitive, and often variable in their responses, which makes them hard to judge with a simple pass/fail mindset.
In this full-day tutorial, we take a practical black-box approach to testing AI chatbots. We start by building a basic understanding of how chatbot behavior is shaped by prompts, memory, tools, rules, and surrounding application logic. We then focus on what can be observed and tested from the outside: responses, conversation flows, boundaries, integrations, fallback behavior, and visible failure modes.
From there, we apply classic test design techniques and look at practical ways to evaluate quality. In the final part, we use exploratory and metamorphic testing to uncover language-specific risks such as ambiguity, manipulation, and hidden weaknesses.