More and more production code is being generated by AI-powered tools such as GitHub Copilot, Claude, and ChatGPT. What at first glance appears to be a productivity gain has a fundamental consequence for quality assurance: there is no longer anyone who knows the code in detail.
At the same time, developers' knowledge about frameworks is eroding. Those who use AI to generate the code are becoming less and less familiar with the mechanisms of Spring Boot, Hibernate or Angular. When problems arise which are deeply rooted in these framework mechanisms, the knowledge to diagnose them is lacking. Observability data — metrics, logs, and traces — become the only window into this increasingly opaque black box. So as QA specialists we have to create the tests that check precisely these values. Prometheus, Loki and Jaeger offer open APIs which we can use to gain inside into the data our application is providing.
This presentation will introduce you into the magic of observility testing. Therefore we use a demo application with OpenTelemetry (OTel) to show how observability data can be tested on two complementary levels. The architecture maps a typical OTel pipeline: an application generates metrics, logs and traces using the OTel SDK, sends them via the OTel Collector to Prometheus (metrics), Loki (logs) and Jaeger (traces), where they are visualised via Grafana and Jaeger UI. Something can go wrong at any point in this chain — that's why we need these tests.