Large language models are non-deterministic. They show emergent reasoning and often take actions their engineers never anticipated. As a result, we no longer need to protect the system only from external threats; we must also protect it from itself. Trust cannot be added at the end. Engineers must build it throughout the application lifecycle: design, development, testing, deployment, and, critically, runtime. Just as human societies depend on checks and balances, agentic systems need oversight structures that balance generators, evaluators, and moderators.
This keynote begins by defining trust and turning it into a practical framework you can engineer, measure, and control. It then presents proven patterns, best practices, tools, and technologies for building agentic evaluation and observability frameworks. We will examine LLM-as-a-judge architectures and feedback loops, and explore techniques to reduce evaluation bias so evaluators themselves can be trusted. The session also introduces a harness design for trustworthy agentic coding and safe execution, addresses the challenge of providing trustworthy data, and concludes with a look at how the new TypeSafe’s Jev model has changed the rules of agentic evaluation.