Who Says It Holds?

Why software is shifting from velocity to liability, and what that does to the tester's
job.
A few weeks ago I sat in on a code review. One pull request, about 1,400 lines, produced in roughly
twelve minutes. The colleague who had kicked it off said the sentence we all know: "Looks good."
Nobody in the room laughed. But nobody contradicted him either. And in that second of silence was pretty much everything I want to write about here.
"Looks good" was a legitimate sentence for decades. The diff was small enough to hold in your head.
The person who wrote it was sitting right there. You could ask why line 40 looked the way it did. The sentence carried weight because behind it stood a human who knew what they had done.
That human doesn't exist for 1,400 generated lines. The sentence sounds the same. It no longer
means anything

The old deal

For thirty years we organized software around one bottleneck: building it. Building took the most
time and tied up the most people. So we pointed everything at building faster. Sprints, story points,
pipelines, continuous everything. Velocity was the scarce resource, and whoever increased it had
status.
Testers never chose their place in that deal. We were the last ones before the door. The people
asking questions when everyone else wanted to go home. They called us "the bottleneck," and a lot
of us got into the habit, over the years, of apologizing for doing our jobs.
I'm not writing this out of bitterness. Under the conditions of the time, the deal even made sense.
When building is expensive, anything that slows down building looks suspicious.
Except building isn't expensive anymore.

What the numbers say

You don't have to like AI tools to see what's happening. It's enough to lay a few studies side by side
that never coordinated with each other.
Google's DORA Report 2024[1] found that as AI adoption in a team went up, delivery stability went
down, and noticeably so. The explanation was uncomfortable: when you can produce code faster,
you build bigger batches, and big batches were always riskier. A year later the DORA Report 2025[2]
showed that throughput had recovered. Stability had not. Teams are shipping faster again, but not
more safely.

In the summer of 2025, METR[3] did something that is surprisingly rare: a randomized trial with
experienced open-source developers working in their own repositories. With AI tools, they took
about 19 percent longer on real issues. Afterwards, they believed they had been about 20 percent
faster. The gap between feeling and measurement is the real headline of that study. If even
professionals can't feel themselves slowing down, who notices when something is wrong?
The Stack Overflow Developer Survey 2025[4], with nearly 50,000 responses, supplies the mood. Two
thirds of developers name as their biggest frustration solutions that are "almost right, but not quite."
More developers distrust the accuracy of these tools than trust it. And the more experienced the
person, the lower the trust.
And then there's Veracode[5], who ran more than 100 language models through 80 coding tasks. In
roughly 45 percent of cases where a secure and an insecure option existed, the model picked the
insecure one. That number has not improved across model generations, even as the code got
syntactically cleaner and cleaner.
Four sources, four methods, one picture. The work has not disappeared. It has moved. From
producing to judging.

Almost right is the most expensive kind of wrong

A bug that's obvious costs little. A bug that looks plausible, compiles, collects its green checkmarks,
and then applies a discount twice in production costs a weekend, a customer, or a report to the
regulator.
Generated code doesn't produce more of the first kind. It produces dramatically more of the second.
That is the shift. Not "AI makes mistakes," people do too. The point is that the volume of plausible
material somebody has to judge is growing faster than the number of people who can judge it.
There's something sneaky about plausibility. Our brains are built to accept fluent things. A well-
formatted diff with sensible variable names and a confident commit message triggers the same
"that's fine" reflex as a well-formatted sentence from a person who sounds like they know what
they're talking about. The METR developers didn't feel slower because everything they saw looked
competent. Fluency is exactly the surface generated output is best at, and exactly the surface our
intuition is worst at seeing through.
Which is why the skill that matters here isn't reading code faster. It's the discipline of separating
what a thing claims from what a thing is. That discipline has a name, and it's older than software:
critical thinking. Asking where a claim came from. Asking what would have to be true for it to hold.
Asking what you would expect to see if it were wrong, and then going to look. None of that gets
easier when the material arrives faster. All of it gets more necessary.
And judgment doesn't scale with tokens.

Why I say "liability" and not "quality"

Quality is a soft word. Everyone is for it, nobody is against it, and when the deadline bites, it's the
thing you cut.
Liability is hard. Liability means that at the end there is a name.
You can't sue a model. It takes no responsibility for domain correctness, for system integrity, for
proof, for operations. Those four responsibilities used to be spread across many heads and slow
cycles. Now they land earlier and harder on fewer people.
This is not a metaphor. On 9 December 2026, the transposition deadline for the new EU Product
Liability Directive 2024/2853[6] runs out. From then on, software is explicitly a product, whether
installed, embedded, or delivered as SaaS. Updates included. Strict liability, no fault required. With
the burden of proof eased for the people harmed. If you want the one-page version for non-lawyers,
the German Chamber of Commerce in Hanover[7] has one. The short form: "Who checked that this
holds?" becomes a question a court asks.
Honestly, I don't find this frightening. I find it overdue. It makes visible what was always true.
Between "generated" and "holds" there has to be a human, and that human has to know what
they're doing.
 

"Is shifting" and "will shift"

I'm not going to claim this shift has arrived everywhere. It hasn't.
I know teams that still hang velocity charts on the wall in 2026 and are proud of their merge
frequency. I know regulated companies where the shift started in 2023, because nobody there
approves a diff they can't explain. And I know startups building a product right now with four people
and three agents who don't yet know that they will have a proof problem in eight months.
The direction isn't in dispute. The pace is. And the pace depends on when the first plausible wrong
thing gets expensive enough that somebody asks: who should have seen that?

What this means for testers

First the good news, because it's real.
What testers have always done is suddenly the scarce skill. Years ago, James Bach and Michael
Bolton drew a line between testing and checking[8]. Checking is what a machine can do: comparing an outcome to a known expectation. Testing is what only a human with judgment can do: exploring, doubting, finding the question nobody has asked yet. For a long time that distinction sounded like an academic quibble. Now it's the heart of the matter. Generated code can write its own checks and turn them green. It cannot test itself.

Strip the tooling away and what's left of testing is critical thinking applied to software. Not
scepticism as a mood, but as a method. The tester's reflex of asking "how would I know if I'm wrong about this?" before asking "does it work?" is the same move a good scientist, a good auditor, or a good journalist makes. We just happened to practise it on code. For years that made us look slow. In a world where anyone can produce a plausible answer in seconds, it makes us the only people in the room whose default setting is the right one.

The posture of treating an artifact as unreliable until shown otherwise used to be a character trait
you had to justify in retros. It is now the thing that protects a company before 9 December.

Now the uncomfortable news.

The job title "tester" does not automatically inherit this role. Proof work goes to whoever claims it.
And right now a lot of people are claiming it. ML teams build evaluation pipelines and don't call it
testing. Platform engineers build observability and don't call it testing. Domain experts sign off on
outputs and don't call it testing. If testers spend this period working through test cases against a
spec someone else wrote, the most valuable work of the next ten years goes to people who have
never been to a testing conference.

What actually changes, if you want to keep the role:

• From case depth to proof architecture. The question is no longer "How many cases do I have?" but "How would we know this holds, and how would we know it doesn't?" That is design work, not execution work.
• From deterministic to non-deterministic proof. A model answers differently today than tomorrow.Drift, false correctness, regression against known failures: these are testing problems with no textbook yet. Whoever solves them first defines the field.
• From sign-off at the end to spec at the beginning. Under AI, vagueness produces wrong code in minutes. The person who knows how to cut a requirement so that it contains an acceptance criterion and not just a wish now sits at the start of the cycle. That is a tester's skill. It was just never called one.
• From reading output to interrogating claims. Every generated artifact arrives with an implicit claim attached: this is what you asked for. The job is to treat that claim the way you'd treat any other unverified statement. What is the evidence? What did the spec actually say, as opposed to what the prompt assumed? Which cases did nobody mention because nobody thought of them? This is where critical thinking stops being a soft skill on a slide and becomes the daily unit of work.
• From bottleneck to gate owner. This is the social shift, and it's the hardest. Status used to hang on building. It has to hang on the gates now: who may close a spec, allow a merge, give a release. If that stays unclear, you get a hierarchy of prompting speed. I can hardly imagine a worse way to run a team. And then there's a question I don't have a finished answer to: where do the next testers come from? Pure implementation was the classic way into the profession. That door is closing. If entry in the future runs through proof and operations, we need learning paths that don't exist yet. And the thing those paths have to teach is the hardest thing to teach: not a tool, not a framework, but the habit of doubting well. You can learn a test framework in a weekend. Learning to notice when you are being convinced rather than informed takes years and usually a few painful production incidents. That may be the most important job almost nobody is talking about.

Stop apologizing

In twenty years I've met a lot of testers who apologized for their job. For the question that delayed
the sprint. For the bug filed on a Friday afternoon. For the "wait, are you sure?".

That reflex was a product of the old deal. The deal is over.
The ability not to believe a plausible result until it has proven itself is right now the most valuable
thing a software team can have. Over the next few years it gets more expensive, not cheaper. And
from December on, it is also, legally, what makes the difference between a product that holds and a product that's liable.
That is not a threat to the tester's profession. It's the first time in a long while that the profession is
being offered the place it always deserved. Provided we take it.

Agile Testing Days 2026[9] takes place 16 to 19 November in Potsdam. Three weeks before software becomes a product in the EU. If this year you decided a gate that demanded proof, or watched one
that was missing, and you can tell us what happened in production afterwards: that is the story that belongs on our stage. Not which tool you used. The Call for Papers[10] is open.

Sources and links

[1] Google Cloud / DORA: Accelerate State of DevOps Report 2024.
https://dora.dev/research/2024/dora-report/
[2] Pariseau, B.: Google DORA: Software delivery caught up to AI coding tools. TechTarget, 23
Sept 2025 (covering the 2025 DORA State of AI-assisted Software Development Report).
https://www.techtarget.com/searchsoftwarequality/news/366631712/Google-DORA-
Software-delivery-caught-up-to-AI-coding-tools
[3] METR: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer
Productivity. Randomized controlled trial, July 2025. https://metr.org/blog/2025-07-10-early-
2025-ai-experienced-os-dev-study/
[4] Stack Overflow: 2025 Developer Survey. 49,000+ responses, 177 countries.
https://survey.stackoverflow.co/2025/
[5] Veracode: 2025 GenAI Code Security Report. Press release, 30 July 2025.
https://www.businesswire.com/news/home/20250730694951/en/AI-Generated-Code-Poses-
Major-Security-Risks-in-Nearly-Half-of-All-Development-Tasks-Veracode-Research-Reveals
[6] Directive (EU) 2024/2853 of the European Parliament and of the Council of 23 October 2024
on liability for defective products. EUR-Lex. https://eur-lex.europa.eu/eli/dir/2024/2853/oj