Article
Testing AI systems is not classic QA — and this is where many organisations fall
A system that answers differently on every run needs a different testing approach than deterministic software.
Classic QA rests on a simple contract: the same input produces the same output, so a test case can assert equality and a green build means the behaviour is unchanged.
AI systems break that contract. The same input can produce a different — and still correct — answer. Teams that carry their old test suite into an AI product usually discover this in production.
What replaces the test case
Instead of asserting one exact answer, you assert properties of the answer across a curated evaluation set: is it grounded in the source, is it complete, is the format valid, does it refuse when it should.
The unit of quality moves from a single test to a score across a dataset, tracked over time like a performance metric.
What an AI test practice actually needs
- A golden dataset drawn from real cases, including the awkward and rare ones.
- Automated evaluators for grounding, completeness, format and policy compliance.
- Regression runs on every prompt, model or retrieval change — not only on code changes.
- Adversarial testing: prompt injection, data leakage, out-of-scope requests.
- Human review of a sample, with the reviewers' verdicts fed back into the dataset.
- Production monitoring, because the real distribution of inputs will always surprise you.
Why it matters commercially
Without this practice, no one can say whether a model upgrade improved or degraded the product, and every change becomes a leap of faith. Risk and compliance functions rightly refuse to sign off on that.
With it, model changes turn into measurable decisions, and going to production stops being an argument.
Non-determinism is not a defect to be eliminated. It is a property to be measured — and measurement is what makes AI shippable in an enterprise.
This article is the English version of a post originally published in Hebrew on our LinkedIn page.