AI glossary Strategy & Business
What are acceptance criteria, and how do they apply to AI systems?
← All glossary termsTestable conditions that define when a feature or a model's output is complete and correct, so that 'done' can be checked rather than argued.
Explained
Acceptance criteria are the specific, observable conditions a piece of work must meet to be accepted. In software they are usually attached to a user story and written so that anyone can check them, often in a Given-When-Then form. For AI systems they translate requirements into measurable checks: the output follows a schema, answers match a set of reference ('golden') answers, sources are cited, safety rules are respected and quality stays above an agreed threshold. They underpin evaluation harnesses, regression suites and human-in-the-loop review, linking the specification to measurable quality.
How acceptance criteria are written
Good criteria are specific, testable and agreed before the work starts. A common template is Given-When-Then:
- Given a context or starting state.
- When an action or event happens.
- Then a specific, observable result is expected.
Example: given a customer with an open claim, when they ask the assistant for its status, then the answer gives the current status from the claims system and the date of the last update, and does not invent a date if none exists.
Acceptance criteria for AI systems
Because AI outputs vary, criteria are often expressed as rates or thresholds over a test set rather than a single pass or fail:
- Format: every response is valid against the required schema.
- Correctness: answers on a reference set match the expected answer or are judged correct by a reviewer or a model grader.
- Groundedness: claims are supported by the retrieved sources, and sources are cited.
- Safety and policy: no personal data is disclosed, and out-of-scope requests are declined.
- Operational: response time and cost per request stay within agreed limits.
Each criterion should say how it is measured: automated check, model-based grading or human review.
From criteria to quality gates
Once written, acceptance criteria become the evaluation suite that every change to the prompt, the model, the retrieval or the tools must pass. This turns quality into a gate in the release process instead of a judgement made after users complain, and it makes switching models or vendors a measurable decision.
Frequently asked questions
What is the difference between acceptance criteria and the definition of done?
Acceptance criteria are specific to one feature or story and describe what it must do. The definition of done applies to all work and describes the quality standard every item must meet, such as being tested, reviewed and documented.
What is the Given-When-Then format?
It is a template for writing acceptance tests: Given sets the context, When describes the action, and Then states the expected observable result. It comes from behaviour-driven development.
How do you write acceptance criteria for an AI model?
Define the task, collect a representative set of test inputs, and state for each criterion what counts as a pass and how it is measured. Use thresholds over the set, because a generative model's outputs vary between runs.
Who writes acceptance criteria?
Usually the product owner or business owner, together with the development and testing team. For AI systems it helps to involve domain experts, who know what a correct answer looks like.
Decision guides that use this term
Where it shows up at Thinkia
Further reading
Related terms
Sources
- Agile Alliance · Given-When-Then
- Anthropic · Define success criteria and build evaluations
- Scrum Alliance · Everything you need to know about acceptance criteria
Updated: 2026-10-21