ISO 42001 Annex A.7.4: Define Data Quality for the AI Use

What control A.7.4 requires in practice

Control A.7.4 has two halves. First, the organisation defines and writes down what quality means for the data its AI system needs. Second, it makes sure that the data used to develop and operate the system actually meets those requirements. Quality is therefore tied to the job, not to a generic scorecard. Annex B guidance explains why: the quality of training, validation, test and live data affects whether outputs can be trusted, so each should be defined, measured and improved as far as practicable, and be suitable for the purpose. It also asks the organisation to consider how bias in data affects performance and fairness and to adjust the model or data where that is needed for the use. The outputs are documented quality requirements, evidence that checks were done against them and decisions about data that falls short.

Define fitness for the intended task

Annex B.7.4 connects data quality with output validity and the suitability of training, validation, test and production data. It considers relevant bias and its effect on performance and fairness. The guidance must be considered through clause 6.1.3(e), with tailored implementation allowed under B.1.

Start with what the system needs to distinguish or infer. Identify which data characteristics affect that task and what evidence will show suitability. A generic statement that data is complete or clean may omit the most important requirement.

Why control A.7.4 matters to the business

Teams often discover that a model scored well on tidy test data and then struggled with real inputs. Defining quality for the intended use is the discipline that prevents this. For the business it protects customer outcomes, avoids rework and makes performance claims defensible. It also gives product, data and risk teams a common vocabulary, which reduces arguments about whether a dataset is good enough. Auditors look for criteria that someone can name, results that can be inspected and a visible response when data failed to qualify. A stamp reading approved, with nothing behind it, is the weakness they most often find here.

How to implement control A.7.4

Quality has to be defined before it can be demonstrated, so begin with the use. Involve people who understand the business setting, not only data specialists, because the requirements that matter most are usually about what the real operating situation looks like and what a wrong output would cost.

1. Describe what the system must infer or distinguish and which data characteristics affect that task.
2. Write quality requirements for each data role: training, validation, testing and live operation.
3. Include representativeness and relevant bias in the requirements, with the evidence you will accept.
4. Agree how each requirement is measured and what happens when a result misses the mark.
5. Run the checks on the actual datasets and keep the results with the decision to proceed, restrict or replace the data.
6. Monitor live inputs and trigger a re-check when sources, users or equipment change.
7. Never loosen a requirement just because current data fails it; record a reasoned basis for any revision.

Illustrative manufacturing example: the images are complete but inadequate

Illustrative software example: operating language differs from evaluation

A software assistant is evaluated using standard technical language. Customers in operation use abbreviations, mixed terminology and incomplete descriptions. The original data is internally consistent, but does not adequately represent important operating conditions.

The organisation examines the effect on performance and affected users. It considers suitable changes to data, evaluation or use and documents the basis. A high result on the original dataset does not demonstrate suitability for the new population.

Consider different data roles

Data used for development and data used for evaluation have different purposes. Operating data introduces continuing conditions that may differ from both. Define relevant quality expectations rather than treat all datasets as interchangeable.

Link quality requirements with acquisition and selection and preparation. A transformation can improve one characteristic while removing information needed for another. The quality decision should remain connected to the task.

Examine bias and consequential differences

The guidance calls attention to the effect of bias on performance and fairness. Examine relevant differences using appropriate evidence and expertise. Do not assume that a numerical imbalance alone proves every form of unfairness, or that balancing a dataset automatically resolves the issue.

Consider how data was collected, what it excludes and how outputs are used. A quality assessment may reveal limitations requiring changes to the model, data or operating scope. The response should address the actual cause and consequence rather than a fashionable metric.

Evidence and response to unmet requirements

Common mistakes