ISO 42001 Clause 9.1: Measure What AI Governance Needs to Know

What clause 9.1 requires in practice

Clause 9.1 asks the organisation to design its own performance evidence. It must decide what is to be monitored and measured, which methods are used for monitoring, measurement, analysis and evaluation so the results can be trusted, the timing of monitoring and measuring, and the timing of analysis and evaluation of the results. Documented information must be available as evidence of results. The clause ends with a broader duty: the organisation evaluates both the performance of the AI management system and its effectiveness.

Two points deserve attention. The clause leaves choice of measures to the organisation, so measures should follow its objectives, risks and controls. And it distinguishes collecting data from analysing and evaluating it: counting something is not the same as judging what the count means. The outputs are a defined monitoring approach, retained results and documented evaluation of how well the system performs.

Start with the question management needs to answer

As an original design method, write the decision before choosing the metric. “Do reviewers detect consequential errors?” is a different question from “How many reviews occurred?” Both can matter, but the second cannot answer the first alone.

Performance information may concern the AI system, its operating process or the management arrangements. A model can perform well in a test while change approval or concern reporting works poorly. Select information that addresses the actual objectives and controls rather than only what a supplier dashboard makes convenient.

Why clause 9.1 matters to the business

Leaders can only manage what they can see, and what they can see depends on what was chosen to be measured. If the chosen measures are convenient rather than relevant, the organisation may report steady improvement while a serious weakness grows unseen. Good evaluation helps the business spot that a supplier's headline results do not represent its own use, or that quick processing is eroding service quality. It also strengthens budget decisions, because resources can be directed to the controls that demonstrably need them. For audits and customer reviews, clear, valid and retained results are persuasive in a way that general assertions are not.

How to implement clause 9.1

Work backwards from decisions to measures. The sequence below is one that Sterling teams apply, and the standard leaves selection of measures to the organisation. Keep each measure tied to a real management question, and drop any figure that nobody uses to decide anything.

1. Write the questions management and process owners need answered, such as whether reviewers detect important errors.
2. Choose a measure or observation for each question, favouring results over activity counts.
3. Specify the population, source, sampling approach and definitions, so that results mean the same thing each time.
4. Set when each item is collected and when it is analysed, tying analysis to dates when decisions are due.
5. Add a short note with every report on limits, for example which languages, products or customer groups were not covered.
6. Record any change of definition or method so that trends are not misread as real improvement or decline.
7. Assign an owner for each adverse result who can act, and retain results as evidence.

Illustrative software example: a high average hides an important gap

Illustrative construction example: quick summaries, unclear decisions

A construction consultancy measures time saved by AI-generated progress summaries. Project managers later find that some summaries omit important qualifications about access restrictions. Speed improves while the quality of coordination decisions may worsen.

The organisation adds suitable evaluation of the information relied on by decision-makers. It examines representative cases and the effect of review arrangements. The chosen method should reveal meaningful failure, not simply confirm that employees are using the tool more frequently.

Make method and timing credible

Determine how evidence is collected and evaluated so that results are valid for the intended question. Consider definitions, sampling, sources and the effect of missing information. A small sample may support a limited conclusion but not a statement about every use in the organisation.

Timing also matters. Some information is useful only if reviewed before a relevant decision. Other trends need enough observations to be interpreted responsibly. The standard does not prescribe one monitoring frequency or universal sample size; choose arrangements that fit the subject and explain their limitations.

Analyse results rather than merely collect them

Separate data collection from interpretation. A rise in reported concerns may indicate worsening performance, improved reporting or both. A low complaint count does not automatically show that no one is adversely affected. Consider context and other evidence before deciding what the trend means.

When methods change, preserve the information needed to understand comparisons. A new definition of an error can alter a rate without any change in underlying performance. Presenting the figures as a continuous improvement trend would be misleading unless the change is explained.

Evidence and links to action

Common nonconformities with clause 9.1

Findings under clause 9.1 commonly relate to how measures were chosen and read.

• Monitoring covers system output quality only, and nothing evaluates whether processes, roles and controls of the management system are effective.
• Measures cannot be traced to a method, so an auditor cannot tell how a reported figure was produced.
• A definition or sample changed between periods and the report presents the resulting trend as genuine improvement.
• Supplier reports are accepted as evidence about the organisation's own use without checking their scope.
• Adverse results are recorded with no owner or follow-up, so evaluation never reaches a decision.