ISO 42001 Treatment Effectiveness: Check the Risk Response
ISO 42001 clause 8.3 requires implementation of the AI risk treatment plan and verification of its effectiveness. When new risks are identified, treatment options are assessed through clause 6.1.3. Where treatment is ineffective, it is reviewed and revalidated and the plan updated. All treatment results are retained. The central question is whether the response works for the risk it was intended to address.
What clause 8.3 requires in practice
Clause 8.3 closes the loop on risk treatment. Having planned how each assessed AI risk will be addressed under clause 6.1.3, the organisation must carry the plan out and verify its effectiveness. Verification asks whether the chosen treatment actually reduced the risk as intended, which is a different question from whether the planned action took place.
Two further obligations follow. When assessments reveal new risks that need treatment, the organisation runs the treatment process for them. When a treatment option turns out not to work, the organisation reviews and revalidates the option through the treatment process and updates the plan. Throughout, it keeps documented information on the outcomes of every AI risk treatment. The outputs are an implemented plan, evidence of effectiveness, an updated plan when needed and a retained history of treatment results.
Why clause 8.3 matters to the business
A control that exists on paper but does not work is worse than no control, because it creates false comfort. Managers may approve wider use, customers may be assured, and the gap surfaces only after harm or complaint. Verifying effectiveness is how the business learns early that a review step misses a type of error or that a restriction is routinely bypassed. It also guards against paying repeatedly for the same ineffective fix. For clients and certification audits, evidence of verification separates organisations that merely list controls from those that can demonstrate results. Keeping all treatment results, including unfavourable ones, supports accurate management decisions.
How to implement clause 8.3
Define what success looks like before choosing how to check it. Sterling teams commonly work through the following sequence, adapting methods to the control concerned.
- State, for each treatment, the intended result in terms of what should or should not happen, not the task to be done.
- Choose a verification method that could reveal failure, including difficult cases and unusual conditions, and explain how cases were selected.
- Check representativeness across shifts, sites, languages or customer groups, so one favourable setting does not stand for all.
- Record the evidence with its limits and uncertainty rather than rounding it into an unqualified assurance.
- Set a trigger for review when verification shows a gap, and decide who may restrict the use while a fix is prepared.
- Update the treatment plan and communicate any changed operating conditions to the people who apply them.
- Keep results of every verification and every plan revision, so the reasoning can be reconstructed later.
Separate implementation evidence from effectiveness evidence
Implementation evidence shows that the planned action happened. Effectiveness evidence addresses whether it achieved the intended result. Installing a review step, issuing an instruction or changing a configuration can demonstrate activity without establishing the effect on risk.
Define the intended result before selecting the verification method. If the treatment is meant to prevent unsupported statements reaching customers, a record that employees received training is not enough by itself. The organisation needs evidence related to what reaches customers and how the control affects that outcome, taking account of limitations and other relevant conditions.
Example: software-generated technical instructions
A hypothetical software business uses AI to draft installation instructions. Assessment identifies a risk that plausible but incorrect steps are published. Treatment requires review by a competent technical colleague before release. The company confirms that all drafts have a reviewer recorded, but that only demonstrates completion of the review process.
Verification could examine whether the review detects relevant errors, whether reviewers have access to the appropriate product information and whether approved instructions work under representative conditions. The organisation should choose methods appropriate to the use; the example does not prescribe a universal test set.
If reviewers repeatedly miss a particular class of error, management should reconsider the treatment. More signatures on the same inadequate review may not change the result. The cause could involve missing information, competence, workload or the limits of the review method.
Example: manufacturing inspection across shifts
A manufacturer uses AI to identify visual defects and routes uncertain cases to an inspector. A daytime evaluation suggests the treatment works. At night, however, lighting conditions and staffing differ. The same control may perform differently even though its formal description is unchanged.
Verification should consider conditions relevant to the actual operating scope. A result obtained from one product and shift may not justify a conclusion about all products and shifts. The organisation can examine whether the evidence is representative and identify where further checks or different operating conditions are needed.
This is management-system guidance about the treatment decision and its evidence. It does not establish that a product meets its technical specifications or replace specialised quality or safety evaluation.
Use evidence that can reveal failure
As a suggested approach, define what would count as evidence that the treatment is inadequate. A verification activity designed only to confirm expected success can miss the very condition the control was intended to manage. Include relevant difficult cases and explain how they were selected.
Consider the denominator and the significance of different outcomes. A high overall success rate can conceal a consequential failure in a small group or unusual condition. Conversely, a small sample may not support a broad conclusion. Record uncertainty rather than translate limited evidence into an absolute assurance claim.
Clause 9.1 supports this work by requiring the organisation to determine monitoring and measurement needs, methods for valid results, timing and analysis, with evidence of results. Operational verification and system evaluation connect, but a management dashboard should not replace the detail needed to understand a particular treatment.
Revisit the plan when evidence changes
Where treatment is ineffective, clause 8.3 requires review, revalidation and an updated treatment plan. Newly identified risks return to the treatment process. This is a reasoned decision cycle, not an instruction to keep repeating the same test until a favourable result appears.
The response might change the control, restrict a use, obtain better evidence or take another suitable treatment option. Relevant approvals and residual-risk acceptance should remain consistent with the process under clause 6.1.3. Communicate changed operating conditions to the people who need to apply them.
Evidence and common mistakes
An auditor may follow a treatment from the assessed risk to the plan, implementation, verification and response to results. Retained evidence should describe the conditions and limitations well enough to make the conclusion understandable. Keep all treatment results rather than preserve only a final successful summary.
Common weaknesses include treating purchase or installation as closure, measuring an easy activity instead of the intended result and generalising supplier test results to a different use. For global operations, preserve relevant local differences in verification instead of assuming that one site's result applies everywhere.
Frequently asked questions
Must verification always be a technical test?
No single method is prescribed. The method should provide credible evidence about the intended treatment result, which may involve technical, procedural or human arrangements.
Can we accept uncertainty?
Decisions should follow the organisation's criteria, applicable requirements and authorised acceptance process. Uncertainty must be understood rather than hidden in an unqualified success statement.
Does one successful check prove the treatment will always work?
No. Conclusions are limited by the conditions and evidence examined. Ongoing monitoring and reassessment address changes and later experience.
Check whether your AI controls address the risk
Discuss AI treatment verification with Sterling, including scoped evidence support and agreed Effivity ISO & GRC automation software arrangements. Explore AI Risk & Impact Assessment to strengthen the assessment-to-treatment connection. Certification decisions remain independent and do not certify every output.
Parent: clause 8. Previous: operational controls. Next: performance evaluation. Return to the hub.