Evaluate success regularly and adjust our approach as we go
Read the full detail of action 2.7 on GOV.UK. (opens in a new tab)
Year 1 progress
In Year 1, we developed an AI Lifecycle, designed for departmental use, to support a more consistent approach to developing, evaluating and improving AI products. It helps teams define what success looks like from the outset, build evaluation into the development and use of AI, and use evidence to inform decisions about whether products should change, scale or stop.
Alongside this, the MOJ now requires all AI tools to be rigorously tested for bias, accuracy, reliability, fairness and security before they can be scaled and deployed more widely, with ongoing monitoring as they mature. We have also brought in dedicated evaluation resource for Justice Transcribe, helping us understand how it performs in practice, the experience of users and whether it is delivering the intended benefits.
Year 2 priorities
In Year 2, our focus will be on embedding the AI Lifecycle more consistently across the department and developing clearer standards and approaches for evaluating AI products over time. This will include more consistent measurement of user experience, performance and impact, alongside continued monitoring of bias, accuracy, fairness and reliability.
We will also develop a central service for measuring user experience consistently across digital services, helping teams gather comparable evidence and use it to improve, scale or stop AI interventions.
