Human oversight (AI Act)
Human oversight is the requirement in Art. 14 of the EU AI Act that high-risk AI systems be designed and developed, including with appropriate human-machine interface tools, so that they can be effectively overseen by natural persons during the period in which they are in use, with the aim of preventing or minimising risks to health, safety or fundamental rights. Providers must build in oversight measures, either integrated into the system or identified for implementation by the deployer, that enable the persons assigned to oversight to understand the system's capacities and limitations, remain aware of automation bias (the tendency to over-rely on outputs), correctly interpret outputs, decide not to use the system or to disregard, override or reverse an output, and intervene or interrupt the system through a "stop" button or similar procedure. For remote biometric identification systems, verification by at least two competent persons is required.
Deployers, under Art. 26, must assign human oversight to natural persons who have the necessary competence, training, authority and support, ensure that input data is relevant and sufficiently representative, monitor the system's operation according to the instructions for use, and inform the provider and authorities of risks and serious incidents. Art. 4 additionally requires providers and deployers to ensure a sufficient level of AI literacy among their staff. The Commission and standardisation bodies (CEN-CENELEC JTC 21) are developing harmonised standards that specify oversight measures in more detail.
Human oversight in the AI Act complements, but is broader than, the protections against solely automated decisions in Art. 22 GDPR: GDPR safeguards apply to decisions with legal or similarly significant effects on individuals, whereas AI Act oversight applies to the operation of the system as a whole, whether or not individual decisions are involved. In healthcare, effective oversight means that a radiologist reviewing AI-flagged images, a clinician using a decision-support tool or a pharmacovigilance scientist validating AI-coded cases must have the time, training and interface design to disagree with the system, and that these arrangements are documented; a nominal "human in the loop" who rubber-stamps outputs satisfies neither the AI Act nor the GDPR. iliomad's AI Officer service designs and monitors these governance arrangements.
