Loop · Human-on-the-Loop
Designing effective monitoring
Human-on-the-Loop is the right model when an AI system operates within defined boundaries and humans intervene on exceptions. This guidance helps organisations check whether monitoring is a genuine control.
When Human-on-the-Loop fits
Human-on-the-Loop is appropriate when the AI system operates continuously or at scale within defined boundaries, and humans monitor performance and intervene when needed. The human is not reviewing every action: they are watching for conditions that require a response.
It works for alerting systems, compliance screening, risk monitoring, and exception-based workflows where normal operations are predictable and human intervention is reserved for anomalies.
Monitoring design checklist
Are baselines defined for normal system behaviour?
Good practice
Expected ranges for key metrics are documented and kept current. The system has a measurable definition of normal.
Failure mode
There is no documented baseline. Monitors rely on intuition or compare the system against itself rather than against a defined standard.
Signal: Without baselines, thresholds cannot be calibrated. Intervention becomes reactive rather than designed.
Are intervention thresholds explicit and tested?
Good practice
Specific conditions trigger human review. Thresholds have been tested against real scenarios and adjusted based on experience.
Failure mode
Thresholds are vague or have never been triggered. It is unclear what would actually cause a human to intervene.
Signal: If thresholds are never crossed, they may be too loose. If they are constantly crossed, they may create alert fatigue.
Is there a named person accountable for monitoring?
Good practice
A specific individual is responsible for monitoring. They have the authority to pause or escalate the system and know what to do in each scenario.
Failure mode
Monitoring is treated as a team responsibility with no named owner. When something happens, no one is clearly accountable.
Signal: If monitoring accountability is diffuse, the model is Human-on-the-Loop in name only.
Can a human intervene quickly enough to matter?
Good practice
Response time SLAs are defined and realistic. The system can be paused before irreversible actions are completed.
Failure mode
Humans are notified after actions are taken. Intervention is possible in principle but arrives too late to change outcomes.
Signal: If the system takes irreversible or externally facing actions faster than a human can intervene, monitoring is not a sufficient control.
Is accountability clear when no one intervenes?
Good practice
The monitoring owner is accountable for outcomes the system produces, including when no intervention occurred. This is accepted and documented.
Failure mode
No one is clearly accountable for what the system does between interventions. Accountability only surfaces when something goes wrong.
Signal: Human-on-the-Loop requires clear ownership of the system's behaviour, not just its exceptions.
Are thresholds and monitoring design reviewed periodically?
Good practice
Monitoring parameters are revisited on a schedule. Changes to the system or its environment trigger a review of whether existing thresholds remain appropriate.
Failure mode
Thresholds are set at launch and never revisited. The system evolves but monitoring does not.
Signal: A monitoring design that made sense at launch may be inadequate as the system handles new volumes, edge cases or changed business conditions.
When to escalate to HAL
Human-on-the-Loop stops being adequate when the system acts faster than humans can respond, or when accountability becomes unclear. These are the signals that Human-Accountable-for-the-Loop governance is required.
- ! The system takes irreversible actions before a human can intervene.
- ! No one is clearly accountable for outcomes between intervention events.
- ! The system creates external obligations, communications or legal consequences.
- ! Monitoring is nominal and alerts are routinely ignored.
- ! Volume or autonomy has increased beyond the original monitoring design.
- ! Failure would create legal, regulatory or significant client impact.