Three ways vendors build human in the loop

No sentence comes up more often in conversations about claims software than "the human stays in the loop". Behind the same phrase sit three very different ways of building a system, and the difference between them determines what a data protection officer, an ombudsman or a works council can do with it. This post describes the three variants, what each one costs, and how to tell in a demo which one is in front of you.

Variant 1: Human by policy

The system can technically do everything: check, decide, close the case in the core system. A rule states that a claims handler approves the result first. The loop exists, but it exists as a rule, not as a construction.

This is the most common variant because it is the easiest to build. Its problem shows up not at introduction but later, in daily operation. Over time the rule comes under pressure: a backlog grows, a team shrinks, someone asks whether approval could be switched off for "simple" cases. Since the system can perform the action anyway, the step from "human checks everything" to "human checks samples" is a configuration change. And the approval itself wears down: someone who has confirmed a hundred correct recommendations reads the hundred-and-first differently. Research calls this automation complacency; in practice it is called waving things through.

How to tell: ask whether the system can be configured to close cases without approval. If the answer is "yes, but we do not do that", it is variant 1.

Variant 2: Human by threshold

The system computes a confidence for every recommendation. Above a threshold the case goes through, below it goes to a person. That sounds like the sensible middle way, and in many areas it is: for high-volume processes with low potential harm per case, it is often the right construction.

In coverage checking it has two catches. First, the threshold is a dial, and dials get turned. As trust grows it drifts downwards, the share of human-reviewed cases falls, without that shift ever being consciously decided. Second, confidence does not mean correctness. A model can be very sure about a case it misunderstands; those cases sit above the threshold, and those are precisely the cases no person sees. The threshold does not route the risky cases to a human; it routes the cases where the model itself notices it is unsure. That is not the same thing.

How to tell: ask what share of cases a person sees, who sets the threshold, and who may change it. If the answer contains a percentage and a settings dialog, it is variant 2.

Variant 3: Human by architecture

The system produces a recommendation with reasoning. That is all it can do. There is no function that automatically closes, denies or confirms a case. These actions do not exist in the software.

Every case is judged by a person; the relief does not come from fewer cases per person, but from less preparatory work per case: the version is determined, the clauses are ready, the five dimensions are individually reasoned before the handler starts.

In return, the answer to the GDPR Art. 22 question is not "we have a policy" but "it is technically not possible otherwise". In the conversation with the data protection officer, that is a different category of answer.

How to tell: ask which write actions the system can perform at all. If the list is empty, it is variant 3.

Why the label is not enough

All three vendors truthfully say "human in the loop". The difference lies in what keeps the human in the loop: a rule, a threshold value, or the absence of an alternative. Rules change under pressure, thresholds drift with trust, missing functions stay missing. For a decision with legal effect towards insured persons, that is not a detail but the actual safety question.

To be fair: variant 3 is not right everywhere. Where cases are numerous, uniform and low-consequence, it is over-cautious. In coverage checking, where a wrong denial leaves an insured person without legal protection and a wrong approval triggers costs, we consider it the only one that can be defended in front of everyone involved.

Where we stand

Our system is built as variant 3. There is no action for closing, denying, or confirming. The recommendation comes per dimension with a clause reference, so the handler can adopt four dimensions and disagree with one without losing the preparatory work. If a dimension or the policy version is not unambiguous, it is flagged and handed over.

Which variant is running in your organisation today, and who decided that?