Four decisions behind our coverage check

What our system does is on the product page: capture documents, determine the applicable policy version, check five dimensions, output a recommendation, the claims handler decides. Why it is built that way is the more useful question. Behind each step sits a decision, a rejected alternative, and in one case an assumption of ours that turned out to be wrong.

The version question can block everything

When we started, we assumed that reading and interpreting clauses would be the hard part. That assumption was wrong. The step that determines the quality of the whole check comes before it: which version of the policy conditions governs this contract? In portfolios grown over years, several generations run in parallel, and the difference between two of them can concern exactly the clause the case turns on.

So version identification is a check in its own right, with its own result, and it can block everything downstream. The rejected alternative was to check first and verify the version afterwards. That would be faster, it would demo better, and it moves the biggest risk to the point of least attention.

The useful question for software of this kind is therefore not how well it reads clauses. It is what happens when the version assignment is ambiguous.

"Unsure" is a result

The obvious build is to always output a best guess with a confidence score next to it. Almost all systems work that way, and that is what makes them risky in daily use. A system that is usually right and always answers earns trust over time, and that trust then covers the cases where it was guessing. Human factors research calls this automation complacency, and every team lead knows it from practice under a different name.

We went the other way. When a version assignment or a dimension cannot be answered cleanly, the system outputs no estimate. It marks the spot and hands it to the handler. This costs automation rate and impresses nobody in a sales meeting. It remains the most important property of the system, because a recommendation is only worth something if it can be trusted at the moment it appears.

Five answers instead of one verdict

An overall result, covered or not covered, would be the simplest interface. Instead the system gives a separate reasoned answer per dimension, each with its clause reference: personal, subject matter, temporal, territorial, exclusions. Not out of thoroughness. An overall verdict forces the handler into all or nothing, adopt it or re-check everything. Five separate answers let her agree with four dimensions, disagree with one, and keep the preparatory work for the rest.

We built the system for the case where the handler disagrees. Software you can only follow completely or not at all is, in daily use, either a dictate or useless.

What does not exist cannot go wrong

Many vendors write that the handler decides in the end. The difference is how that is secured. With us it is not written into a policy, it is missing from the code: the software contains no action for "deny the case", "confirm the benefit" or "close the file". These actions do not exist and can therefore not be triggered by a bug or by a later configuration change. A policy can be changed under pressure. A missing function cannot.

In parallel, the system records at every step what was present, what was checked, where uncertainty was flagged, and what the handler did with it. That audit trail gets a post of its own.

If you are evaluating systems of this kind

Do not ask about average accuracy. Ask what happens with an ambiguous policy version, whether a single dimension can be overruled on its own, and which actions the software can technically perform at all. Those three answers say more than a demo.

How does your team handle cases today where the applicable policy version is not obvious at first glance?