"Unclear" Is Not One Answer: Five Decision States for an AI Inspection

This is for anyone designing the output of an AI-assisted check where "I don't know" is a legitimate answer. RPOMS AI looks at a photo of a router casing and answers ACCEPTABLE, NOT ACCEPTABLE or UNCLEAR. Most answers are UNCLEAR, and by design. The weak open model is only allowed to decide what it has proved it can decide, as explained in A Vision Model That Only Observes.

UNCLEAR on its own turned out to be a poor answer, because it was covering four situations that need different responses. The photo might be unusable, which needs a retake. The model might be unreachable, which is our problem and not the user's. The model might have answered but not be trusted on that question. Or the model might have seen something clearly that the written criteria never cover. The last two look identical on screen and mean opposite things about what to fix.

Since 22 September 2026, every decision carries one of five states:

The last two must not be run together. AI_UNCERTAIN is a weakness of the model, and it improves when the model improves. ERTH_RULE_INSUFFICIENT is a gap in the source. The acceptance criteria describe their categories in words, not numbers, so a mark that falls between a documented "small" and a documented "long" has no answer. No model can close that gap. Recording the two separately tells you whether to spend effort on the model or on a conversation with the client.

The decision engine already produced a reason code for every path: quality_failed, surface_partial, analysis_unavailable, model_not_qualified, undocumented_case and so on. The state is derived from the reason in exactly one table. It is a Record, not a switch with a default:

export const DECISION_STATE: Record<ReasonCode, DecisionState> = {
  rule_accepted: "DECISION_READY",
  rule_rejected: "DECISION_READY",
  no_defect: "DECISION_READY",
  quality_failed: "PHOTO_QUALITY_FAILED",
  surface_not_visible: "PHOTO_QUALITY_FAILED",
  surface_partial: "PHOTO_QUALITY_FAILED",
  analysis_unavailable: "AI_UNAVAILABLE",
  analysis_failed: "AI_UNAVAILABLE",
  no_qualified_model: "AI_UNAVAILABLE",
  low_confidence: "AI_UNCERTAIN",
  model_not_qualified: "AI_UNCERTAIN",
  undocumented_case: "ERTH_RULE_INSUFFICIENT",
};

Two details make this robust. First, decide() is the only exit from the rules, and it attaches the state from this table on every path, so the stored state and the stored reason cannot disagree. Second, decisions saved before the state field existed still have a reason code, so reading them back derives their state from the same table. No data migration was needed.

A member of the public does not need five states. They need to know what to do next. A second table collapses the five into four public states, and the page styles those:

The distinction that must survive into the public view is the outage. Before 20 September, three situations arrived as UNCLEAR in the same amber box. That meant a service outage looked like a bad photo, and someone would be told to retake a photo that was fine. An outage now says whose problem it is.

One behaviour change came with this. With no active model, the public used to see "needs a human check". That wording implied the automatic check had looked and failed to decide, when it had not looked at all. It now shows AI CHECK UNAVAILABLE, the message for a problem on our side where the photo is fine.

This is not hypothetical. RPOMS AI has no model activated in production, so every live photo that passes the quality gate currently takes exactly this path. It has also never been used on real line photos.

On the thirteen reference photos, under the qualification gate, eleven answers were UNCLEAR. The states split them: eight AI_UNCERTAIN, two ERTH_RULE_INSUFFICIENT and one PHOTO_QUALITY_FAILED, where the model judged the surface only partly visible. Without states, those were just "eleven unclears", and the natural reaction is to loosen something. With states, the eight point at the model and the two at the written criteria. Neither points at the rules engine. The same principle of letting a system say it cannot answer, and why, runs through Making an LLM Admit the Paper Doesn't Say That.