Skip to main content

Docs · Trust

Model trust: how to evaluate AI output before using it

Trust in an AI result depends on the task, the evidence and the consequence of being wrong. A benchmark score or fluent explanation cannot establish that a particular requirement has been read correctly.

By Nebula Platform Pty Ltd. Sources checked 15/10/2026. This guide is an evaluation checklist, not a certification, independent assessment or claim that Nebula or a model has passed an evaluation.

What does model trust mean in delivery work?

For a document-reading workflow, model trust means having enough evidence to decide whether a result is fit for its intended use. That decision is narrower than declaring a model trustworthy for every task. It depends on the source material, the type of finding, the review process and the downstream action.

Three questions need separate answers: is the result supported by the source; is the source current and applicable; and does anyone have authority to act on the result? An accurate reading can still be stale. An accurate, current reading can still require an authorised decision.

The NIST AI Risk Management Framework is intended for voluntary use. NIST says AI RMF 1.0 is being revised. It can help structure a review, but it does not replace evidence from the intended workflow or establish that a model is fit for a particular decision.

Build an evaluation around decisions, not impressive answers

Start with a small, versioned collection of representative documents that the organisation is authorised to use. Record expected findings before running the model. Keep development examples separate from the final evaluation set. Use questions such as:

  • Source support. Can each important finding be checked against the relevant passage?
  • Coverage. Which required obligations were missed, not merely which returned answers were correct?
  • Conditions. Did the answer preserve exceptions, dependencies, units and notice periods?
  • Conflict. Did it identify incompatible documents without silently choosing a winner?
  • Uncertainty. Did it preserve an unresolved question instead of inventing a term?
  • Freshness. Was the result produced from the version that governs the work?
  • Boundary. Did hostile instructions inside the source remain data rather than become commands?

Classify errors by consequence. Missing a heading and inventing a payment condition are not equivalent failures. Report the document count, selection method, model version, configuration, review method and unresolved errors alongside any score. Do not present a small selected sample as an industry-wide performance claim.

Worked example: preserve the condition on a deadline

Suppose an illustrative agreement requires a response within ten business days after receipt of a valid notice. Returning "ten days" loses both the calendar basis and the triggering condition. A useful finding retains the original wording and source reference, and flags any missing definition of a valid notice or business day.

The reviewer should be able to mark that finding incomplete without changing the source. A later correction should preserve what was originally returned and who corrected it. This example is an evaluation case, not evidence that a particular model passed it.

Re-test when the system changes

A provider or model update is one reason to repeat an evaluation. Changes to document extraction, prompts, retrieval, tools or review rules can also change the outcome. Keep earlier results so the team can identify regressions, not just compare an aggregate score.

Before using output for a material action, evaluate the action boundary as well. Model quality and permission to act are different controls. The agent governance guide covers approvals, tool access and recovery.

Inspect the reading and its limits

Nebula's public reading is a prepared example that can be inspected without supplying a private document. The Nebula on Nebula record provides published context on how Nebula uses its own record. Neither page is a guarantee of accuracy on a new document collection.

Proof you can check explains another distinction: checking that a record is unchanged does not establish that its business statement is true or authorised. For a procurement review, continue to the AI evidence checklist.