Says done, but nothing happened

This block explains why an agent's "done" proves nothing, how a system finds out what really happened, and how to measure the difference.

The Northwind agent tells a customer: "Your refund of $38.00 for order 4417 has been processed." Five days later the customer writes that no money arrived. The payment records hold no refund for that order, and the run is logged as a success.

The colleague and the statement

Back to the colleague on the phone. They tell you: "All paid." You now have three witnesses to that payment. The colleague's word. The bank clerk who said "OK" on the call. And the line on the bank statement.

Only the statement is the fact. The colleague may be describing what they meant to do. The clerk may have said "OK" to a transfer that never went through.

The final message of a run is the colleague's word. False success is the name for a claim of completion that the actual state does not support.

Three things usually go wrong, one on each hop:

The tool reports something false. A service answers 200 although nothing was written, or the write was partial. After an integration change, every call can come back that way.

The code around the model hides the error. This wrapper turns every failure into good news:

try:
    payments.refund(order_id, amount, idempotency_key=key)
except Exception:
    log.warning("refund failed")
return "Refund submitted."

The model retells its intention. It asked for the refund, or only planned to, so it writes that the refund happened, even if the result said pending or an error. No amount of confidence in that sentence tells you which one occurred.

How often? One 2026 workshop paper, from a single author, studied 9,876 runs from 8 model families on the tau2-bench customer-service benchmark. Among failed runs, the share where the agent claimed completion anyway was this high:

Reading the statement

First, a write tool returns fields that code can use, instead of a sentence:

{
  "status": "succeeded",
  "refund_id": "re_31",
  "order_id": "4417",
  "amount_cents": 3800
}

These fields can still be wrong, as the 200 above showed. So after the write your code fetches the record and compares it with what it asked for. A check that runs outside the model and decides the outcome is a verifier.

def verify_refund(op):
    if op.refund_id is None:
        return "unknown"    # no confirmed reply, so the refund may still exist
    try:
        refund = payments.get_refund(op.refund_id)
    except (Timeout, ServiceError):
        return "unknown"
    if refund.amount != op.amount_cents or refund.metadata["order_id"] != op.order_id:
        return "not_landed"
    return {"succeeded": "landed", "pending": "pending", "requires_action": "pending"}.get(refund.status, "not_landed")

A failed check is a third answer. If the read times out, you do not know, and the previous block already covered what to do then.

The outcome then decides what the customer reads:

One read is one moment in time. At Stripe a refund's status can be pending, requires_action, succeeded, failed or canceled, and in rare cases a refund moves to failed because the customer's bank cannot process it. Stripe then sends a refund.failed event, so something on your side has to listen for it.

Verification has a price: an extra read and extra latency on each write. Pay it every time for money and permissions, and check a sample of the rest.

Counting the gap

Each run produces two separate facts: what the agent said and what the check found.

{
  "run_id": "run_9312",
  "order_id": "4417",
  "claimed": "done",
  "effect": "not_landed"
}

Ask the agent for a structured claim at the end of the run, so you do not have to parse sentences. Then count both on the same runs. Here is one week at Northwind:

Self-reported success alone would have looked healthy all week, and so would an uptime graph. Alert on the 170. A run still pending waits until it settles before it is counted.

Why not ask another model

A natural idea is an LLM judge that reads the transcript and decides. In the same paper, no judge configuration exceeded an AUROC of 0.65 on tau2-bench, where 0.5 is a coin flip and 1 is perfect. The author's explanation is that judges lean on confident closing language, which false-success runs have plenty of. The paper proposes lightweight trained detectors as a way to triage runs.

Anthropic's guide to agent evals draws the same line in general terms. The transcript is the record of the run, with what the agent said and did, and the outcome is the final state in the environment. Its example support eval uses a state check for the refund and an LLM rubric for tone. Keep a model for judging tone. Use code to read the record.

Conclusion

• The final message is a claim, and a 200 is only a stronger claim.
• Code reads the record after the write, compares it and sets the outcome. "Could not check" is its own result.
• Count claims and checked outcomes on the same runs, and alert on the difference.

Go deeper