Reflexion

This block explains how an agent can use a failed attempt to do better on the next one, and when that is worth paying for.

Northwind's support agent needs track_order to work with a new carrier, so a background agent writes the parser that turns the carrier's status events into Northwind statuses. Northwind has 40 real events labelled by hand. The first attempt gets 37 right. A second run with the same prompt gets the same 3 wrong.

A note in the pocket

Picture a learner driver on a closed practice track. They fail a run at the second junction, and the examiner says only "fail". If they drive again with nothing changed, they will most likely fail in the same place. So before the next run they write themselves a note: "At the second junction, check the right lane first." The track is cleared, they start again from the gate, and the note goes with them.

Reflexion, from a 2023 paper, does this to an agent. It needs a track you can clear, an examiner and a pocket for notes.

Why the same prompt fails again

In the paper, baseline agents on HotPotQA (multi-step questions), retrying at temperature 0.7, solved none of the tasks they had failed on the first try. On AlfWorld (household tasks in a text world), plain retries did gain for a while, then stopped improving between trials 6 and 7.

A retry suits failures that clear by themselves, as in the block on stopping. A wrong approach does not clear by itself.

One round, step by step

1. The actor, your usual agent loop, runs the whole task in a fresh environment. Its chain of steps and observations is the trajectory.
2. The evaluator scores the trajectory. The paper used exact match on HotPotQA, self-written unit tests for code, and on AlfWorld either a language model or a hand-written rule (same action and same response for more than 3 cycles, or more than 30 actions). Several of these involve no model at all.
3. On a pass, your code stops.
4. On a fail, the self-reflection model gets the trajectory, the evaluator's signal and the earlier notes. It writes what went wrong and what to do differently.
5. Your code stores the note and keeps only the latest few. In the paper the limit was usually 1 to 3, because the notes have to fit in the context.
6. Your code resets the environment and the actor starts over with the notes in its prompt.

After the parser's first failure, Northwind's record looks like this:

{
  "trial": 1,
  "evaluator": {
    "passed": false,
    "failed_fixtures": [
      "evt_12",
      "evt_29",
      "evt_33"
    ],
    "example": {
      "event": "DELIVERY_ATTEMPT_FAILED",
      "expected": "in_transit",
      "got": "delivered"
    }
  },
  "note": "I treated any status containing DELIVER as delivered. Match full status codes. DELIVERY_ATTEMPT_FAILED means in_transit."
}

The whole loop is short:

def reflexion(task, max_trials=4, keep_notes=3):
    notes = []
    for trial in range(1, max_trials + 1):
        env = reset_environment(task)
        trajectory = run_actor(task, notes, env)      # a full agent run
        verdict = run_fixtures(env)                   # plain code, no model
        if verdict["passed"]:
            return {"status": "passed", "trials": trial, "result": trajectory}
        note = reflect(task, trajectory, verdict, notes)  # one model call
        notes = (notes + [note])[-keep_notes:]
    return {"status": "gave_up", "trials": max_trials}

In the paper, a note feeds the next attempt at the same task. "Match full status codes" helps this parser, and nothing here carries it to next week's job.

Three loops that look alike

Evaluator-optimizer, from the workflows block, critiques one draft and edits it. In the paper's AlfWorld runs, Reflexion throws the attempt away, resets the environment and carries over only the text. The paper's coding agent is an exception: its next attempt also sees its previous code and the test results, so for code the line between the two is thin. Northwind's loop above carries only the notes.

What it costs

Each round is a full run, an evaluator and, after a failure, one reflection call. Two failures and a pass cost three runs and two reflections, plus three evaluations if your evaluator calls a model. Each of these adds time, so the technique is a poor fit for low-latency answers.

The examiner sets the ceiling

HumanEval and MBPP are sets of short programming problems. The paper puts the MBPP loss down to false passes. When tests pass on wrong code, the loop stops and reports a wrong answer. Any evaluator that can approve bad work has this problem, including a second model grading something as loose as tone.

An ablation switches parts of a method off to see what each one adds. One on the 50 hardest HumanEval problems in Rust, with GPT-4, suggests that both halves matter. Pass@1 is the share of problems where the solution the system submits passes the benchmark's own tests:

The gap between 60% and 68% is 4 problems out of 50. The authors' reading of the 52% row is that with no tests the agent could not tell whether its code was right, so it kept going and made harmful edits.

Reflexion also fell short in two other places. On WebShop, a shopping-site benchmark that needs varied exploration, the authors stopped after four trials because the agent showed no improvement. On HumanEval Python with the small starchat-beta model, it scored 26%, the same as the baseline. These results come from 2023 models and benchmarks, so measure on your own tasks.

No reset, no Reflexion

A failed run on the practice track harms nobody. issue_refund on a real order is the public road: the money has moved, and a second attempt can move it again. Run attempts only against a sandbox or a copy of the data. The authors advise isolated environments for generated code too.

Conclusion

• When the approach is wrong, the same prompt tends to repeat the mistake. Reflexion adds a written note on what to change.
• It needs an environment you can reset and an evaluator you trust. False passes limit the gain.
• Every attempt is a full run, so it fits offline work like code with tests. It does not fit live answers or irreversible actions.

Go deeper