Skip to content

// Feedback loops · ~2 min read

Evaluator-Optimizer

Reach for it when: a scorer can reject work until an optimizer improves it.

After this lesson you can: Loop a scorer and a reviser until the score clears a threshold or the round limit is reached.

// 60-second mental model

How to hold it in your head

One role proposes an answer. Another scores it against a rubric. If the score is too low, an optimizer revises — and you loop until the score clears or you hit a hard stop.

Evaluator-Optimizer separates proposal from scoring: an evaluator returns a numeric or structured score; an optimizer revises until a threshold is met or maxIterations is exhausted.

How this differs

Three lessons use a draft → check → revise loop. Reflection: the same model critiques its own draft against a rubric. Evaluator-Optimizer: a separate scorer returns a number, and you revise until it clears a threshold. Critic-Actor: a separate critic returns pass or revise with notes, and after a capped number of retries a human decides.

// Mini architecture

Propose → score → revise (threshold or stop)

A task and a rubric go to an Optimizer, which writes candidate N. An Evaluator returns a score and reasons. If the score meets the threshold or the iteration limit is reached, the candidate is final; otherwise the Optimizer revises and the loop repeats.

The evaluator only scores. The optimizer only revises. Neither invents a new goal mid-loop.

// Mini-project

Score-gated product blurb (stub)

Optimizer + evaluator stubs with a numeric threshold

Goal: Improve a canned product blurb until a stub evaluator scores ≥8/10 on clarity and honesty, or hit maxIterations=3.

  1. Start with a hype-heavy canned blurb.
  2. Rubric axes (stub): clarity, honesty, length ≤80 words — average 0–10.
  3. Evaluator stub returns `{ score, reasons[] }` — never rewrite the blurb itself.
  4. Optimizer revises only the failing axes; log score each round.
  5. Stop at score ≥8 or maxIterations=3; emit final + history.
  6. Refuse a fourth round even if the score is still low.

Stubs are the point. This is a pattern to implement in your own stack: score, revise, and stop at the threshold or after three rounds.

// Common failure

What goes wrong

Symptom

Endless polishing — the optimizer chases tiny issues forever and never clears the threshold.

Fix

Hard maxIterations plus a “good enough” rule: ship when remaining failures are style-only, or when iterations are exhausted.

// Self-reflection

Sit with this

Where are you rewriting by vibe instead of letting a scorer reject and a reviser improve?

Sources

Session only. Nothing is saved.