A developer tunes a prompt repeatedly against one benchmark and then reports that same benchmark as final quality. What should be added?

Published
  1. Only a longer system prompt.
  2. A separately held-out representative evaluation set not used for prompt selection.
  3. Only the best response from several repeated attempts.
  4. More tuning against the same examples while calling them unseen.

Comments — 0