A judge model scores outputs highly, but expert review finds a systematic error. What should the evaluation process do?

Published
  1. Increase judge output verbosity as proof of accuracy
  2. Validate and calibrate the judge against trusted reviewed examples
  3. Assume any model-generated score is definitive
  4. Remove all expert-reviewed failures

Comments — 0