A model-as-judge evaluation gives a candidate model high scores, but domain experts find serious factual errors. What is the best next step?

1. Calibrate the rubric and judge against expert-labeled examples and inspect disagreement by failure type.
2. Use only a prompt asking the judge to always prefer the candidate.
3. Treat every judge score as objective proof and discard expert feedback.
4. Evaluate only examples already used to tune the candidate.