A fine-tuned model improves one task but regresses on important safety and generalization cases. What should the release process do?

Published
  1. Remove the regression cases from evaluation
  2. Promote every fine-tune because training completed
  3. Treat a newer model timestamp as proof of superiority
  4. Compare against the base and current production model using independent release criteria

Comments — 0