A team prepares examples for model customization. The evaluation set contains near-duplicates of training examples. Why should this be corrected?

Published
  1. The evaluation can overestimate generalization; separate duplicate groups across splits.
  2. Only the number of evaluation tokens matters, regardless of overlap.
  3. The model must be deployed before any split review can occur.
  4. Duplicates guarantee the evaluation is more independent.

Comments — 0