A support-ticket dataset contains repeated conversations from the same customer. What split best measures generalization to customers the model has never seen?

Published
  1. Evaluate only on the messages used for training.
  2. Randomly distribute individual messages from each conversation across all splits.
  3. Choose the split with the highest test score after many attempts.
  4. Keep every conversation from a customer entirely within one split.

Comments — 0