Hyperparameters are selected using the same test set repeatedly. Which revised process is appropriate?

Published
  1. Report only the highest score from all attempted test configurations
  2. Keep tuning until the test score stops increasing
  3. Tune on training and validation folds, reserving the test set for final assessment
  4. Train only on the test set

Comments — 0