A small task dataset cannot support full fine-tuning of a very large model reliably. What should be evaluated?

1. Using the test labels as training input
2. Parameter-efficient adaptation or frozen-feature approaches
3. Removing regularization from every experiment
4. Doubling parameters without measuring generalization