A model fits on one accelerator, but training time should be reduced across several devices using independent example batches. Which approach best fits?

1. Evaluate data-parallel distributed training.
2. Evaluate a supported model-parallel or sharded training strategy.
3. Use a bounded, tracked hyperparameter tuning strategy.
4. Profile and optimize the input pipeline with suitable caching and parallelism.