An endpoint experiences predictable daily peaks. What should guide autoscaling configuration?

Published
  1. Scale only on the model training accuracy.
  2. Assume autoscaling makes cold starts and downstream bottlenecks impossible.
  3. Set maximum instances to one regardless of the tested load.
  4. Load-test appropriate scaling metrics, targets, and bounds against the latency and throughput requirements.

Comments — 0