This website uses cookies to ensure you get the best experience.
Learn more.
An endpoint experiences predictable daily peaks. What should guide autoscaling configuration?
Published
- Scale only on the model training accuracy.
- Assume autoscaling makes cold starts and downstream bottlenecks impossible.
- Set maximum instances to one regardless of the tested load.
- Load-test appropriate scaling metrics, targets, and bounds against the latency and throughput requirements.
Comments — 0
Sign in to comment