A deployment's p95 latency rises while each replica is highly utilized. Which change should be evaluated with load testing?

Published
  1. Removing all input validation
  2. Only renaming the model artifact
  3. Suitable replica capacity, shape, and endpoint bottlenecks
  4. Only increasing the training dataset size

Comments — 0