A model loads a large artifact during endpoint startup. A rollout times out before the container becomes ready. Which response is most appropriate?

Published
  1. Assume a larger model always loads instantly on every instance.
  2. Increase only the client request timeout after the rollout has already failed.
  3. Measure artifact loading and resource needs and use supported startup settings or optimized packaging.
  4. Delete part of the trained weights at random to shrink the artifact.

Comments — 0