A model endpoint meets its CPU target but users experience frequent failed predictions. Which operating practice best aligns monitoring with the product?

Published
  1. Treat all HTTP 200 responses as correct predictions without outcome checks.
  2. Declare success based only on average CPU.
  3. Monitor user-visible success, latency, and relevant model-quality outcomes alongside infrastructure metrics.
  4. Remove failed requests from the success-rate denominator.

Comments — 0