This website uses cookies to ensure you get the best experience.
Learn more.
A model endpoint meets its CPU target but users experience frequent failed predictions. Which operating practice best aligns monitoring with the product?
Published
- Treat all HTTP 200 responses as correct predictions without outcome checks.
- Declare success based only on average CPU.
- Monitor user-visible success, latency, and relevant model-quality outcomes alongside infrastructure metrics.
- Remove failed requests from the success-rate denominator.
Comments — 0
Sign in to comment