A RAG experiment improves answer correctness but doubles latency beyond the service target. What should the team do?

Published
  1. Hide slow examples from the evaluation report.
  2. Convert milliseconds to seconds and claim latency decreased.
  3. Evaluate the quality-latency tradeoff against explicit acceptance criteria.
  4. Ship automatically because correctness is the only possible requirement.

Comments — 0