An interactive assistant produces unnecessarily long responses, increasing token use and time until completion. Users need concise answers with the option to ask for detail. Which change is the best initial fit?

Published
  1. Enable streaming and assume the model no longer generates or bills output tokens.
  2. Increase the maximum output limit for every request and require the longest possible response.
  3. Reduce temperature and assume it alone imposes a guaranteed one-sentence output format.
  4. Use concise output instructions and a suitable output-token budget, and evaluate streaming if the interface benefits from progressive display.

Comments — 0