An application sends more input than a deployed model's supported context limit. What is the appropriate design response?

Published
  1. Disable authentication to allow a larger prompt
  2. Increase sampling randomness to enlarge the context window
  3. Rename the deployment to remove token limits
  4. Reduce or select relevant context, or choose a suitable supported model

Comments — 0