This website uses cookies to ensure you get the best experience.
Learn more.
A long-context assistant regularly sends the same large reference prefix followed by a short changing question. What optimization should be evaluated on a supported model?
Published
- Assume caching guarantees a hit for every request and every model.
- Insert a unique timestamp before all shared context on every request.
- Keep reusable content at the prefix and use supported prompt caching, measuring actual cache usage.
- Cache the final answer globally without considering user authorization.
Comments — 0
Sign in to comment