A text classifier performs well on short examples but fails on long documents because its tokenizer truncates the decisive section. What should be investigated?

Published
  1. Only a lower HTTP timeout.
  2. Tokenization limits and a validated document chunking or long-context modeling strategy.
  3. Only the model output vocabulary size.
  4. Delete all long documents from production without reviewing the business requirement.

Comments — 0