This website uses cookies to ensure you get the best experience.
Learn more.
A developer loads a text model whose tokenizer differs from the tokenizer used during training. Responses become nonsensical. What should be checked first?
Published
- Change only the web page background color.
- Disable every log before further diagnosis.
- Sort the checkpoint filenames alphabetically.
- Load the tokenizer and special-token configuration compatible with the model checkpoint.
Comments — 0
Sign in to comment