This website uses cookies to ensure you get the best experience.
Learn more.
A team reports an improved AI assistant using a test set repeatedly inspected during prompt tuning. What is needed for a credible final quality estimate?
Published
- Use only the training examples because their expected answers are known.
- Choose the best single run and omit all other outcomes.
- Use a separately held-out representative evaluation set and document the acceptance criteria.
- Remove difficult cases after seeing their results.
Comments — 0
Sign in to comment