A RAG corpus contains duplicate copies of the same policy in several folders. The top results repeat those passages and crowd out complementary evidence. Which ingestion improvement is best?

Published
  1. Use a larger output limit as the only ingestion correction.
  2. Increase every duplicate similarity score.
  3. Deduplicate by stable document identity or content and preserve useful provenance metadata.
  4. Remove all document identifiers and ingest every copy as new content.

Comments — 0