A data pipeline loads a large daily dataset from S3 into Amazon Redshift. It currently issues one INSERT for every row, causing long load times. Which loading approach should the engineer evaluate?

Published
  1. Have one client stream all rows through individual INSERT statements with a larger network timeout.
  2. Run VACUUM before every individual INSERT as the primary bulk-load optimization.
  3. Use COPY to load appropriately sized source files in parallel, with the required IAM role and format settings.
  4. Use a Data Catalog crawler and assume catalog discovery loads the rows into Redshift.

Comments — 0