This website uses cookies to ensure you get the best experience.
Learn more.
A data pipeline loads a large daily dataset from S3 into Amazon Redshift. It currently issues one INSERT for every row, causing long load times. Which loading approach should the engineer evaluate?
Published
- Have one client stream all rows through individual INSERT statements with a larger network timeout.
- Run VACUUM before every individual INSERT as the primary bulk-load optimization.
- Use COPY to load appropriately sized source files in parallel, with the required IAM role and format settings.
- Use a Data Catalog crawler and assume catalog discovery loads the rows into Redshift.
Comments — 0
Sign in to comment