New files arrive continuously in object storage. The pipeline needs supported incremental file discovery and resumable processing. Which capability should be evaluated?

Published
  1. Reimport every historical file on every trigger with no progress tracking
  2. A table comment as the ingestion cursor
  3. Auto Loader with appropriate checkpoint and schema state
  4. A DNS lookup for each data row

Comments — 0