This website uses cookies to ensure you get the best experience.
Learn more.
New files arrive continuously in object storage. The pipeline needs supported incremental file discovery and resumable processing. Which capability should be evaluated?
Published
- Reimport every historical file on every trigger with no progress tracking
- A table comment as the ingestion cursor
- Auto Loader with appropriate checkpoint and schema state
- A DNS lookup for each data row
Comments — 0
Sign in to comment