This website uses cookies to ensure you get the best experience.
Learn more.
A feature engineering job repeatedly scans a large text dataset to select a few columns for one date. Which data organization should be evaluated?
Published
- Duplicate all columns into every file regardless of query needs.
- Partition on useful predicates and use a suitable columnar format such as Parquet.
- Combine every date into one uncompressed text object.
- Use random partition labels unrelated to the queries.
Comments — 0
Sign in to comment