An append-only batch contains duplicate rows by order_id. What does dropDuplicates(["order_id"]) address?

1. Duplicate keys within the processed DataFrame
2. Every future duplicate across all independent jobs automatically
3. Guaranteed selection of the latest timestamp per key
4. Automatic enforcement of referential integrity in all tables