A team must choose between a general foundation model and a smaller task-adapted model for high-volume extraction. Which experiment best supports the decision?

Published
  1. Compare only outputs from training examples.
  2. Choose the largest model solely because it has more parameters.
  3. Choose the cheapest model without testing required fields.
  4. Compare both on a held-out task set using extraction correctness, latency, and total cost per successful result.

Comments — 0