A request has high end-to-end latency despite fast GPU kernels. What should be profiled?

Published
  1. Only the model's parameter count
  2. Preprocessing, transfers, queueing and postprocessing
  3. Only the number of classes in the training labels
  4. Only the kernel's best single timing

Comments — 0