This website uses cookies to ensure you get the best experience.
Learn more.
A GPU has enough arithmetic throughput but insufficient memory for the chosen model and batch. What should be considered?
Published
- Add network bandwidth while ignoring model placement
- Increase only host swap and assume GPU capacity is unlimited
- Choose the largest batch without measuring device memory use
- Memory capacity, model partitioning or a smaller validated workload configuration
Comments — 0
Sign in to comment