The GPU-as-service cloud market is evolving rapidly. By now, it’s common knowledge that many so-called neoclouds aren’t efficiently utilizing their GPUs. Often, they buy too many GPUs per customer in an effort to cover all necessary processing, only to end up with the costly components sitting idle, particularly in inference workloads with bursty traffic.
