Why does using a higher-end GPU actually result in slower image generation speeds?

First, the total GPU time for generating images = GPU server startup time + runtime for image generation.

There's a difference in the startup speed among GPUs of different levels. The A100, as a popular high-end GPU, might require waiting in line so the startup time could be longer.

Additionally, if your workflow isn't complex, it won't utilize the full performance of the A100, and you might only save 1 or 2 seconds at most. However, the startup time could be 3 or 4 seconds slower, so the total time ends up being longer.

But if you're running a text-to-video workflow, which is comparatively complex, it can take advantage of the A100's capabilities, and the speed for running this text-to-video workflow could even be faster than a local RTX 4090 (based on my tests, the same workflow took 52.8 seconds on the A100 and 81.38 seconds on the 4090).

Therefore, after our testing, we recommend the following:

• If you are using the SD1.5 model or a relatively simple workflow, the T4 will suffice.
• For the SDXL model or more complex workflows, use the A10G.
• If you are generating videos, we suggest using the A100.