Whisper V3 Large Turbo – Words/Sec Capped at ~284? Bottleneck or Parallelism Limit?

[AryanSakhala]

During a benchmarking run I have been doing, I found openai/whisper-large-v3-turbo showing some strange behaviour.
Irrespective of the concurrency or Sampling Rate of the audio, Words/Sec were constant around ~284.

Am I missing something?
I am using Loadbalancer - nginx
I have deployed it using Vllm

The architecture only uses 4 decoder layers (compared to 32 in Whisper Large), so I expected higher parallelism, but it seems capped.
Is this:

• A limitation of the model architecture?
• A runtime/framework bottleneck?
• Or am I missing an optimization?

Would love to hear from others who’ve tried pushing this model to its limits.