[Bug]: Fix TPS/User glitches in BTK

[MrGeva]

System Info

spikes of TPS/User is seen in the IBP graphs, especially for VLLM as seen in the image below. This is due to cpu bottleneck in the sampler, and can be fixed by increasing the number of api workers. See this: https://nvidia.slack.com/archives/C09H79C4MB8/p1772128612444079?thread_ts=1772088513.942399&cid=C09H79C4MB8

This should be fixed in all our supported frameworks.

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

na

Expected behavior

na

actual behavior

na

additional notes

na

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.