CUDA performance regression due to DefaultMaxRegistersPerThread

[ilehtoranta]

There is a severe performance regression in the CUDA backend, introduced by DefaultMaxRegistersPerThread:

public static int DefaultMaxRegistersPerThread { get; set; } = 255;

In one of my kernels, this caused the kernel to use 94 registers instead of 42 when compiled against the old version (4.7.2). When set to 0, I get identical kernels with 4.7.2.

This probably need some rethought, because the default is performance killer.

[LostBeard]

Thanks for reporting this. I'll look into it asap.

[LostBeard]

Fixed in 4.9.6 (commit 2ec94d6, currently 4.9.6-local.1 on my local feed; rc to nuget.org shortly).

Root cause was that CU_JIT_MAX_REGISTERS=255 doesn't simply cap usage — it changes ptxas's behavior. Ptxas treats the limit as a signal that the caller is OK with high register usage and stops applying its occupancy-favoring heuristics. For a normal kernel that would have been happy at 42 registers, it picks higher-register optimizations because nothing is pushing it the other way. Exactly the 42 → 94 jump you saw.

CudaAccelerator.DefaultMaxRegistersPerThread is back to 0 (pre-rc.24 behavior — let ptxas decide). The property still exists for the edge case it was added for, so if you ever hit CUDA_ERROR_LAUNCH_OUT_OF_RESOURCES on a register-heavy kernel you can set it to 255 explicitly per-kernel or per-accelerator.

Thanks for the report — including the before/after register counts made the root cause immediately clear.