GPU: Possibility to use NO_FAST_MATH and deterministic mode for RTC

[davidrohr]

Fixes a related bug in constexpr optimization, that is very minor but was breaking the deterministic mode.
Full deterministic mode not yet possible with runtime switch, since the thrust sort kernels are not recompiled with RTC, need to check what to do.
This setup will now also allow a follow-up PR to set NO_FAST_MATH for compression / decompression kernels, fixing the float inconsistency in the track model when used with RTC.

[github-actions]

REQUEST FOR PRODUCTION RELEASES:
To request your PR to be included in production software, please add the corresponding labels called "async-" to your PR. Add the labels directly (if you have the permissions) or add a comment of the form (note that labels are separated by a ",")

+async-label <label1>, <label2>, !<label3> ...

This will add <label1> and <label2> and removes <label3>.

The following labels are available
async-2023-pbpb-apass4
async-2023-pp-apass4
async-2024-pp-apass1
async-2022-pp-apass7
async-2024-pp-cpass0
async-2024-PbPb-apass1
async-2024-ppRef-apass1
async-2024-PbPb-apass2
async-2023-PbPb-apass5

[davidrohr]

After trying a bit more, fully deterministic via only a runtime switch with RTC is impossible, since the host code preprocesses some values that are used for the initialization, thus if the host code is compiled with -ffast-math, it is already non-reproducible. Nothing we can do about it.

[alibuild]

Error while checking build/O2/fullCI_slc9 for 0c652eb at 2025-03-25 20:24:

## sw/BUILD/O2-RTC-test-latest/log
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:1: error: unknown type name 'GPUCA_DETERMINISTIC_CODE'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:37: error: expected unqualified-id
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:37: error: expected ')'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:395:28: error: expected expression
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:395:53: error: expected expression
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:1: error: unknown type name 'GPUCA_DETERMINISTIC_CODE'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:1: error: unknown type name 'GPUCA_DETERMINISTIC_CODE'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:37: error: expected unqualified-id
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:37: error: expected ')'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:395:28: error: expected expression
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:395:53: error: expected expression
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:1: error: unknown type name 'GPUCA_DETERMINISTIC_CODE'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:37: error: expected unqualified-id
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:37: error: expected ')'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:395:28: error: expected expression
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251u:1: error: unknown type name 'GPUCA_DETERMINISTIC_CODE'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:37: error: expected unqualified-id
251/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:395:53: error: expected expression
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:1: error: unknown type name 'GPUCA_DETERMINISTIC_CODE'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:395:28: error: expected expression
  1: error: unknown type name 'GPUCA_DETERMINISTIC_CODE'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:395:53: error: expected expression
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:37: error: expected ')'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:395:28: error: expected expression
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:395:53: error: expected expression
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:1: error: unknown type name 'GPUCA_DETERMINISTIC_CODE'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:37: error: expected unqualified-id
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:37: error: expected ')'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:395:28: error: expected expression
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:395:53: error: expected expression
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:1: error: unknown type name 'GPUCA_DETERMINISTIC_CODE'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:1: error: unknown type name 'GPUCA_DETERMINISTIC_CODE'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:1: error: unknown type name 'GPUCA_DETERMINISTIC_CODE'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:37: error: expected unqualified-id
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:395:28: error: expected expression
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:395:53: error: expected expression
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:37: error: expected ')'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:37: error: expected ')'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:37: error: expected unqualified-id
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:37: error: expected ')'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:395:28: error: expected expression
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:395:53: error: expected expression
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/GPUTracking/Merger/GPUTPCGMMergerTypes.h:35:3: error: a type specifier is required for all declarations
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/GPUTracking/Merger/GPUTPCGMMergerTypes.h:35:40: error: expected ';' at end of declaration list
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/GPUTracking/Merger/GPUTPCGMMergerTypes.h:35:3: error: a type specifier is required for all declarations
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/GPUTracking/Merger/GPUTPCGMMergerTypes.h:35:40: error: expected ';' at end of declaration list
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:1: error: unknown type name 'GPUCA_DETERMINISTIC_CODE'
/sw/SOURCES/O2/14109-slc9_x86-64/0/GPU/Common/GPUCommonMath.h:251:37: error: expected unqualified-id
[0 more errors; see full log]

Full log here.

[alibuild]

Error while checking build/O2/fullCI_slc9 for 7cb35d8 at 2025-03-26 04:25:

[603/5741] Building CXX object Framework/Core/CMakeFiles/o2-test-framework-core.dir/test/test_Expressions.cxx.o
[604/5741] Building CXX object Framework/Core/CMakeFiles/O2bench-framework-benchmark-ASoA.dir/__/__/Common/Utils/src/fpu.cxx.o
[605/5741] Building CXX object Framework/Core/CMakeFiles/o2-test-framework-core.dir/test/test_RootConfigParamHelpers.cxx.o
[606/5741] Building CXX object Framework/Core/CMakeFiles/o2-test-framework-core.dir/test/test_OverrideLabels.cxx.o
[607/5741] Building CXX object Framework/Core/CMakeFiles/O2bench-framework-benchmark-ASoAHelpers.dir/__/__/Common/Utils/src/fpu.cxx.o
[608/5741] Building CXX object Framework/Core/CMakeFiles/O2bench-framework-benchmark-EventMixing.dir/__/__/Common/Utils/src/fpu.cxx.o
[609/5741] Building CXX object Framework/Core/CMakeFiles/o2-test-framework-core.dir/test/test_Services.cxx.o
[610/5741] Building CXX object Framework/Core/CMakeFiles/O2bench-framework-benchmark-HistogramRegistry.dir/__/__/Common/Utils/src/fpu.cxx.o
[611/5741] Building CXX object Framework/Core/CMakeFiles/o2-test-framework-core.dir/test/test_WorkflowSerialization.cxx.o
[612/5741] Building CXX object Framework/Core/CMakeFiles/O2bench-framework-benchmark-TableToTree.dir/__/__/Common/Utils/src/fpu.cxx.o
[613/5741] Building CXX object Framework/Core/CMakeFiles/O2bench-framework-benchmark-DataDescriptorMatcher.dir/test/benchmark_DataDescriptorMatcher.cxx.o
[614/5741] Building CXX object Framework/Core/CMakeFiles/O2bench-framework-benchmark-ExternalFairMQDeviceProxies.dir/__/__/Common/Utils/src/fpu.cxx.o
[615/5741] Building CXX object Framework/Core/CMakeFiles/o2-test-framework-core.dir/test/unittest_DataSpecUtils.cxx.o
[616/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-CallbackService.dir/__/__/Common/Utils/src/fpu.cxx.o
[617/5741] Building CXX object Framework/Core/CMakeFiles/o2-test-framework-core.dir/test/test_TimeParallelPipelining.cxx.o
[618/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-RegionInfoCallbackService.dir/__/__/Common/Utils/src/fpu.cxx.o
[619/5741] Building CXX object Framework/Core/CMakeFiles/O2exe-framework-ao2d-to-ao3d.dir/test/o2AO2DToAO3D.cxx.o
[620/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-DanglingInputs.dir/__/__/Common/Utils/src/fpu.cxx.o
[621/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-Timers.dir/test/test_Timers.cxx.o
[622/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-DanglingOutputs.dir/__/__/Common/Utils/src/fpu.cxx.o
[623/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-AlgorithmWrapper.dir/test/test_AlgorithmWrapper.cxx.o
[624/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-DataAllocator.dir/__/__/Common/Utils/src/fpu.cxx.o
[625/5741] Building CXX object Framework/Core/CMakeFiles/O2bench-framework-benchmark-InputRecord.dir/test/benchmark_InputRecord.cxx.o
[626/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-StaggeringWorkflow.dir/__/__/Common/Utils/src/fpu.cxx.o
[627/5741] Building CXX object Framework/Core/CMakeFiles/O2exe-dpl-run.dir/src/dplRun.cxx.o
[628/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-Forwarding.dir/__/__/Common/Utils/src/fpu.cxx.o
[629/5741] Building CXX object Framework/Core/CMakeFiles/O2exe-dpl-null-sink.dir/src/o2NullSink.cxx.o
[630/5741] Building CXX object Framework/Core/CMakeFiles/o2-test-framework-core.dir/test/test_TableBuilder.cxx.o
[631/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-ParallelPipeline.dir/__/__/Common/Utils/src/fpu.cxx.o
[632/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-ParallelProducer.dir/__/__/Common/Utils/src/fpu.cxx.o
[633/5741] Building CXX object Framework/Core/CMakeFiles/O2bench-framework-benchmark-DataRelayer.dir/test/benchmark_DataRelayer.cxx.o
[634/5741] Building CXX object Framework/Core/CMakeFiles/O2bench-framework-benchmark-DeviceMetricsInfo.dir/test/benchmark_DeviceMetricsInfo.cxx.o
[635/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-SlowConsumer.dir/__/__/Common/Utils/src/fpu.cxx.o
[636/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-SlowProducerWithWildCard.dir/__/__/Common/Utils/src/fpu.cxx.o
[637/5741] Building CXX object Framework/Core/CMakeFiles/o2-test-framework-core.dir/test/test_IndexBuilder.cxx.o
[638/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-SimpleDataProcessingDevice01.dir/__/__/Common/Utils/src/fpu.cxx.o
[639/5741] Building CXX object Framework/Core/CMakeFiles/o2-test-framework-core.dir/test/test_WorkflowHelpers.cxx.o
[640/5741] Building CXX object Framework/Core/CMakeFiles/o2-test-framework-core.dir/test/test_ASoA.cxx.o
[641/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-SimpleStatefulProcessing01.dir/__/__/Common/Utils/src/fpu.cxx.o
[642/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-SimpleStringProcessing.dir/__/__/Common/Utils/src/fpu.cxx.o
[643/5741] Building CXX object Framework/Core/CMakeFiles/O2exe-test-framework-ConsumeWhenAllOrdered.dir/test/test_ConsumeWhenAllOrdered.cxx.o
[644/5741] Building CXX object Framework/Core/CMakeFiles/o2-test-framework-core.dir/test/test_Variants.cxx.o
[645/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-SimpleTimer.dir/__/__/Common/Utils/src/fpu.cxx.o
[646/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-SimpleWildcard.dir/__/__/Common/Utils/src/fpu.cxx.o
[647/5741] Building CXX object Framework/Core/CMakeFiles/O2exe-framework-crashing-workflow.dir/test/test_CrashingWorkflow.cxx.o
[648/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-SimpleWildcard02.dir/__/__/Common/Utils/src/fpu.cxx.o
[649/5741] Building CXX object Framework/Core/CMakeFiles/O2bench-framework-benchmark-WorkflowHelpers.dir/test/benchmark_WorkflowHelpers.cxx.o
[650/5741] Building CXX object Framework/Core/CMakeFiles/o2-test-framework-core.dir/test/test_GroupSlicer.cxx.o
[651/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-SingleDataSource.dir/__/__/Common/Utils/src/fpu.cxx.o
[652/5741] Building CXX object Framework/Core/CMakeFiles/O2test-framework-Task.dir/__/__/Common/Utils/src/fpu.cxx.o

Full log here.