Fill alternative TPC dedx according to dEdxClusterRejectionFlagMaskAlt

[shahor02]

Use this TrackTPC::getdEdxAlt() to populate AOD TrackQA tpcdEdx... data while the standard TrackTPC::getdEdx() will be used to fill AOD TPC tracks tpcSignal.

@davidrohr this hack was discussed today at the meeting to provide the "TrackQA" table in the AOD with dedx info calculated with minimal cluster cuts (different from the ones going to AOD TPCsignal). I've checked on 38 kHz run, works fine. Could you confirm that no explicit extra memory allocation is needed to accommodate for this https://github.com/AliceO2Group/AliceO2/compare/dev...shahor02:pr_tpcdedxAlt?expand=1#diff-9d885f1b8b692362a0e978b866b7978ad80636bded126bafacd22e96dd382a45R302 ?

Here how it looks for GPU_rec_tpc.dEdxClusterRejectionFlagMask=23;GPU_rec_tpc.dEdxClusterRejectionFlagMaskAlt=1:

tpcrec->Scan("TPCTracks.mdEdx.NHitsIROC : TPCTracks.mdEdx.dEdxTotIROC : TPCTracks.mdEdxAlt.NHitsIROC : TPCTracks.mdEdxAlt.dEdxTotIROC")
***********************************************************************
*    Row   * Instance * TPCTracks *  TPCTrack *  TPCTrack *  TPCTrack *
***********************************************************************
*        0 *        0 *         8 *  1147.625 *        14 * 1096.3125 *
*        0 *        1 *        10 * 84.541671 *        12 * 65.642860 *
*        0 *        2 *         3 *     22.75 *         8 *   87.0625 *
*        0 *        3 *         1 *        61 *        11 * 42.642860 *
*        0 *        4 *         5 * 33.083335 *         8 *        56 *
*        0 *        5 *         2 *    115.75 *        12 * 30.357143 *
*        0 *        6 *         1 *    174.25 *         7 *    97.375 *
*        0 *        7 *         3 *     73.25 *         4 *     70.25 *
*        0 *        8 *         8 * 57.200000 *        10 *      41.5 *
*        0 *        9 *        14 * 59.083332 *        19 * 58.729167 *
*        0 *       10 *         4 *        48 *         8 *      58.5 *
*        0 *       11 *         9 * 72.849998 *        11 * 78.541671 *
*        0 *       12 *         9 * 218.95834 *        10 * 218.95834 *
*        0 *       13 *         2 *     136.5 *         2 *     136.5 *
*        0 *       14 *         0 *         0 *         9 * 122.80000 *
*        0 *       15 *         2 *     130.5 *        12 * 88.035720 *
*        0 *       16 *         6 * 55.166667 *        11 * 57.458335 *
*        0 *       17 *         2 *       119 *         5 * 89.916671 *
*        0 *       18 *         3 *        63 *         8 *      38.5 *
*        0 *       19 *         8 *    66.375 *         8 *    66.375 *
*        0 *       20 *        10 * 62.416667 *        11 * 62.416667 *
*        0 *       21 *        11 *    489.75 *        16 * 344.33334 *
*        0 *       22 *         6 *     90.75 *         9 * 81.599998 *
*        0 *       23 *         4 *     90.75 *         8 *    96.875 *

[github-actions]

REQUEST FOR PRODUCTION RELEASES:
To request your PR to be included in production software, please add the corresponding labels called "async-" to your PR. Add the labels directly (if you have the permissions) or add a comment of the form (note that labels are separated by a ",")

+async-label <label1>, <label2>, !<label3> ...

This will add <label1> and <label2> and removes <label3>.

The following labels are available
async-2023-pbpb-apass4
async-2023-pp-apass4
async-2024-pp-apass1
async-2022-pp-apass7
async-2024-pp-cpass0
async-2024-PbPb-apass1
async-2024-ppRef-apass1
async-2024-PbPb-apass2
async-2023-PbPb-apass5

[alibuild]

Error while checking build/O2/fullCI_slc9 for 4cac3dc at 2025-04-03 02:08:

## sw/BUILD/O2Physics-latest/log
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:


## sw/BUILD/O2-full-system-test-latest/log
Detected critical problem in logfile reco_NOGPU.log
reco_NOGPU.log:[30092:internal-dpl-ccdb-backend]: [02:08:46][ERROR] Exception while running: Fatal error. Rethrowing.
reco_NOGPU.log-[30092:internal-dpl-ccdb-backend]: [02:08:46][FATAL] Unhandled o2::framework::runtime_error reached the top of main of o2-itsmft-stf-decoder-workflow, device shutting down. Reason: Fatal error
[30092:internal-dpl-ccdb-backend]: [02:08:46][ERROR] CCDBDownloader CURL transfer error - Timeout was reached
[30092:internal-dpl-ccdb-backend]: [02:08:46][ERROR] CcdbDownloader finished transfer http://alice-ccdb.cern.ch/GLO/Config/GRPECS for 1550600800000 (agent_id: alimetal04.cern.ch-1743638915-Zmtl4U) with http code: 0
[30092:internal-dpl-ccdb-backend]: [02:08:46][ERROR] File GLO/Config/GRPECS could not be retrieved. No more hosts to try.
[30092:internal-dpl-ccdb-backend]: [02:08:46][FATAL] Unable to find CCDB object GLO/Config/GRPECS/1550600800000
[30092:internal-dpl-ccdb-backend]: [02:08:46][ERROR] Exception while running: Fatal error. Rethrowing.
[30092:internal-dpl-ccdb-backend]: [02:08:46][FATAL] Unhandled o2::framework::runtime_error reached the top of main of o2-itsmft-stf-decoder-workflow, device shutting down. Reason: Fatal error
[ERROR] Workflow crashed - PID 30092 (internal-dpl-ccdb-backend) did not exit correctly however it's not clear why. Exit code forced to 128.


## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -v clang-diagnostic-error error-log.txt
++ grep ' error:'
++ GRERR=1
++ [[ 1 == 0 ]]
++ mkdir -p /sw/INSTALLROOT/6d9c7cf4fce3182ef0d8600aa9ff5811f1eb9f42/slc9_x86-64/o2checkcode/1.0-local70/etc/modulefiles
++ cat
--

Full log here.

[miranov25]

Suport information: dE/dx Calibration and Encoding Precision

The TPC is geometrically calibrated, so the mean ratio of all dE/dx estimators should ideally be 1.0. This implies that the mean of the uchar8-coded value will be 100.

There is a small deviation from unity due to the transfer function, which I explained on Monday — this deviation is typically within ±10%.

The relative resolution of the dE/dx measurement at low interaction rates is:

• Approximately 10% for per-stack dE/dx
• Around 6% for the combined dE/dx

Encoding a 10% resolution using 1% steps is already acceptable.

For the 6% resolution, a 1% encoding step yields:

• A quantization noise (standard deviation) of approximately 1% / √12 ≈ 0.29%,
  which is much smaller than the intrinsic spread, and sufficient for calibration purposes.

In conclusion, 1% binning is precise enough to preserve the information content and is well matched to the physical resolution of the measurement.

[miranov25]

Test: Relative Error After Decoding

As expected, the precision of uchar8-coded dE/dx values follows the statistical estimate:
σ ≈ 1% / √12 ≈ 0.29%

This was validated using debug streamers with the following command:

trackQA->Draw("((trackQAHolder.tpcdEdxMax0R * mdEdx.dEdxTotTPC) / mdEdx.dEdxMaxIROC / 100.) - 1 >> his(100, -0.1, 0.1)", "mdEdx.dEdxTotIROC < 1000 && mdEdx.dEdxTotIROC > 1", "", 10000000)

The measured σ is slightly above 0.3%, as expected, due to natural fluctuation around the 1.0 mean.
Still, I strongly prefer to have a variable stored with 0.29% precision even if it fluctuates by 10%, rather than not storing it at all.

✅ Please keep the data encoding as suggested — uchar8 is sufficient for this use case.

---

Additional Remark (from WP Meeting Discussion): Avoiding Aliasing in AO2D Decoding

In the decoding step, I intentionally apply a small random shift up to ±1% to each value. This serves two purposes:

• It avoids aliasing artifacts in ratio-based comparisons.
• It compensate the rounding bias ~0.5%.

This technique has been consistently applied in all of my dE/dx calibration workflows and will also be used for track deltas stored in trackQA.

[miranov25]

Hello @shahor02 , @f3sch, and @wiechula

We need to ensure that normalization is applied consistently across all components. I assume you already have a version of the code where the dE/dx values are normalized to the reference value stored in trackExtra. However, this does not appear to be the case in the current version of the pull request.

If normalization is not handled correctly, it will result in incorrectly encoded values, which will cause major inconsistencies in downstream analysis — this would be a serious issue.

@f3sch, I’m not sure how the pull request is scheduled to be merged, but before that happens, I’d strongly prefer to test a new version of the AO2DStreamer.root used in the test above, to confirm that the data is internally consistent.

Best regards,
Marian

[shahor02]

@miranov25 this is a version storing

const float dEdxNorm = (dEdxInfoAlt.dEdxTotTPC > 0) ? 100. / dEdxInfoAlt.dEdxTotTPC : 0;
trackQAHolder.tpcSignalTodEdxNorm = uint8_t(aodTPCSignal * dEdxNorm);

with aodTPCSignal (value stored in the extraInfo.TPCSignal) overridden to the dEdxInfoAlt.dEdxTotTPC if dEdxInfo.dEdxTotTPC == 0.

[miranov25]

Hello @shahor02,

Regarding the implementation — after some further discussion, I agree it’s better to use clipping instead of simple casting. In a similar way as for the track delta in the trackQA.

For example:

trackQAHolder.tpcSignalTodEdxNorm = ((aodTPCSignal * dEdxNorm) > 255) ? 255 : uint8_t(aodTPCSignal * dEdxNorm);

This will code overflow while preserving maximum precision.

---

On Switching Between Default and Alt Configurations

We need to know what normalization factor was used during encoding — otherwise, it becomes impossible to decode or compare the original values correctly. Maybe I’m misunderstanding how the new implementation works, and perhaps this is already accounted for — but from what I’ve seen, it still feels like this information is missing.

---

I strongly suggest we meet in person to clarify all the details, to avoid any miscommunication.

It will also be critical to include the applied normalization and algorithm information in the streamers, so we can validate things directly at the data level.

Best regards,
Marian

[shahor02]

Closing in favour of #14144