Bad interference for TH1.Fit and EnableImplicitMT

[VanyaBelyaev]

Check duplicate issues.

• Checked for duplicates

Description

Dear Experst

I've observed a and interference between TH1.Fit and EnableImplicitMT in PyROOT

ROOT from LCG dev3 cvmfs nightly build

Reproducer is trivial:

import ROOT, random

xmin = 0
xmax = 1 

h1 = ROOT.TH1F ( 'h1' , '' , 20 , xmin , xmax ) 

for i in range ( 1000 ) : h1.Fill ( random.gauss ( 0 , 1 ) )

def func ( x , pars ) :
    x0  = x [ 0 ]    
    return pars [ 0 ] + pars [ 1 ] * x0 

my_func = ROOT.TF1( 'f1' , func, xmin , xmax , npar = 2 , ndim = 1 )

my_func.SetParNames  ( 'A'  , 'B'  , )
my_func.SetParameter ( 0 ,  1 )
my_func.SetParameter ( 1 , -1 )


## If commented OUT: ok 
ROOT.ROOT.EnableImplicitMT()   ## THIS LINE

r = h1.Fit ( my_func , 'SE0Q' , '' , xmin , xmax ) 

print ( r ) 

In case EnableImplicitMT is commented out it works nicely:

( source /cvmfs/sft.cern.ch/lcg/views/dev3/latest/x86_64-el9-clang19-opt/setup.sh ; python ./hfit.py ) 

But if this line is uncommented, it hangs....

Reproducer

see above

ROOT version

ROOT from LCG dev3 nighly slot i cvmfs

Installation method

LCG nightly builds at cvmfs, dev3 slot

Operating system

linux, lxplus

Additional context

[ferdymercury]

Seems a duplicate of #20792
(https://root-forum.cern.ch/t/tgraph-fit-usage-with-enableimplicitmt-is-tgraph-fit-thread-safe/64541/4) see comment by @guitargeek

[VanyaBelyaev]

surely it migth be somehow related...
but in my case I do not have results at all...
It just hangs forever... and no results produced
... and the reproducer is much simpler - just trivial.

[ferdymercury]

It seems that it works fine in C++ so it might be a Python-GIL issue:

#include <TROOT.h>
#include <TH1F.h>
#include <TF1.h>
#include <TFitResultPtr.h>

double func(double *x, double *par)
{
    return par[0] + par[1] * x[0];
}

void fit_example()
{
    double xmin = 0.0;
    double xmax = 1.0;

    TH1F *h1 = new TH1F("h1", "", 20, xmin, xmax);
    h1->FillRandom("gaus", 1000);

    TF1 *my_func = new TF1("f1", func, xmin, xmax, 2);
    my_func->SetParNames("A", "B");
    my_func->SetParameter(0,  1.0);
    my_func->SetParameter(1, -1.0);

    ROOT::EnableImplicitMT();
    TFitResultPtr r = h1->Fit(my_func, "SE0Q", "", xmin, xmax);
    r->Print();
}

[vepadulano]

Dear @VanyaBelyaev ,

Thank you for reporting your issue! Just to clarify, Are you expecting a Python function to behave any differently when running with or without ROOT.EnableImplicitMT()? Python takes the GIL anyway so I wouldn't expect any practical difference in terms of performance.

OTOH, making your program hang just because you are calling EnableImplicitMT is bad, so we'll investigate that in any case.

[VanyaBelyaev]

Dear @vepadulano

Actuallty I do not have any explicit intentions to enable or disable MT for the histogram fit :-)
In my real application EnableImplcitMT comes much earlier (and totally independently).
This is just a very simplified reproducer of the problem. For my real script I have a sequence of steps,
some of them rely on data-frames (and EnableImplicitMT comes from this earlier step,
surely I cannot avoid it) and at the end of sequnce I have a simple histogram fit.

[vepadulano]

Dear @VanyaBelyaev ,

Thanks for explaining further. I agree in that case there's no reason to remove EnableImplicitMT from the earlier steps of the program.

[VanyaBelyaev]

Seems a duplicate of #20792 (https://root-forum.cern.ch/t/tgraph-fit-usage-with-enableimplicitmt-is-tgraph-fit-thread-safe/64541/4) see comment by @guitargeek

Just for clarity, following the TGrath.Fit issue #20792, I've added explicit kNo argement to TF1 costructor. And no changes are obseved. It still hangs. It looks like the issue indeed is totally different from #20792

[hageboeck]

OK, the cause of the problem is found: The python function in this fit needs the interpreter lock, but the lock was also held by the call to TH1.Fit. We can release the latter, so the fit completes (albeit one thread at a time).
So there is no significant gain from using IMT, but at least the workflow won't deadlock.

[VanyaBelyaev]

Dear @hageboeck

Thank you for good news!

I do not worry too much on the perfomance gain here.

The upstream appearence of EnableImplicitMT is just unavoidable for my flow, since input data are first processed
with data-frames.

Currently as a purely temporary solution I use a trivial context manager that disables locally ImplicitMT,
and, at the exit, restores it's state. I am explicitely using this manager around TH1.Fit and it works for me,
It is a kind of temporary "solution" , but proper solution is welcome.
Thank you!