Writing to disk interpreted class with data member needing a collection proxy fails with RNTuple but not TTree

[vepadulano]

Check duplicate issues.

• Checked for duplicates

Description

The following example writes an interpreted class to disk via TTree

#include <TInterpreter.h>
#include <TFile.h>
#include <TTree.h>
#include <iostream>

void write_tree(const char *datasetname, const char *filename)
{
    auto f = std::make_unique<TFile>(filename, "recreate");
    auto t = std::make_unique<TTree>(datasetname, datasetname);

    gInterpreter->ProcessLine("MyEvent myEvt{};");
    std::unique_ptr<TInterpreterValue> v = gInterpreter->MakeInterpreterValue();
    gInterpreter->Evaluate("myEvt", *v);
    void *voidPtr = v->GetAsPointer();
    t->Branch("evt", "MyEvent", &voidPtr);
    t->Fill();
    f->Write();
}

void read_tree(const char *datasetname, const char *filename)
{
    auto f = std::make_unique<TFile>(filename);
    std::unique_ptr<TTree> t{f->Get<TTree>(datasetname)};

    t->Print();
    t->Scan();
}

void generate_class()
{
    gInterpreter->Declare(R"(
        struct MyEvent{
            int evtId{42};
            std::vector<int> charge{1, -1, 1};
            std::vector<float> pt{5, 10, 15};
            std::vector<float> eta{0, 1, 2};
            std::map<short, short> map{{1, 1}, {2, 2}, {3, 3}};
        };
        )");
}

int main()
{
    auto tree_name{"tree"};
    auto tree_file{"tree.root"};
    generate_class();

    std::cout << "Storing TTree with TTree\n\n";
    write_tree(tree_name, tree_file);
    std::cout << "Reading TTree with TTree\n";
    read_tree(tree_name, tree_file);
}

Running the example results in

Storing TTree with TTree

Error in <TStreamerInfo::Build>: The class "MyEvent" is interpreted and for its data member "map" we do not have a dictionary for the collection "map<short,short>". Because of this, we will not be able to read or write this data member.
Reading TTree with TTree
******************************************************************************
*Tree    :tree      : tree                                                   *
*Entries :        1 : Total =            3409 bytes  File  Size =       1182 *
*        :          : Tree compression factor =   1.00                       *
******************************************************************************
*Branch  :evt                                                                *
*Entries :        1 : BranchElement (see below)                              *
*............................................................................*
*Br    0 :evtId     : Int_t                                                  *
*Entries :        1 : Total  Size=        565 bytes  File Size  =         76 *
*Baskets :        1 : Basket Size=      32000 bytes  Compression=   1.00     *
*............................................................................*
*Br    1 :charge    : vector<int>                                            *
*Entries :        1 : Total  Size=        600 bytes  File Size  =        107 *
*Baskets :        1 : Basket Size=      32000 bytes  Compression=   1.00     *
*............................................................................*
*Br    2 :pt        : vector<float>                                          *
*Entries :        1 : Total  Size=        580 bytes  File Size  =        103 *
*Baskets :        1 : Basket Size=      32000 bytes  Compression=   1.00     *
*............................................................................*
*Br    3 :eta       : vector<float>                                          *
*Entries :        1 : Total  Size=        585 bytes  File Size  =        104 *
*Baskets :        1 : Basket Size=      32000 bytes  Compression=   1.00     *
*............................................................................*
***********************************************************************
*    Row   * Instance * evt.evtId * evt.charg *    evt.pt *   evt.eta *
***********************************************************************
*        0 *        0 *        42 *         1 *         5 *         0 *
*        0 *        1 *        42 *        -1 *        10 *         1 *
*        0 *        2 *        42 *         1 *        15 *         2 *
***********************************************************************

That is, TStreamerInfo reports an error about missing dictionary and the fact the class can't be stored to disk, then TTree goes on and writes to disk a valid dataset with the data members for which there was enough information.

In the case of RNTuple (see the attached reproducer), the same TStreamerInfo error appears, but an assertion is triggered and the application aborts:

Error in <TStreamerInfo::Build>: The class "MyEvent" is interpreted and for its data member "map" we do not have a dictionary for the collection "map<short,short>". Because of this, we will not be able to read or write this data member.
Fatal: 0 violated at line 1530 of `/Users/vpadulan/Programs/rootproject/root/io/io/src/TGenCollectionProxy.cxx'
aborting
TGenCollectionProxy__VectorNext(void*, void const*) /Users/vpadulan/Programs/rootproject/root/io/io/src/TGenCollectionProxy.cxx:1530
ROOT::RProxiedCollectionField::RCollectionIterableOnce::RIterator::Advance() /Users/vpadulan/Programs/rootproject/root/tree/ntuple/inc/ROOT/RField/RFieldProxiedCollection.hxx:79
ROOT::RProxiedCollectionField::RCollectionIterableOnce::RIterator::RIterator(ROOT::RProxiedCollectionField::RCollectionIterableOnce const&, void*) /Users/vpadulan/Programs/rootproject/root/tree/ntuple/inc/ROOT/RField/RFieldProxiedCollection.hxx:89
ROOT::RProxiedCollectionField::RCollectionIterableOnce::RIterator::RIterator(ROOT::RProxiedCollectionField::RCollectionIterableOnce const&, void*) /Users/vpadulan/Programs/rootproject/root/tree/ntuple/inc/ROOT/RField/RFieldProxiedCollection.hxx:89
ROOT::RProxiedCollectionField::RCollectionIterableOnce::begin() /Users/vpadulan/Programs/rootproject/root/tree/ntuple/inc/ROOT/RField/RFieldProxiedCollection.hxx:119
ROOT::RProxiedCollectionField::AppendImpl(void const*) /Users/vpadulan/Programs/rootproject/root/tree/ntuple/src/RFieldMeta.cxx:1108
ROOT::RFieldBase::Append(void const*) /Users/vpadulan/Programs/rootproject/root/tree/ntuple/src/RFieldBase.cxx:749
ROOT::RFieldBase::CallAppendOn(ROOT::RFieldBase&, void const*) /Users/vpadulan/Programs/rootproject/root/tree/ntuple/inc/ROOT/RFieldBase.hxx:490
ROOT::RClassField::AppendImpl(void const*) /Users/vpadulan/Programs/rootproject/root/tree/ntuple/src/RFieldMeta.cxx:368
 ROOT::RFieldBase::Append(void const*) /Users/vpadulan/Programs/rootproject/root/tree/ntuple/src/RFieldBase.cxx:749
 /Users/vpadulan/Programs/rootproject/rootbuild/rdf-awkward-snapshot-distrdf-debug-conda-awkward-dev/include/ROOT/RFieldBase.hxx:812
/Users/vpadulan/Programs/rootproject/rootbuild/rdf-awkward-snapshot-distrdf-debug-conda-awkward-dev/include/ROOT/REntry.hxx:108
void ROOT::RNTupleFillContext::FillNoFlushImpl<ROOT::REntry>(ROOT::REntry&, ROOT::RNTupleFillStatus&) /Users/vpadulan/Programs/rootproject/rootbuild/rdf-awkward-snapshot-distrdf-debug-conda-awkward-dev/include/ROOT/RNTupleFillContext.hxx:94
unsigned long ROOT::RNTupleFillContext::FillImpl<ROOT::REntry>(ROOT::REntry&) /Users/vpadulan/Programs/rootproject/rootbuild/rdf-awkward-snapshot-distrdf-debug-conda-awkward-dev/include/ROOT/RNTupleFillContext.hxx:108
ROOT::RNTupleFillContext::Fill(ROOT::REntry&) /Users/vpadulan/Programs/rootproject/rootbuild/rdf-awkward-snapshot-distrdf-debug-conda-awkward-dev/include/ROOT/RNTupleFillContext.hxx:132
ROOT::RNTupleWriter::Fill() /Users/vpadulan/Programs/rootproject/rootbuild/rdf-awkward-snapshot-distrdf-debug-conda-awkward-dev/include/ROOT/RNTupleWriter.hxx:188

Reproducer

#include <ROOT/RFieldBase.hxx>
#include <ROOT/RNTupleModel.hxx>
#include <ROOT/RNTupleWriter.hxx>
#include <ROOT/RNTupleReader.hxx>
#include <TInterpreter.h>
#include <iostream>

void write_rntuple(const char *datasetname, const char *filename)
{
    auto model = ROOT::RNTupleModel::Create();
    model->AddField(ROOT::RFieldBase::Create("evt", "MyEvent").Unwrap());
    auto writer = ROOT::RNTupleWriter::Recreate(std::move(model), datasetname, filename);
    writer->Fill();
}

void read_rntuple(const char *datasetname, const char *filename)
{
    auto reader = ROOT::RNTupleReader::Open(datasetname, filename);
    reader->PrintInfo();
    reader->Show(0);
}

void generate_class()
{
    gInterpreter->Declare(R"(
        struct MyEvent{
            int evtId{42};
            std::vector<int> charge{1, -1, 1};
            std::vector<float> pt{5, 10, 15};
            std::vector<float> eta{0, 1, 2};
            std::map<short, short> map{{1, 1}, {2, 2}, {3, 3}};
        };
        )");
}

int main()
{
    auto ntpl_name{"ntpl"};
    auto ntpl_file{"ntpl.root"};
    generate_class();

    std::cout << "Storing RNTuple with RNTuple\n\n";
    write_rntuple(ntpl_name, ntpl_file);
    std::cout << "Reading RNTuple with RNTuple\n";
    read_rntuple(ntpl_name, ntpl_file);
}

ROOT version

master

Installation method

Build from source

Operating system

MacOS

Additional context

No response

[jblomer]

Related #21274

[jblomer]

I am not sure if we want to address this. I would argue it is correct to not write a partial type. If, in a different context, a dictionary for the type appears, having it on disk with missing members is confusing.

[vepadulano]

I am not sure if we want to address this. I would argue it is correct to not write a partial type. If, in a different context, a dictionary for the type appears, having it on disk with missing members is confusing.

I believe deciding not to have the same behaviour as TTree is a valid choice, but it would be good to address the behaviour perhaps by throwing an error instead.