Saving `dataclass` objects field-wise.

[yfukai]

I would like to propose an option to save dataclass objects into field-wise attributes, and automatically loads into the original object using the code like this:

import importlib

def resolve_class(path: str):
    module_name, qualname = path.split(":")
    module = importlib.import_module(module_name)
    obj = module
    for part in qualname.split("."):
        obj = getattr(obj, part)
    return obj

cls = resolve_class("mypkg.models:MyClass")
instance = cls(...)

This option can offer two benefits:

• Enhances compatibility between different environments. (Current cloudpickle "can only be used to send objects between the exact same version of Python." from the doc)
• By default, one can search nodes or edges by their attributes of dataclasses (Usecase: Splitting bbox and masks #119).

Prerequisite

• Built-in and user-defined metadata store API #192 will be useful to store the class information for the composite fields.
• Explicit dtype for the node and edge attrs? #193 will be useful to ensure that the graph knows the dtype at node_attr_key etc.

[JoOkuma]

Hi @yfukai,

You have some good points. I'll try to address #192 and #193 this week.
Geff already supports metadata, so #192 should be straightforward. In the SQL case, I'll create a new metadata table.

Is your use case for pickling mainly related to SQL? Today, when saving geffs, we are avoiding saving pickles
Ideally, when we have better typing support (#193), we would avoid pickling the data.

Do you have a use case where you are storing "objects" besides ndarray (not considering the Mask, which is part of the main library)?

[yfukai]

Hi thanks @JoOkuma for your comment!

Is your use case for pickling mainly related to SQL?

The current implementation pickles the objects only for the SQLGraph, but I believe the second point (searching nodes or edges by their attributes of dataclasses) is relevant also for the RustWorkXGraph!

Do you have a use case where you are storing "objects" besides ndarray (not considering the Mask, which is part of the main library)?

Not sure how much common they are, but I can imagine people who are tracking animals (or cells) might want to have their Pose objects with annotations for different landmarks. Also, usage of unfixed-length list for point annotations or polygons may be common. About the GEFF, maybe we can raise a warning for unsupported data types so that the users can convert them to whatever supported.

[yfukai]

I was thinking we should find a way to use "pl.Struct" rather than sticking to complicated class inference/recovery. In SQLGraph, those structs can be saved as JSON datatype, which can be searched field-wise or split into field-wise columns.