[CALCITE-7393] Support RelDataTypeDigest

[zhuwenzhuang]

Use RelDataTypeDigest to reduce composite type's digest memory and digest relative operation's latency.
test case: HepPlannerTest.testLargeTypeDigest
This is the first part of latency/memory optimization of large plan.

[xiedeyantu]

Thank you for your contribution! I have a few suggestions:
First, please ensure consistency between the Jira title, commit message, and PR title.
Second, please do not arbitrarily change or delete public/protected methods or variables in existing classes (they can be marked as deprecated).
Third, any errors in the CI need to be checked again.

[mihaibudiu]

This could be two interfaces, in case Digest is useful somewhere else.

[zhuwenzhuang]

I noticed that the hashCode caching in RelDigest and RexCall already addresses some of the hashCode caching/comparison latency issues. Could you please clarify what you mean by “be two interfaces”?

[mihaibudiu]

interface HasDigestString {
   String getDigestString()
}

interface RelDataTypeDigest extends HasDigestString {
   RelDataType getType();
}

Other classes may implement HasDigestString.

[mihaibudiu]

I think you'll need to leave the old implementations around too, they may break users who expect them to exist.

[zhuwenzhuang]

If we allow user to implement equals/hashCode for nested type, cache for hashCode and pointer compare for deepEquals will not work for them.
This situation is similiar to https://issues.apache.org/jira/browse/CALCITE-3786

[mihaibudiu]

Changing protected fields is a breaking change, which has to be reflected in the semantic versioning. This would require a 2.0 release.

[mihaibudiu]

On the other hand, I don't know what else you can do to make the digest lazy. Maybe other people can weigh on this dilemma. This is why you should probably file a JIRA issue just for this topic and discuss the design there.

[zhuwenzhuang]

Maybe we can allow user to set String digest to define their own digest. otherwise, we use the RelDataTypeDigest

[xiedeyantu]

My suggestion is that some protected/public variables and methods must still be guaranteed to work properly. For example, this
"String digest" might eventually call
"toString" from
"RelDataTypeDigest" to get the value? Because these variables are exposed to users, and someone might currently be using them, we need to provide a compatibility fix. If you think reassigning the original variable would go against your design intention (e.g., it might increase memory usage instead of reducing it), then you need to declare that this is a breaking change.

[mihaibudiu]

I don't know how to review this; the change is reasonable in itself, but I don't know if this is an acceptable breaking change. I would appreciate comments from people who have dealt with similar breaking changes in the past.

[xiedeyantu]

I think this change has a fairly wide impact. I'm wondering if it's possible to control the new/old logic by introducing configuration options. To be honest, I can't judge at this point how the current changes will affect future compatibility. Although your changes may be good, the community probably values ​​compatibility more.

[zhuwenzhuang]

@xiedeyantu
Thanks, I think use CalciteSystemProperties to control compatibility is a good idea.
The patch is updated base on 'calcite.disable.generate.type.digest.string', and a benchmark TypeDigestBenchmark added.

[sonarqubecloud]

Quality Gate passed

Issues
13 New issues
0 Accepted issues

Measures
0 Security Hotspots
28.4% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud