Enable external payloads by default

[simvlad]

What changed?

Enable server support for external payloads stats by default

Why?

https://docs.temporal.io/external-storage is in pre-release and available to anyone who wants to try it out from SDK. The server support for collecting the stats is gated by history.externalPayloadsEnabled dynamic config. Enable it by default, as otherwise the executions stats won't be available.

How did you test it?

• built
• run locally and tested manually
• covered by existing tests
• added new unit test(s)
• added new functional test(s)

Potential risks

The change enables the collection of workflow execution stats for external storage, but it is only when the users try out the pre-release feature. So in the case of problems, they can turn the dynamic config off.

[yycptt]

Is it safe to downgrade to 1.30 with this feature flag enabled?

[simvlad]

Yes, it just enables the payload stats on the workflow execution. Not sure though, why tests are failing

[yycptt]

Yes, it just enables the payload stats on the workflow execution. Not sure though, why tests are failing

🤔 maybe rebase?

[simvlad]

I think it's a bug in #9664, where search attributes in registerTestSearchAttributes are registered for each clusters independently, but the code in addSearchAttributes is non-deterministic. I asked Alan to fix

[awln-temporal]

I think this is an actual test failure, I don't see any successful functional test runs for this PR. Also mentioned offline, but I think it's due to this verification https://github.com/temporalio/temporal/blob/main/tests/xdc/visibility_test.go#L178, since the external payload predefined search attributes are then included here, making the count 5, not 3

[simvlad]

Yeah, looks like this is an actual failure here, although I still think the test is flaky.