PHOENIX-7759: Preserve buffered mutations when batch size limit is exceeded

[haridsv]

Summary

• Adds new configuration property phoenix.mutate.preserveOnLimitExceeded that changes mutation limit behavior, which can be overridden at the connection level via connection properties.
• When enabled, limit checks occur BEFORE state modification, preserving existing buffered mutations
• Introduces MutationLimitReachedException (for executeUpdate) and MutationLimitBatchException (for executeBatch) to signal recoverable limit conditions
• Allows applications to commit buffered mutations and continue processing instead of losing state, which facilitates "dynamic sizing" for the batch.

Test plan

• Added integration tests for row count and byte size limits with preserve mode
• Added tests for executeBatch with autoCommit on/off
• Added tests for edge cases: row merge, conflicting rows (ON DUPLICATE KEY), constructor limit exceeded
• Added test for legacy behavior (preserveOnLimitExceeded=false)
• Verified estimatedSize tracking for conflicting rows

JIRA: PHOENIX-7759

[sanjeet006py]

Was this an existing gap which will be fixed with your change as I don't see size of conflicting rows being accounted right now?

[sanjeet006py]

Seems like this also was an existing gap that the serialized bytes of ON DUPLICATE KEY was not accounted in mutation size along with accounting for merged indexes length.
I wonder if it can turn into backward incompatible change how size of same mutations can increase due to fixing of existing gaps. That means some previous use cases which were succeeding could fail now.
Should we tie up this new behaviour with this change i.e. only account for new size calculations when this feature is enabled?

[haridsv]

@sanjeet006py Good eye on catching the gaps! Yes, both would be backwards incompatible but I didn't make it conditional because both are pretty corner case scenarios that are very unlikely to be of practical use cases. In fact, it can be considered a good idea if some remote use case is slipping through larger mutations than intended, as they would get caught and will make the developers to make adjustments, after all these can be considered as bug fixes. However, I am open to make them conditional if anyone has a solid counter argument.

[haridsv]

The spotless errors are outside the changed files in the PR and they are coming even though I already ran spotless.

[haridsv]

Most of checkstyle errors are also in existing lines. Only the below are in the changed lines:

./phoenix-core-client/src/main/java/org/apache/phoenix/execute/MutationState.java:2469:            colValuesSizeDiff -= (entry.getKey().getEstimatedSize() + oldValue.length);�:31: Unnecessary parentheses around assignment right-hand side. [UnnecessaryParentheses]
./phoenix-core-client/src/main/java/org/apache/phoenix/query/QueryServices.java:100:  public static final String PRESERVE_MUTATIONS_ON_LIMIT_EXCEEDED_ATTRIB =�:17: Redundant 'final' modifier. [RedundantModifier]

The second one is following the existing pattern so I will leave it as is for consistency. I will fix the first one.

[haridsv]

There us ibe junit test failure:

[org.apache.phoenix.end2end.LongViewIndexDisabledBaseRowKeyMatcherIT.testViewsWithExtendedPK](https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-2371/1/testReport/org.apache.phoenix.end2end/LongViewIndexDisabledBaseRowKeyMatcherIT/testViewsWithExtendedPK/)
 Error Details
isMultiTenant(false), extendPK(true), partition(640), tenant(3), rowId(6120), pkInfo(ID1,ID2,ID3), testInfo(DECIMAL,DECIMAL,INTEGER), sortInfo(DESC,DESC,DESC)
 Stack Trace
java.lang.AssertionError: isMultiTenant(false), extendPK(true), partition(640), tenant(3), rowId(6120), pkInfo(ID1,ID2,ID3), testInfo(DECIMAL,DECIMAL,INTEGER), sortInfo(DESC,DESC,DESC)
	at org.junit.Assert.fail(Assert.java:89)
	at org.apache.phoenix.end2end.BaseRowKeyMatcherTestIT.createViewHierarchy(BaseRowKeyMatcherTestIT.java:983)
	at org.apache.phoenix.end2end.BaseRowKeyMatcherTestIT.createViewHierarchy(BaseRowKeyMatcherTestIT.java:917)
	at org.apache.phoenix.end2end.BaseRowKeyMatcherTestIT.testViewsWithExtendedPK(BaseRowKeyMatcherTestIT.java:795)
...
 Standard Error
...
2026-02-09T22:47:18,744 ERROR [main] end2end.BaseRowKeyMatcherTestIT(981): 1
java.lang.ArrayIndexOutOfBoundsException: 1
	at org.apache.phoenix.schema.types.PDataType.getDecimalPrecisionAndScale(PDataType.java:767)
	at org.apache.phoenix.schema.types.PDecimal.isSizeCompatible(PDecimal.java:335)
	at org.apache.phoenix.schema.PTableImpl.newKey(PTableImpl.java:1251)
	at org.apache.phoenix.compile.UpsertCompiler.setValues(UpsertCompiler.java:181)
	at org.apache.phoenix.compile.UpsertCompiler.access$500(UpsertCompiler.java:125)
	at org.apache.phoenix.compile.UpsertCompiler$UpsertValuesMutationPlan.execute(UpsertCompiler.java:1393)
	at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:665)
	at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:612)
	at org.apache.phoenix.call.CallRunner.run(CallRunner.java:52)
	at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:612)
	at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:593)
	at org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:182)
	at org.apache.phoenix.end2end.BaseRowKeyMatcherTestIT.upsertTenantViewRows(BaseRowKeyMatcherTestIT.java:428)
	at org.apache.phoenix.end2end.BaseRowKeyMatcherTestIT.createViewHierarchy(BaseRowKeyMatcherTestIT.java:972)
	at org.apache.phoenix.end2end.BaseRowKeyMatcherTestIT.createViewHierarchy(BaseRowKeyMatcherTestIT.java:917)
	at org.apache.phoenix.end2end.BaseRowKeyMatcherTestIT.testViewsWithExtendedPK(BaseRowKeyMatcherTestIT.java:795)
...

This can't be related to my change and must be a flapper.