Skip to content

PHOENIX-7860 Fix cell serialization and deserialization when CP cells are merged#2483

Open
tkhurana wants to merge 1 commit into
apache:PHOENIX-7562-feature-newfrom
tkhurana:PHOENIX-7860
Open

PHOENIX-7860 Fix cell serialization and deserialization when CP cells are merged#2483
tkhurana wants to merge 1 commit into
apache:PHOENIX-7562-feature-newfrom
tkhurana:PHOENIX-7860

Conversation

@tkhurana
Copy link
Copy Markdown
Contributor

@tkhurana tkhurana commented May 19, 2026

Summary

When a coprocessor adds cells to a mutation (local index, conditional TTL, ON DUPLICATE KEY UPDATE), HBase's checkAndMergeCPMutations merges those cells into the data mutation. The resulting mutation can mix row keys and cell types (e.g., a Put containing DeleteColumn/DeleteFamily cells, or cells with different row keys for local indexes). Replication previously appended the merged mutation as-is and the codec relied on the mutation type to drive cell-level deserialization, which round-tripped incorrectly and lost cell-type fidelity.

This change:

  • IndexRegionObserver:
    • Splits a merged mutation back into individual Put/Delete mutations grouped by (row key, put-vs-delete) before appending to the replication log, using the same algorithm as HBase's ReplicationSink.
    • Refactors replicateEditOnWALRestore to use the same splitter rather than its own row-grouping loop.
  • LogFileCodec:
    • Persists the per-cell type byte and reconstructs cells via KeyValue on decode, preserving DeleteColumn/DeleteFamily/DeleteFamilyVersion distinctions.
  • LogFileRecord:
    • Removes the now-redundant MutationType variants for delete subtypes; a Delete is just DELETE because cell types are encoded per-cell.
  • ReplicationLogGroup:
    • Keys the INSTANCES cache by serverName + haGroupName so multiple region servers sharing a JVM (mini-cluster) don't collide.
  • Tests:
    • LogFileCodecTest: covers mixed cell types within one Delete, and round-trip for all 5 cell type bytes (Put, Delete, DeleteColumn, DeleteFamily, DeleteFamilyVersion).
    • ReplicationLogGroupIT: replays log files on cluster 2 and asserts cross-cluster cell-level equality for the indexed-table, multi-CF, ON DUPLICATE KEY UPDATE, and conditional-TTL paths.
    • LogFileFormatTest: relaxes a brittle exception-message assertion.

Test plan

  • mvn test -pl phoenix-core -Dtest=LogFileCodecTest
  • mvn test -pl phoenix-core -Dtest=LogFileFormatTest
  • mvn verify -pl phoenix-core -Dit.test=ReplicationLogGroupIT
  • mvn spotless:check

@tkhurana tkhurana requested a review from apurtell May 19, 2026 20:19
@apurtell
Copy link
Copy Markdown
Contributor

apurtell commented May 19, 2026

@tkhurana Did you include other changes here by accident ?

There are some changes to how ReplicationLogGroups are keyed, but that's nothing to do with cell ser-de. Seems like a real bug but should have its own PR.

Some unrelated cleanups in IndexRegionObserver ?

@tkhurana
Copy link
Copy Markdown
Contributor Author

@tkhurana Did you include other changes here by accident ?

There are some changes to how ReplicationLogGroups are keyed, but that's nothing to do with cell ser-de. Seems like a real bug but should have its own PR.

@apurtell It is a test bug which surfaced when you spin multiple RS in the same JVM. I didn't file a separate PR for it.

@apurtell
Copy link
Copy Markdown
Contributor

It should probably be its own PR or else it's a hidden change, effectively.

Copy link
Copy Markdown
Contributor

@apurtell apurtell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes related to how ReplicationLogGroups are keyed should be in their own PR.

There are some unrelated cleanups in IndexRegionObserver also but not consequential.

I'm approving this because the changes to cell ser-de look good to me and can be merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants