fix: skip action rows with extra_data exceeding MySQL TEXT column limit#371
Open
netcrafts wants to merge 1 commit into
Open
fix: skip action rows with extra_data exceeding MySQL TEXT column limit#371netcrafts wants to merge 1 commit into
netcrafts wants to merge 1 commit into
Conversation
When deeply-nested item NBT (e.g. a shulker box containing 27 beehives each with 3 named/potioned bees and long lore) is serialised, the resulting string can exceed 65,535 bytes — the MySQL TEXT column limit for extra_data. This caused the entire batchInsert to fail and the execute() retry wrapper to spin up to MAX_QUERY_RETRIES times on the same unresolvable batch, halting all logging on the server. Partition the action list before batchInsert: any row whose extraData exceeds MAX_EXTRA_DATA_BYTES (65,535) is dropped and a WARN is emitted with the action type, coordinates, world, and player name so operators can identify the trigger. The remaining rows are inserted normally.
Author
|
seems like this would resolve #285 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this change?
Prevents a permanent retry loop when logging actions whose serialized NBT exceeds the MySQL
TEXTcolumn limit (65,535 bytes) forextra_data.When a batch insert fails due to an oversized row, the offending action is now skipped with a
WARNlog entry. The remaining rows in the batch are inserted normally.Why?
The
extra_datacolumn is defined asTEXTin MySQL, which has a hard 65,535-byte limit. When a player picks up or interacts with a deeply nested item — such as a shulker box containing beehives with full bee entity NBT — the serialized NBT string can exceed this limit.Before this fix, the
batchInsertcall ininsertActions()would throw aBatchUpdateException: Data too long for column 'extra_data', and theexecute()retry wrapper would retry the same unresolvable batch up toMAX_QUERY_RETRIES(10) times before giving up, halting all logging on the server.This was reproduced live: a fully-packed shulker box of beehives produces ~42KB of
extra_datain vanilla, growing beyond 65,535 bytes when bees carry heavier entity NBT (custom names, active potion effects, hives with long lore). The overflow and retry loop were confirmed on a live server.How?
In
insertActions()inDatabaseManager.kt, the incoming action list is partitioned beforebatchInsert:extraDatastring length exceedsMAX_EXTRA_DATA_BYTES(65,535) is droppedWARNis emitted for each dropped row with the action type, coordinates, world, and player name so operators can identify the triggerbatchInsertas normalTruncating was ruled out as it would produce invalid NBT, breaking rollback and restore. Catching
BatchUpdateExceptionafter the fact was ruled out because Exposed does not identify which row caused the failure and the transaction is already in a failed state.Testing
Steps to reproduce the bug (unpatched):
CustomNametag (named with an anvil)BatchUpdateException: Data too long for column 'extra_data'in the log followed by repeatedTransaction attempt #N failedretries — logging halts entirelyVerified on patched build (live Fabric / Minecraft 1.26.1 server):
extra_datareached 76,996 chars (117% of the 65,535-byteTEXTlimit)WARN: Skipping action log: extra_data too large (76996 chars) for action block-break at [minecraft:overworld ...] by Netcrafts