Skip to content

Conversation

@dAdAbird
Copy link
Member

This is a WIP, and the main goal is to gather feedback on the storage of bigger (32-byte) keys.
The current _keys file format supposes 16-byte-long keys, and extending it would mean the migration for users. So I tried to keep the current format by splitting bigger (32-byte) keys into 16-byte blocks and storing them in two consecutive disk entries. The writes of such sequential entries happen in one go (as one write) to keep the atomicity of changes and not to end up in situations where the first part of the key is updated and then a crash happens, leaving the whole key in a broken state.
The code becomes more sophisticated, but it's a trade-off for the _keys format compatibility with the older versions.

It's a raw draft to test the idea.
What has to be improved:

  1. 32-byte keys work only for WAL internal keys.
  2. Rotation of 32-byte WAL keys doesn't work.
  3. Nor do the MGR 32 keys work.
  4. AES encryption currently works in one of two states - 16 or 32 bytes. AES contexts should be separated, and the appropriate one should be applied for decryption based on the key metadata (length) and for encryption based on the GUC option.
  5. FE tools that generate keys (pg_basebackup, anything else?) should have a flag.
  6. All GUC and FE tools changes are currently a bit of a mess and were made mainly to make the server run to test the idea.
  7. Changes in SMGR should be aligned with WAL keys. The current one is just to make the code compile.

So, yes, the main area I'd like your feedback on is changes in pg_tde_xlog_keys.c.

For later, proper PR, I'll split it into separate commits (or even PRs).

Store 32-byte keys split into multiple disk entries.
@codecov-commenter
Copy link

Codecov Report

❌ Patch coverage is 71.42857% with 26 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.71%. Comparing base (e24e386) to head (dc401a7).
⚠️ Report is 153 commits behind head on TDE_REL_17_STABLE.

❌ Your project status has failed because the head coverage (83.71%) is below the target coverage (90.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@                  Coverage Diff                  @@
##           TDE_REL_17_STABLE     #583      +/-   ##
=====================================================
- Coverage              84.20%   83.71%   -0.49%     
=====================================================
  Files                     25       25              
  Lines                   3247     3286      +39     
  Branches                 511      514       +3     
=====================================================
+ Hits                    2734     2751      +17     
- Misses                   400      421      +21     
- Partials                 113      114       +1     
Components Coverage Δ
access 83.76% <74.54%> (-0.97%) ⬇️
catalog 87.65% <100.00%> (ø)
common 77.77% <ø> (ø)
encryption 69.67% <58.82%> (-3.31%) ⬇️
keyring 73.21% <100.00%> (ø)
src 92.89% <61.53%> (-1.30%) ⬇️
smgr 96.53% <100.00%> (ø)
transam ∅ <ø> (∅)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@dAdAbird
Copy link
Member Author

Decided to go with changing the _keys format

@dAdAbird dAdAbird closed this Sep 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants