feat(data_privacy)!: introduce data_privacy_core #427
Open
Vaiz wants to merge 1 commit into
Open
Conversation
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #427 +/- ##
========================================
- Coverage 100.0% 99.9% -0.1%
========================================
Files 286 286
Lines 22978 22853 -125
========================================
- Hits 22978 22841 -137
- Misses 0 12 +12 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Vaiz
force-pushed
the
issue-371-split-data-privacy
branch
8 times, most recently
from
52a562a to
1cf8061
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a new data_privacy_core crate that owns the core classification/redaction traits and types, and refactors the existing data_privacy crate to re-export those core APIs while keeping RedactionEngine and built-in redaction strategies. The change also updates dependent crates/macros to use the new Redactor-based interfaces and adjusts CI/scripts to include the new crate.
Changes:
- Add
crates/data_privacy_coreand move core types/traits (e.g.,Sensitive,DataClass,Redactor, redacted formatting traits) into it. - Refactor
data_privacyto re-exportdata_privacy_core, implementRedactorforRedactionEngine, and update redactor implementations/tests accordingly. - Update macros and
templated_uriintegration to useRedactor(&dyn Redactor) rather thanRedactionEnginein redacted formatting, and update snapshots/metadata.
Reviewed changes
Copilot reviewed 63 out of 64 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/run-examples.rs | Comment-only adjustment in excluded examples list. |
| scripts/publish-gh-release.rs | Comment-only adjustment in changelog parsing logic. |
| scripts/mutants.rs | Add data_privacy_core to mutation test grouping. |
| justfiles/basic.just | Comment-only adjustment in examples recipe docs. |
| crates/templated_uri/src/uri.rs | Switch redacted formatting to accept &dyn Redactor; adjust tests. |
| crates/templated_uri/src/path_and_query.rs | Switch redacted formatting to accept &dyn Redactor. |
| crates/templated_uri/Cargo.toml | Allow data_privacy_core::* types for external-types checking. |
| crates/templated_uri_macros_impl/src/struct_template.rs | Update generated RedactedDisplay signature to take &dyn Redactor. |
| crates/templated_uri_macros_impl/src/enum_template.rs | Update generated RedactedDisplay signature to take &dyn Redactor. |
| crates/templated_uri_macros_impl/src/snapshots/templated_uri_macros_impl__tests__templated_uri_impl.snap | Snapshot updates for new Redactor signature. |
| crates/templated_uri_macros_impl/src/snapshots/templated_uri_macros_impl__tests__templated_unredacted_uri_impl.snap | Snapshot updates for new Redactor signature. |
| crates/templated_uri_macros_impl/src/snapshots/templated_uri_macros_impl__tests__template_enum_impl.snap | Snapshot updates for new Redactor signature. |
| crates/templated_uri_macros_impl/src/snapshots/templated_uri_macros_impl__tests__query_param_is_kv_expansion.snap | Snapshot updates for new Redactor signature. |
| crates/templated_uri_macros_impl/src/snapshots/templated_uri_macros_impl__tests__optional_reference_field_codegen.snap | Snapshot updates for new Redactor signature. |
| crates/templated_uri_macros_impl/src/snapshots/templated_uri_macros_impl__tests__optional_field_with_unredacted_codegen.snap | Snapshot updates for new Redactor signature. |
| crates/templated_uri_macros_impl/src/snapshots/templated_uri_macros_impl__tests__optional_field_codegen.snap | Snapshot updates for new Redactor signature. |
| crates/templated_uri_macros_impl/src/snapshots/templated_uri_macros_impl__tests__field_level_unredacted.snap | Snapshot updates for new Redactor signature. |
| crates/data_privacy/tests/xxh3_redactor.rs | Add coverage for new would_redact behavior. |
| crates/data_privacy/tests/rapidhash_redactor.rs | Add coverage for new would_redact behavior. |
| crates/data_privacy/tests/redaction_engine.rs | Update to new Redactor-based redact call signature. |
| crates/data_privacy/tests/sensitive.rs | Add additional regression tests for redacted formatting and boundary behavior (with insta). |
| crates/data_privacy/src/sensitive.rs | Remove old Sensitive implementation (moved to core). |
| crates/data_privacy/src/redactors/xxh3_redactor.rs | Implement new would_redact method; adjust test coverage attrs. |
| crates/data_privacy/src/redactors/rapidhash_redactor.rs | Implement new would_redact method. |
| crates/data_privacy/src/redactors/simple_redactor.rs | Implement new would_redact method; import cleanup. |
| crates/data_privacy/src/redactors/redactor.rs | Remove old Redactor trait (moved to core). |
| crates/data_privacy/src/redactors/mod.rs | Remove local Redactor re-export; adjust tests for new trait contract. |
| crates/data_privacy/src/redaction_engine.rs | Make RedactionEngine implement Redactor; remove inherent redact/would_redact. |
| crates/data_privacy/src/redaction_engine_inner.rs | Update would_redact to delegate to the selected redactor/fallback; add tests. |
| crates/data_privacy/src/macros.rs | Remove local macro re-export module (moved to core re-exports). |
| crates/data_privacy/src/lib.rs | Re-export data_privacy_core APIs; update crate docs and module exports. |
| crates/data_privacy/README.md | Regenerated README reflecting the new data_privacy_core split and links. |
| crates/data_privacy/Cargo.toml | Bump version, add data_privacy_core, wire serde feature through, add insta dev-dep. |
| crates/data_privacy_macros/Cargo.toml | Version bump aligned with macro changes. |
| crates/data_privacy_macros_impl/tests/snapshots/taxonomy__success.snap | Snapshot updates to reference data_privacy_core paths. |
| crates/data_privacy_macros_impl/tests/snapshots/derive__redacted_display_unit.snap | Snapshot updates for &dyn Redactor and core paths. |
| crates/data_privacy_macros_impl/tests/snapshots/derive__redacted_display_single.snap | Snapshot updates for &dyn Redactor and core paths. |
| crates/data_privacy_macros_impl/tests/snapshots/derive__redacted_display_multiple.snap | Snapshot updates for &dyn Redactor and core paths. |
| crates/data_privacy_macros_impl/tests/snapshots/derive__redacted_display_multiple_named.snap | Snapshot updates for &dyn Redactor and core paths. |
| crates/data_privacy_macros_impl/tests/snapshots/derive__redacted_display_enum.snap | Snapshot metadata update. |
| crates/data_privacy_macros_impl/tests/snapshots/derive__redacted_debug_unit.snap | Snapshot updates for &dyn Redactor and core paths. |
| crates/data_privacy_macros_impl/tests/snapshots/derive__redacted_debug_single.snap | Snapshot updates for &dyn Redactor and core paths. |
| crates/data_privacy_macros_impl/tests/snapshots/derive__redacted_debug_multiple.snap | Snapshot updates for &dyn Redactor and core paths. |
| crates/data_privacy_macros_impl/tests/snapshots/derive__redacted_debug_multiple_named.snap | Snapshot updates for &dyn Redactor and core paths. |
| crates/data_privacy_macros_impl/tests/snapshots/derive__redacted_debug_enum.snap | Snapshot metadata update. |
| crates/data_privacy_macros_impl/tests/snapshots/classified__success.snap | Snapshot updates for core paths and &dyn Redactor usage. |
| crates/data_privacy_macros_impl/tests/snapshots/classified__success_named_field.snap | Snapshot updates for core paths and &dyn Redactor usage. |
| crates/data_privacy_macros_impl/src/taxonomy.rs | Use crate-path resolver so generated code targets data_privacy or data_privacy_core. |
| crates/data_privacy_macros_impl/src/lib.rs | Export new crate_path helper module. |
| crates/data_privacy_macros_impl/src/derive.rs | Update derive output to use resolved crate path and &dyn Redactor. |
| crates/data_privacy_macros_impl/src/crate_path.rs | New helper to resolve whether to generate ::data_privacy or ::data_privacy_core paths. |
| crates/data_privacy_macros_impl/src/classified.rs | Update classified macro output to use resolved crate path and &dyn Redactor. |
| crates/data_privacy_macros_impl/Cargo.toml | Add proc-macro-crate dependency; version bump. |
| crates/data_privacy_core/src/lib.rs | New core crate root exposing core traits/types and macro re-exports. |
| crates/data_privacy_core/src/macros.rs | New macro re-export module for core crate. |
| crates/data_privacy_core/src/redactor.rs | New Redactor trait definition (now in core). |
| crates/data_privacy_core/src/redacted.rs | Update redacted formatting traits to accept &dyn Redactor; add tests. |
| crates/data_privacy_core/src/sensitive.rs | New Sensitive implementation moved into core; add tests. |
| crates/data_privacy_core/src/data_class.rs | Restructure tests; separate serde-gated tests into a dedicated module. |
| crates/data_privacy_core/src/classified.rs | Update docs and minor trait doc tweaks for core crate. |
| crates/data_privacy_core/README.md | New crate README describing core responsibilities. |
| crates/data_privacy_core/Cargo.toml | New crate manifest with serde feature and dev-deps. |
| Cargo.toml | Add data_privacy_core and proc-macro-crate to workspace dependencies; bump versions. |
| Cargo.lock | Lockfile updates for new crate and new dependency versions. |
Comments suppressed due to low confidence (1)
crates/data_privacy_core/src/redacted.rs:106
- These tests use inline insta snapshots (e.g.
assert_snapshot!(result, @"...")). To align with the repository’s usual insta usage (file-backed.snapartifacts), consider converting these to external snapshots or simpleassert_eq!(the expected output is short).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Vaiz
force-pushed
the
issue-371-split-data-privacy
branch
from
0b9f929 to
4582bc7
Compare
ralfbiedert
approved these changes
May 22, 2026
| mod redacted; | ||
| // Re-export everything from data_privacy_core. | ||
| pub use data_privacy_core::{ | ||
| Classified, DataClass, IntoDataClass, RedactedDebug, RedactedDisplay, RedactedToString, Redactor, Sensitive, classified, taxonomy, |
Collaborator
There was a problem hiding this comment.
This didn't preserve the macro documentation.
| #[must_use] | ||
| #[cfg_attr(coverage_nightly, coverage(off))] | ||
| pub fn resolve() -> TokenStream { | ||
| use proc_macro_crate::{FoundCrate, crate_name}; |
Collaborator
There was a problem hiding this comment.
Not sure how I feel about this over just saying data_privacy is the way to go. It's a large crate, but it doesn't help lowering our dependency count, adding 8 more or so to our list (7 of them uniquely for this) for a use case we could say we just don't support.
Contributor
Author
There was a problem hiding this comment.
I'm actually expect people to use data_privacy_core crate in libraries if they define reusable types. for example, a library defines TenantId, it doesn't need to have a dependency on full data_privacy crate
Collaborator
There was a problem hiding this comment.
Like in the oxidizer case this would be an antipattern, at least when it comes to trying to make proc macros work. Random example, serde:
In crates that derive an implementation of Serialize or Deserialize, you must depend on the serde crate, not serde_core.
Contributor
Author
There was a problem hiding this comment.
maybe crate names are not the best,
what if we change data_privacy_core -> data_privacy and data_privacy -> data_privacy_redactors ?
in that case, we can avoid re-exporting macros all together
Collaborator
There was a problem hiding this comment.
That's a possible conversation, but that seems to conflict with the premise of this PR. I'm all for creating a _core crate of stable items to avoid breakage. In this example though you seem to imply that everything in data_privacy then is stable except redactors?
Now that I think about it, I think what would improve this PR is actually moving out the macros of _core. These are mainly crate-specific, and we can happily rename macro attributes or change behavior over crate versions, without breaking overall ecosystem composability.
Vaiz
force-pushed
the
issue-371-split-data-privacy
branch
from
4582bc7 to
0525c8f
Compare
Vaiz
force-pushed
the
issue-371-split-data-privacy
branch
from
0525c8f to
cba397a
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 70 out of 71 changed files in this pull request and generated 5 comments.
Comments suppressed due to low confidence (1)
crates/data_privacy_core/src/redacted.rs:106
- These unit tests use inline insta snapshots (e.g.,
@"..."). In this repo, snapshot tests are commonly file-backed via.snapfiles (for examplecrates/templated_uri_macros_impl/src/lib.rs:199-208). Consider converting to external snapshots to reduce noise in code diffs.
Comment on lines
173
to
176
| impl ::data_privacy::RedactedDisplay for #ident { | ||
| fn fmt(&self, engine: &::data_privacy::RedactionEngine, f: &mut ::std::fmt::Formatter) -> ::std::fmt::Result { | ||
| fn fmt(&self, engine: &dyn ::data_privacy::Redactor, f: &mut ::std::fmt::Formatter) -> ::std::fmt::Result { | ||
| #redacted_display | ||
| } |
Comment on lines
82
to
86
| impl ::data_privacy::RedactedDisplay for #ident { | ||
| fn fmt(&self, engine: &::data_privacy::RedactionEngine, f: &mut ::std::fmt::Formatter) -> ::std::fmt::Result { | ||
| fn fmt(&self, engine: &dyn ::data_privacy::Redactor, f: &mut ::std::fmt::Formatter) -> ::std::fmt::Result { | ||
| match self { | ||
| #(#variant_matches => ::data_privacy::RedactedDisplay::fmt(template_variant, engine, f)?),* | ||
| } |
Comment on lines
+10
to
+19
| struct PassthroughRedactor; | ||
|
|
||
| impl Redactor for PassthroughRedactor { | ||
| fn would_redact(&self, _data_class: &DataClass) -> bool { | ||
| true | ||
| } | ||
|
|
||
| fn redact(&self, _data_class: &DataClass, _value: &str, output: &mut dyn Write) -> core::fmt::Result { | ||
| write!(output, "[REDACTED]") | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Breaking changes in data_privacy crate can lead to cascading breaking changes across all consumer crates. This PR decouples redaction engines from basic data_privacy types so we can have more stable crate for classification, and have more freedom when we do breaking changes in redaction itself.