-
Notifications
You must be signed in to change notification settings - Fork 250
Description
Background
I've been working on a deterministic testing simulator. Some libraries provide a way to provide your own random number generator or allow for seeding. Unfortunately most don't provide any way to seed their RNGs. That said, almost all libraries use getrandom (mostly transitively through rand::rng()) in order to generate random numbers when they want nondeterminism.
What is your motivation?
Many libraries don't provide a way to seed their RNGs which make writing deterministic tests difficult.
What type of application is this? (E.g. cryptography, game, numerical simulation)
emulation/testing
Feature request
It would be awesome if there could be an "deterministic-testing" opt-in backend to make getrandom use a seeded backend for the purpose of writing deterministic tests. It would only be deterministic when both the backend was opted-into and a feature flag was set (such as "unsafe-deterministic"). Having it be an opt-in backend ensures that a downstream dependency cannot unilaterally turn the tests deterministic while having a feature flag for it allows enabling the feature only in dev-dependencies so that only tests are deterministic.
While I understand the potential security ramifications of making a crate whose sole job is extreme nondeterminism deterministic, I believe hiding such a feature behind both the opt-in backend configuration and the feature flag should be sufficient to reduce the risk of abuse of such a feature.
Alternatives Considered
I tried to use the custom getrandom opt-in backend in .cargo/config.toml coupled with a seeded RNG, but that doesn't let me easily exclusively use the custom backend only when testing (See rust-lang/cargo#8170). This is not ideal because I'd like my production code to use the OS syscall for randomness and I would ideally not have to run all my tests with RUSTFLAGS='--cfg getrandom_backend="custom"'. Ideally I'd default to the normal behavior of getrandom when running outside of a test, but I cannot do so (without forking getrandom) because any call into getrandom within my custom backend would end up in a recursive loop of calls into my custom backend.
I also tried using #[cfg(test)] instead of the feature flag to allow for different behavior for tests/for production, but unfortunately that doesn't work as #[cfg(test)] is not transitively set across dependencies and cargo exposes no way for a downstream dependency to know whether the build is for a test or not.
One last alternative is providing a third-party crates-io patch but that's incredibly brittle to transitive dependencies upgrading to newer versions of getrandom.