Improve performance of encrypted sqlite DB

[kegsay]

As seen in matrix-org/complement-crypto#122 which times out due to how slow it gets with encryption enabled via .Passphrase being set on the client_builder.

Example timings:

• TestAliceBobEncryptionWorks/{rust_hs1}|{rust_hs1} - this creates 2 clients, 1 room and Alice sends 1 message to Bob. It's the simplest E2EE test. 910ms without encrypted DB, 4.86s with.
• TestFallbackKeyIsUsedIfOneTimeKeysRunOut - this consumes all OTKs (blocking the client uploading more) then claims the fallback key, so it will cause the client to re-upload 50 OTKs. 2.52s without, 8.43s with.
• Overall: 3m29s to run without, 9m28s to run with.

This is 3-5x worse.

This means I can't realistically enable encrypted DBs in tests, which is desirable in order to match EX.

[poljar]

I think that this is mostly due to our PBKDF usage in the default case when we encrypt the store. The usage of a random key is possible with our primitive itself:

matrix-rust-sdk/crates/matrix-sdk-store-encryption/src/lib.rs

Lines 151 to 182 in 7a85b7a

	/// Encrypt the store cipher using the given key and export it.
	///
	/// This method can be used to persist the `StoreCipher` in an unencrypted
	/// key/value store in a safe manner.
	///
	/// The `StoreCipher` can later on be restored using
	/// [`StoreCipher::import_with_key`].
	///
	/// # Arguments
	///
	/// * `key` - The 32-byte key to be used to encrypt the store cipher. It's
	/// recommended to use a freshly and securely generated random key.
	///
	/// # Examples
	///
	/// ```rust,no_run
	/// # let example = || {
	/// use matrix_sdk_store_encryption::StoreCipher;
	/// use serde_json::json;
	///
	/// let store_cipher = StoreCipher::new()?;
	///
	/// // Export the store cipher and persist it in your key/value store
	/// let export = store_cipher.export_with_key(&[0u8; 32]);
	///
	/// // Save the export in your key/value store.
	/// # anyhow::Ok(()) };
	/// ```
	pub fn export_with_key(&self, key: &[u8; 32]) -> Result<Vec<u8>, Error> {
	let store_cipher = self.export_helper(key, KdfInfo::None)?;
	Ok(rmp_serde::to_vec_named(&store_cipher).expect("Can't serialize the store cipher"))
	}

But it's not easily exposed in the ClientBuilder like the passphrase based mechanism is:

matrix-rust-sdk/crates/matrix-sdk/src/client/builder.rs

Lines 216 to 228 in 7a85b7a

	/// Set up the store configuration for a SQLite store.
	#[cfg(feature = "sqlite")]
	pub fn sqlite_store(
	mut self,
	path: impl AsRef<std::path::Path>,
	passphrase: Option<&str>,
	) -> Self {
	self.store_config = BuilderStoreConfig::Sqlite {
	path: path.as_ref().to_owned(),
	passphrase: passphrase.map(ToOwned::to_owned),
	};
	self
	}

We would need a new method to let us create stores with a random key instead.

[multisme]

Hi, can you extend more on the need here? Are you talking about adding to BuilderStoreConfig a Option field that would contain the encryption key?

[poljar]

Yes, instead of requiring the usage of a passphrase that is used to derive the encryption key, we should let users provide an encryption key themselves.

This avoids the slow PBKDF step.