fix: Identity validation, storage singleton, and iOS handshake compatibility #827

umwelt · 2026-01-20T13:52:41Z

Summary

Fix identity ID extraction: extract from DID hex instead of computing Blake3(DID)
Add zhtp_client_identity_to_handshake_json() FFI for iOS handshake compatibility
Use global storage singleton to fix sled lock contention
Add TTY detection for non-interactive systemd startup

Changes

lib-identity

Fix new(), from_observed_handshake(), new_from_seed(), new_external(), from_serialized()
ID is now extracted from DID (did:zhtp:{id_hex}) instead of recomputed

lib-client

New FFI: zhtp_client_identity_to_handshake_json() - exports identity in lib-network format
iOS must use this for handshakes (structured PublicKey with dilithium_pk, kyber_pk, key_id)

zhtp server

api/server.rs, health_check.rs, metrics.rs - use get_global_storage() instead of opening sled directly
did_startup.rs - auto-generate identity when no TTY (systemd services)

Test plan

iOS handshake with new zhtp_client_identity_to_handshake_json()
Server starts without sled lock errors
Systemd service auto-generates identity

- Add zhtp_client_identity_get_kyber_public_key() and zhtp_client_identity_get_created_at() C exports - Add zhtp_client_free_string() and zhtp_client_free_bytes() convenience aliases - Add ___chkstk_darwin() stub for iOS assembly compatibility - Update build.rs to use environment variable check for uniffi feature - Remove unused cc dependency from Cargo.toml These changes enable iOS app to use lib-client for post-quantum cryptography, ensuring signature compatibility with server (pqcrypto-dilithium backend).

- Add Kyber public key and device ID C FFI exports for iOS - Add function aliases (zhtp_client_free_string, zhtp_client_free_bytes) - Fix build.rs feature detection using env var - Add iOS Darwin stack check stub (___chkstk_darwin) - Reorganize Cargo.toml dependencies for clarity

…computing The identity_id field should be extracted from the DID (format: did:zhtp:HEXID) rather than recomputed via Blake3(DID). The DID is already derived from the identity_id during identity creation, so validation must extract and compare the hex component, not recompute it.

…ibility lib-identity: - Fix ID extraction: extract from DID hex instead of computing Blake3(DID) - Fixes new(), from_observed_handshake(), new_from_seed(), new_external(), from_serialized() lib-client: - Add zhtp_client_identity_to_handshake_json() FFI for lib-network format - iOS must use this for handshakes instead of serialize() zhtp server: - Use global storage singleton instead of opening sled DB per-component - Fixes sled lock contention in api/server.rs, health_check.rs, metrics.rs - Add TTY detection for non-interactive startup (systemd services)

…elds The uhp-ffi crate calls ZhtpIdentity::from_serialized() which requires: - id, did, identity_type, public_key (structured), node_id - device_node_ids, primary_device, dao_member_id, ownership_proof Added all required and optional fields with appropriate defaults.

Copilot

Pull request overview

This PR addresses three main technical issues in the ZHTP protocol implementation: identity ID extraction, storage singleton initialization, and iOS FFI compatibility. The changes span identity validation logic, server storage access patterns, and mobile client bindings.

Changes:

Fixed identity ID extraction to parse from DID hex string instead of recomputing Blake3(DID), correcting the identity validation logic across multiple methods
Migrated server components to use global storage singleton to prevent sled database lock contention issues
Added iOS-compatible FFI functions including handshake JSON serialization and BIP39 seed phrase generation

Reviewed changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
lib-identity/src/identity/lib_identity.rs	Corrects ID extraction from DID in `new()`, `from_observed_handshake()`, `new_from_seed()`, `new_external()`, and `from_serialized()`
lib-identity/src/identity/manager.rs	Adds `register_external_citizen_identity()` for server-side wallet creation with full citizenship benefits
zhtp/src/api/server.rs	Migrates to use `get_global_storage()` instead of creating new sled database instances
zhtp/src/monitoring/metrics.rs	Updates storage metrics collection to use global storage singleton
zhtp/src/monitoring/health_check.rs	Updates storage health checks to use global storage singleton
zhtp/src/runtime/did_startup.rs	Adds TTY detection for automatic identity generation in systemd services
zhtp/src/api/handlers/identity/mod.rs	Updates client identity registration to create 3 wallets with seed phrases returned in response
lib-client/src/lib.rs	Adds FFI functions: `zhtp_client_identity_get_seed_phrase()`, `zhtp_client_identity_to_handshake_json()`, `zhtp_client_sign_uhp_challenge()`, `zhtp_client_identity_get_kyber_public_key()`, and `___chkstk_darwin()` stub
lib-client/src/identity.rs	Implements BIP39 mnemonic generation from 32-byte entropy with checksum calculation
lib-client/src/bip39_wordlist.rs	Adds standard 2048-word BIP39 English wordlist
lib-client/build.rs	Changes UniFFI feature detection from `#[cfg]` to runtime environment variable check
lib-client/Cargo.toml	Adds `sha2` dependency and moves `uniffi` from optional section
Cargo.lock	Records `sha2` dependency addition

Copilot · 2026-01-20T14:01:24Z

zhtp/src/api/handlers/identity/mod.rs

+            "wallet_seed_phrases": {
+                "primary": citizenship_result.wallet_seed_phrases.primary_wallet_seeds.words.join(" "),
+                "ubi": citizenship_result.wallet_seed_phrases.ubi_wallet_seeds.words.join(" "),
+                "savings": citizenship_result.wallet_seed_phrases.savings_wallet_seeds.words.join(" ")
+            }


SECURITY CRITICAL: Wallet seed phrases are being returned in the API response and transmitted over the network. This is a serious security vulnerability. Seed phrases are highly sensitive cryptographic material that should NEVER leave the device where they're generated. Even over HTTPS, they could be logged by proxies, debugging tools, or application logs.

The proper approach is:

Have clients generate their own wallets locally

Only transmit public keys/wallet addresses to the server

Clients store seed phrases securely in their local keychain

If the server must create wallets (which is questionable), the seed phrases should be encrypted with the client's public key before transmission, and the client should immediately store them securely and clear them from memory.

Copilot · 2026-01-20T14:01:25Z

lib-client/src/identity.rs

+    #[test]
+    fn test_get_seed_phrase_word_count() {
+        let mut identity = generate_identity("test-device".into()).unwrap();
+        identity.master_seed = vec![0u8; 32];
+
+        let phrase = get_seed_phrase(&identity).unwrap();
+        assert_eq!(phrase.split_whitespace().count(), 24);
+    }


The test only validates that 24 words are produced, but doesn't verify the correctness of the BIP39 encoding. Consider adding a test with a known test vector (entropy + expected mnemonic) from the BIP39 specification to ensure the implementation is correct and compatible with standard BIP39 implementations.

Copilot · 2026-01-20T14:01:25Z

lib-client/build.rs

-    {
+    println!("cargo:rerun-if-changed=uniffi/zhtp_client.udl");
+
+    if std::env::var_os("CARGO_FEATURE_UNIFFI").is_some() {


The change from checking #[cfg(feature = "uniffi")] to std::env::var_os("CARGO_FEATURE_UNIFFI") is functionally equivalent, but the cfg attribute approach is more idiomatic Rust and is checked at compile-time rather than runtime. The #[cfg(feature = "uniffi")] approach is preferred unless there's a specific reason for runtime detection. This change doesn't break functionality but moves away from the standard Rust pattern.

Suggested change

if std::env::var_os("CARGO_FEATURE_UNIFFI").is_some() {

#[cfg(feature = "uniffi")]

{

Copilot · 2026-01-20T14:01:25Z

lib-identity/src/identity/manager.rs

+        // Create 3 wallets WITH seed phrases
+        let (primary_wallet_id, primary_seed_phrase) = identity.wallet_manager.create_wallet_with_seed_phrase(
+            WalletType::Primary,
+            "Primary Wallet".to_string(),
+            None
+        ).await?;
+
+        let (ubi_wallet_id, ubi_seed_phrase) = identity.wallet_manager.create_wallet_with_seed_phrase(
+            WalletType::UBI,
+            "UBI Wallet".to_string(),
+            None
+        ).await?;
+
+        let (savings_wallet_id, savings_seed_phrase) = identity.wallet_manager.create_wallet_with_seed_phrase(
+            WalletType::Savings,
+            "Savings Wallet".to_string(),
+            None
+        ).await?;
+


The function creates server-side wallets with seed phrases for an external identity where keys were generated client-side. This creates a security architecture inconsistency: the identity keys are client-side (secure), but the wallet keys are server-side (less secure). This mixed trust model is confusing and potentially problematic. Consider either: (1) having clients generate all keys including wallets, or (2) documenting clearly why wallets must be server-generated and implementing additional security measures (e.g., encrypting seed phrases before storage/transmission).

Copilot · 2026-01-20T14:01:26Z

zhtp/src/monitoring/health_check.rs

+            Err(_) => {
+                // Global storage not initialized - fallback to system disk usage
+                use sysinfo::Disks;
+                let disks = Disks::new_with_refreshed_list();
+
+                if let Some(disk) = disks.iter().next() {
+                    Some(StorageStats {
+                        total_storage: disk.total_space(),
+                        used_storage: disk.total_space() - disk.available_space(),
+                        dht_nodes: 1,
+                    })
+                } else {
+                    None
                }
-            },
-            Err(_) => None
+            }


Similar to metrics.rs, the error handling here silently falls back when global storage is unavailable. The comment on line 762 says "Global storage not initialized" which is helpful, but consider logging this at least once during startup to aid in debugging. Silent fallbacks can make it difficult to diagnose why storage metrics show unexpected values.

Copilot · 2026-01-20T14:01:27Z

lib-client/src/lib.rs

 #[cfg(feature = "uniffi")]
 uniffi::setup_scaffolding!();

+// iOS compatibility: provide stub for ___chkstk_darwin (PQC assembly references this)


The empty ___chkstk_darwin stub could be problematic. This symbol is used by the Windows MSVC compiler for stack probing on Darwin platforms. Providing an empty implementation might cause stack overflow issues if the PQC assembly code actually needs this functionality. This should either: (1) have a proper implementation that performs stack checking, or (2) be conditionally compiled only for iOS targets where it's actually needed, with a comment explaining why it's safe to be empty.

Suggested change

// iOS compatibility: provide stub for ___chkstk_darwin (PQC assembly references this)

// iOS compatibility: provide stub for ___chkstk_darwin.

//

// Some precompiled PQC object files originating from MSVC toolchains reference

// the `___chkstk_darwin` symbol when linked into iOS targets. On Rust/LLVM

// iOS/aarch64 builds we do not rely on MSVC-style stack probing, and our PQC

// assembly does not actually perform dynamic stack checks via this symbol.

//

// Restricting this stub to iOS/aarch64 ensures we satisfy the symbol reference

// only where it is known to be benign, and we do not accidentally override the

// platform's own stack probing mechanism on other targets.

#[cfg(all(target_os = "ios", target_arch = "aarch64"))]

Copilot · 2026-01-20T14:01:28Z

lib-client/src/lib.rs

+/// Serialize identity to JSON format compatible with ZhtpIdentity::from_serialized().
+///
+/// This creates the FULL format expected by lib-identity for UHP handshakes.
+/// The uhp-ffi crate calls ZhtpIdentity::from_serialized() which requires all these fields.
+///
+/// Caller must free with `zhtp_client_string_free`.
+#[no_mangle]
+pub extern "C" fn zhtp_client_identity_to_handshake_json(handle: *const IdentityHandle) -> *mut std::ffi::c_char {
+    if handle.is_null() {
+        return std::ptr::null_mut();
+    }
+    let identity = unsafe { &(*handle).inner };
+
+    // Compute key_id = Blake3(dilithium_public_key)
+    let key_id = crypto::Blake3::hash(&identity.public_key);
+
+    // Extract identity ID from DID (format: "did:zhtp:{id_hex}")


The documentation clearly explains the JSON format expected by lib-network, which is helpful. However, there's a subtle issue: the comment says key_id is 32 bytes (line 353), but it's actually a Blake3 hash which is indeed 32 bytes. The documentation should clarify that this is a Blake3 hash of the Dilithium public key, matching the format expected by lib-network's NodeIdentity.

Copilot · 2026-01-20T14:01:28Z

lib-client/Cargo.toml

+# UniFFI (for mobile bindings)
+uniffi = { version = "0.25", optional = true }


Moving the UniFFI dependency to always be included (line 40) instead of being optional increases the dependency tree for all builds, even when the uniffi feature is not enabled. The build-dependencies change on line 59 correctly keeps uniffi with default-features = false to minimize build-time dependencies. However, making the runtime uniffi dependency non-optional may not be necessary. Consider keeping it optional unless there's a specific reason it needs to be always available.

Copilot · 2026-01-20T14:01:28Z

lib-client/src/identity.rs

+        identity.master_seed = vec![0u8; 32];
+
+        let phrase = get_seed_phrase(&identity).unwrap();
+        assert_eq!(phrase.split_whitespace().count(), 24);


The test uses a zero-filled seed (vec![0u8; 32]) which is not a realistic test case. For a proper test, use a known entropy value with the expected BIP39 mnemonic output from the specification to verify correctness. The BIP39 specification includes official test vectors that should be used here.

Copilot · 2026-01-20T14:01:28Z

zhtp/src/runtime/did_startup.rs

-                save_to_keystore(&keystore_path, &result)
-                    .map_err(|e| anyhow!("Failed to save identity to keystore: {}", e))?;
-                println!("✓ Identity saved to {:?}", keystore_path);
+        let is_interactive = atty::is(atty::Stream::Stdin);


The atty crate is unmaintained and archived. Consider using the is-terminal crate instead, which is the recommended replacement. You can use std::io::IsTerminal trait available in Rust 1.70+ or the is-terminal crate for broader compatibility.

Suggested change

let is_interactive = atty::is(atty::Stream::Stdin);

let is_interactive = io::stdin().is_terminal();

The lib-identity NodeId struct has 3 fields: bytes, creation_nonce, network_genesis. The JSON serialization was sending just a raw Vec<u8> which caused deserialization error: "invalid type: integer 19, expected an array of length 32" Now serializes node_id and device_node_ids with the full NodeId struct format expected by ZhtpIdentity::from_serialized(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

When a new peer authenticates via UHP handshake, their identity was registered as "observed" but without any wallet funding. This meant they couldn't participate in paid operations like domain registration. Now auto_register_peer_identity() also: 1. Creates a primary wallet for the new identity 2. Funds it with 5000 ZHTP welcome bonus 3. Registers the wallet in blockchain.wallet_registry This ensures all authenticated peers can immediately participate in the network economy. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

…zed() After thorough analysis of lib-identity's from_serialized() function: 1. identity_type: Changed from "Human" to "Device" - Human type REQUIRES age and jurisdiction fields (lines 965-966) - Mobile clients don't have these, causing runtime errors - Device type uses default [0u8; 32] for credential hash 2. ownership_proof: Fixed to match ZkProof struct format - Added: proof_system, proof_data, public_inputs, verification_key, plonky2_proof, proof - Removed: proof_type, verified, timestamp (wrong struct) 3. key_id: Converted to [u8; 32] array format for proper serialization 4. reputation: Set to 100 (Device default) instead of 0 This should be the complete fix - no more incremental field fixes. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The iOS uhp-ffi module requires access to: - Dilithium secret key (for signing) - Kyber secret key (for key exchange) - Master seed (for derivation) Added FFI functions: - zhtp_client_identity_get_dilithium_secret_key() - zhtp_client_identity_get_kyber_secret_key() - zhtp_client_identity_get_master_seed() SECURITY: These keys stay on-device. They are passed from lib-client to uhp-ffi in the same process space, never transmitted over network. iOS needs to call these and pass the bytes to uhp-ffi's handshake. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

…wallet - lib-crypto: Change key generation and signing from Dilithium2 to Dilithium5 - generation.rs: Use dilithium5::keypair() (4896 byte keys) - derivation.rs: Use dilithium5::keypair() for seed-based generation - operations.rs: sign() and sign_dilithium() now use dilithium5::SecretKey - Fixes iOS handshake failure due to key size mismatch (was trying to parse 4896-byte Dilithium5 keys as 2420-byte Dilithium2) - lib-identity: Actually credit 5000 ZHTP welcome bonus to Primary wallet - onboard_citizen_identity(): Credit wallet after provide_welcome_bonus() - register_external_citizen_identity(): Same fix for client-side registration - Previously only logged bonus but never updated wallet.balance Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

…auto-detection UBI Distribution (lib-blockchain): - Add ubi_registry and ubi_blocks to Blockchain struct - Add UbiRegistryEntry struct tracking identity, wallet, daily_amount, last_payout - process_automatic_ubi_distribution(): runs every block, credits ~33 ZHTP daily to citizens due for payout (last_payout + 8640 blocks = 1 day) - register_for_ubi(): registers citizen when identity is created - Auto-register citizens for UBI during identity registration when identity_type is "verified_citizen", "citizen", or "external_citizen" - Handles remainder accumulation (1000/30 = 33 r10) for exact monthly distribution Dilithium Signing (lib-crypto): - sign() now auto-detects Dilithium2 vs Dilithium5 based on secret key size - 4896 bytes → Dilithium5 (new mobile client keys) - 2528 bytes → Dilithium2 (legacy server keys) - Matches verify_signature() which already auto-detects by public key size Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Add dilithium_sign/dilithium_verify auto-detection functions based on key size - Fix DILITHIUM2_SECRETKEY_BYTES constant (2560, not 2528) - Add parse_did_to_identity_id helper to lib-identity/did - Fix identity ID mismatch bug in quic_handler (was hashing DID instead of extracting hex) - Fix identity ID mismatch bug in web4/domains.rs for owner/author lookups - Replace hardcoded Dilithium2/5 calls with auto-detection across codebase Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

umwelt added 5 commits January 19, 2026 22:40

Add BIP39 seed phrase export for client identity

1deae5d

Copilot AI review requested due to automatic review settings January 20, 2026 13:52

Copilot started reviewing on behalf of umwelt January 20, 2026 13:53 View session

Copilot AI reviewed Jan 20, 2026

View reviewed changes

umwelt and others added 7 commits January 20, 2026 14:27

umwelt merged commit 297ca7f into development Jan 20, 2026
5 of 6 checks passed

umwelt deleted the feat/ios-c-ffi-exports branch January 20, 2026 23:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix: Identity validation, storage singleton, and iOS handshake compatibility #827

fix: Identity validation, storage singleton, and iOS handshake compatibility #827

Uh oh!

umwelt commented Jan 20, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 20, 2026

Uh oh!

Copilot AI Jan 20, 2026

Uh oh!

Copilot AI Jan 20, 2026

Uh oh!

Copilot AI Jan 20, 2026

Uh oh!

Copilot AI Jan 20, 2026

Uh oh!

Copilot AI Jan 20, 2026

Uh oh!

Copilot AI Jan 20, 2026

Uh oh!

Copilot AI Jan 20, 2026

Uh oh!

Copilot AI Jan 20, 2026

Uh oh!

Copilot AI Jan 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	if std::env::var_os("CARGO_FEATURE_UNIFFI").is_some() {
	#[cfg(feature = "uniffi")]
	{

-// iOS compatibility: provide stub for ___chkstk_darwin (PQC assembly references this)
+// iOS compatibility: provide stub for ___chkstk_darwin.
+//
+// Some precompiled PQC object files originating from MSVC toolchains reference
+// the `___chkstk_darwin` symbol when linked into iOS targets. On Rust/LLVM
+// iOS/aarch64 builds we do not rely on MSVC-style stack probing, and our PQC
+// assembly does not actually perform dynamic stack checks via this symbol.
+//
+// Restricting this stub to iOS/aarch64 ensures we satisfy the symbol reference
+// only where it is known to be benign, and we do not accidentally override the
+// platform's own stack probing mechanism on other targets.
+#[cfg(all(target_os = "ios", target_arch = "aarch64"))]

		# UniFFI (for mobile bindings)
		uniffi = { version = "0.25", optional = true }

	let is_interactive = atty::is(atty::Stream::Stdin);
	let is_interactive = io::stdin().is_terminal();

fix: Identity validation, storage singleton, and iOS handshake compatibility #827

fix: Identity validation, storage singleton, and iOS handshake compatibility #827

Uh oh!

Conversation

umwelt commented Jan 20, 2026

Summary

Changes

lib-identity

lib-client

zhtp server

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants