Add automatic rustls crypto provider initialization

.github/copilot-instructions.md

@@ -0,0 +1,182 @@
+# GitHub Copilot Custom Instructions for OneIO
+
+## Project Overview
+OneIO is a Rust library providing unified IO operations for reading and writing compressed files from local and remote sources with both synchronous and asynchronous support.
+
+## Code Quality Standards
+
+### Before Committing - Always Run:
+1. **Format code**: `cargo fmt`
+2. **Run linter**: `cargo clippy --all-features` and fix all warnings
+3. **Update README if lib.rs docs changed**: `cargo readme > README.md`
+4. **Update CHANGELOG.md**: Add entries under `[Unreleased]` section
+5. **Run tests**: `cargo test --all-features`
+
+### Formatting Rules
+- Always run `cargo fmt` before completing any task
+- Fix all clippy warnings with `cargo clippy --all-features -- -D warnings`
+- No trailing whitespace
+- Follow Rust standard formatting conventions
+
+### Documentation Requirements
+- When modifying `src/lib.rs` documentation, always regenerate README:
+  ```bash
+  cargo readme > README.md
+  ```
+- Keep documentation examples up-to-date
+- Add doc comments for all public APIs
+- Include usage examples in module-level documentation
+
+### Changelog Requirements
+- **Always update CHANGELOG.md** when making changes
+- Add entries under the `[Unreleased]` section
+- Use proper formatting:
+  - `### Added` - for new features
+  - `### Changed` - for changes in existing functionality
+  - `### Deprecated` - for soon-to-be removed features
+  - `### Removed` - for now removed features
+  - `### Fixed` - for bug fixes
+  - `### Security` - for security fixes
+- Be specific and clear about what changed
+- Include context for why changes were made when relevant
+
+## Commit and PR Guidelines
+
+### Commit Messages
+- Use imperative mood: "Add feature" not "Added feature"
+- First line: concise summary (50 chars or less)
+- **NO EMOJIS** in commit messages or PR descriptions
+- Add blank line, then detailed explanation if needed
+- Reference issues when applicable (e.g., "Fixes #123")
+
+### Pull Requests
+- **NO EMOJIS** in PR titles or descriptions
+- Use clear, professional language
+- Include sections: Summary, Changes, Testing
+- List all integration points and breaking changes
+- Provide code examples for new features
+
+## Testing Requirements
+
+### Test with Multiple Feature Combinations
+```bash
+# Default features
+cargo test
+
+# All features
+cargo test --all-features
+
+# No default features
+cargo test --no-default-features
+
+# Specific feature combinations
+cargo test --features https,s3
+cargo test --features http,gz,bz
+```
+
+### Before Finalizing
+- Run `cargo clippy --all-features` and fix all issues
+- Verify all tests pass with different feature flags
+- Check documentation builds: `cargo doc --no-deps --all-features`
+- Run examples if relevant: `cargo run --example <name> --features <required>`
+
+## Feature Structure
+
+### Available Features
+- **Compression**: `gz`, `bz`, `lz`, `xz`, `zstd`
+- **Protocols**: `http`, `https`, `ftp`, `s3`
+- **TLS**: `rustls`, `native-tls` (mutually exclusive)
+- **Additional**: `async`, `json`, `digest`, `cli`
+
+### Feature Guidelines
+- Use `#[cfg(feature = "...")]` for feature-gated code
+- Test feature combinations to avoid conflicts
+- Document feature requirements in examples
+- Keep feature flags orthogonal when possible
+
+## Code Style Preferences
+
+### Error Handling
+- Use `OneIoError` enum for all errors
+- Provide clear error messages
+- Use `?` operator for error propagation
+- Add context to errors when relevant
+
+### Module Organization
+- Keep modules focused and cohesive
+- Use `pub(crate)` for internal APIs
+- Export public APIs through `lib.rs`
+- Group related functionality in submodules
+
+### Thread Safety
+- All public APIs should be thread-safe where applicable
+- Use proper synchronization primitives
+- Document thread-safety guarantees
+- Test concurrent usage patterns
+
+## CI/CD Structure
+
+### Workflow Jobs
+1. **format**: Fast formatting check (independent)
+2. **check**: Compilation with different features
+3. **clippy**: Linting with strict warnings
+4. **build**: Various feature combination builds
+5. **test**: Comprehensive test suite
+
+### CI Requirements
+- All jobs must pass before merge
+- Format check fails on `cargo fmt --check` errors
+- Clippy fails on any warnings (`-D warnings`)
+- Tests must pass with all feature combinations
+
+## Common Patterns
+
+### Adding New Features
+1. Add feature flag to `Cargo.toml`
+2. Implement feature-gated code with `#[cfg(feature = "...")]`
+3. Add tests for the feature
+4. Document in lib.rs and regenerate README
+5. Update CHANGELOG.md
+6. Add example if applicable
+
+### Modifying Public APIs
+1. Consider backward compatibility
+2. Update all call sites
+3. Update documentation and examples
+4. Add migration guide if breaking
+5. Update version in CHANGELOG.md
+
+### Adding Dependencies
+1. Use minimal feature flags
+2. Make dependencies optional when possible
+3. Document why the dependency is needed
+4. Test build without default features
+
+## Specific to OneIO
+
+### Crypto Provider Initialization
+- Always call `crypto::ensure_default_provider()` in HTTPS/S3/FTP paths
+- Prefer AWS-LC with fallback to ring
+- Make initialization idempotent and thread-safe
+- Handle "already installed" cases gracefully
+
+### Remote Operations
+- Use `get_protocol()` to detect URL schemes
+- Handle all error cases with clear messages
+- Support custom HTTP clients
+- Test with actual remote files when possible
+
+### Compression
+- Auto-detect compression from file extension
+- Support all declared compression formats
+- Test with actual compressed files
+- Handle decompression errors gracefully
+
+## Remember
+- Quality over speed - take time to get it right
+- Test thoroughly with different features
+- Document changes clearly
+- **Always run cargo fmt and cargo clippy before committing**
+- **Always update CHANGELOG.md with changes**
+- **No emojis in commits or PRs**
+- **Regenerate README.md when lib.rs docs change**

.github/workflows/rust.yml

@@ -14,16 +14,23 @@ env:
   CARGO_TERM_COLOR: always
 
 jobs:
-  # Check code formatting and linting first
-  check:
-    name: Check
+  # Check code formatting (fast, separate job)
+  format:
+    name: Format
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
 
       - name: Check formatting
         run: cargo fmt --check
 
+  # Check compilation with different features
+  check:
+    name: Check
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
       - name: Check with default features
         run: cargo check
 

CHANGELOG.md

@@ -2,6 +2,25 @@
 
 All notable changes to this project will be documented in this file.
 
+## [Unreleased]
+
+### Added
+- New `crypto` module with `ensure_default_provider()` helper function to initialize rustls crypto providers
+  - Automatically tries AWS-LC first, then falls back to ring
+  - Called automatically from all HTTPS/S3/FTP code paths
+  - Can be called explicitly at startup for better control over initialization
+  - Fully thread-safe and idempotent
+  - Prevents "Could not automatically determine the process-level CryptoProvider" panic
+- GitHub Copilot custom instructions file (`.github/copilot-instructions.md`) documenting development practices
+
+### Changed
+- All HTTPS, S3, and FTP operations now automatically initialize the rustls crypto provider on first use
+- Added comprehensive documentation and examples for crypto provider initialization
+- `rustls` feature now explicitly includes `dep:rustls_sys` dependency
+- Simplified redundant feature flag checks (since `https` → `rustls`, only check `rustls`)
+- Split CI workflow: formatting check now runs as separate independent job for faster feedback
+- Updated `remote_file_exists()` to use `ensure_default_provider()` helper
+
 ## v0.19.2 -- 2025-10-17
 
 ### Changed

Cargo.toml

@@ -47,7 +47,7 @@ serde_json = { version = "1.0", optional = true }
 
 # feature: s3
 rust-s3 = { version = "0.37", optional = true, default-features = false, features = [
-    "sync",
+    "sync-rustls-tls",
 ] }
 
 # feature: cli
@@ -103,6 +103,7 @@ native-tls = [
     "rust-s3?/sync-native-tls",
 ]
 rustls = [
+    "dep:rustls_sys",
     "reqwest?/rustls-tls",
     "suppaftp?/rustls",
     "rust-s3?/sync-rustls-tls"
@@ -135,6 +136,10 @@ required-features = ["s3"]
 name = "async_read"
 required-features = ["async"]
 
+[[example]]
+name = "test_crypto_provider"
+required-features = ["https"]
+
 [package.metadata.binstall]
 pkg-url = "{ repo }/releases/download/v{ version }/{ name }-{ target }.tar.gz"
 pkg-fmt = "tgz"

README.md

@@ -79,6 +79,11 @@ oneio = { version = "0.19", features = ["async"] }
 
 Environment: Set `ONEIO_ACCEPT_INVALID_CERTS=true` to accept invalid certificates.
 
+**Crypto Provider Initialization**: When using rustls features (`https`, `s3`, `ftp`), oneio
+automatically initializes the crypto provider (AWS-LC or ring) on first use. You can also
+initialize it explicitly at startup using [`crypto::ensure_default_provider()`] for better
+control over error handling.
+
 ### Usages
 
 #### Reading Files
@@ -254,6 +259,31 @@ if s3_exists("my-bucket", "path/to/file.txt")? {
 let objects = s3_list("my-bucket", "path/", Some("/".to_string()), false)?;
 ```
 
+### Crypto Provider Initialization (Rustls)
+
+When using HTTPS, S3, or FTP features with rustls, oneio automatically initializes
+a crypto provider (AWS-LC or ring) on first use. For more control, you can initialize
+it explicitly at startup:
+
+```rust
+use oneio;
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    // Initialize crypto provider explicitly at startup
+    oneio::crypto::ensure_default_provider()?;
+
+    // Now all HTTPS/S3/FTP operations will work
+    let content = oneio::read_to_string("https://example.com/data.txt")?;
+
+    Ok(())
+}
+```
+
+This is particularly useful in libraries or applications that want to:
+- Handle initialization errors early
+- Control when the provider is set up
+- Make the dependency on crypto providers explicit
+
 ### Error Handling
 
 Three error types in v0.19:

examples/test_crypto_provider.rs

@@ -0,0 +1,20 @@
+//! Example demonstrating crypto provider initialization.
+//!
+//! This example shows how to explicitly initialize the crypto provider
+//! before making HTTPS requests. While oneio now initializes the provider
+//! automatically, you can still call it explicitly for clarity or to handle
+//! initialization errors early.
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    // Initialize crypto provider explicitly
+    oneio::crypto::ensure_default_provider()?;
+    println!("✓ Crypto provider initialized successfully!");
+
+    // Test HTTPS download - crypto provider is already set up
+    println!("\nDownloading test file via HTTPS...");
+    let content = oneio::read_to_string("https://spaces.bgpkit.org/oneio/test_data.txt")?;
+    println!("✓ Downloaded content:\n{}", content.trim());
+
+    println!("\n✓ All operations completed successfully!");
+    Ok(())
+}

src/lib.rs

@@ -73,6 +73,11 @@ oneio = { version = "0.19", features = ["async"] }
 
 Environment: Set `ONEIO_ACCEPT_INVALID_CERTS=true` to accept invalid certificates.
 
+**Crypto Provider Initialization**: When using rustls features (`https`, `s3`, `ftp`), oneio
+automatically initializes the crypto provider (AWS-LC or ring) on first use. You can also
+initialize it explicitly at startup using [`crypto::ensure_default_provider()`] for better
+control over error handling.
+
 ## Usages
 
 ### Reading Files
@@ -255,6 +260,31 @@ let objects = s3_list("my-bucket", "path/", Some("/".to_string()), false)?;
 # Ok::<(), Box<dyn std::error::Error>>(())
 ```
 
+## Crypto Provider Initialization (Rustls)
+
+When using HTTPS, S3, or FTP features with rustls, oneio automatically initializes
+a crypto provider (AWS-LC or ring) on first use. For more control, you can initialize
+it explicitly at startup:
+
+```rust,ignore
+use oneio;
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    // Initialize crypto provider explicitly at startup
+    oneio::crypto::ensure_default_provider()?;
+
+    // Now all HTTPS/S3/FTP operations will work
+    let content = oneio::read_to_string("https://example.com/data.txt")?;
+
+    Ok(())
+}
+```
+
+This is particularly useful in libraries or applications that want to:
+- Handle initialization errors early
+- Control when the provider is set up
+- Make the dependency on crypto providers explicit
+
 ## Error Handling
 
 Three error types in v0.19:
@@ -281,6 +311,11 @@ mod oneio;
 
 pub use error::OneIoError;
 
+#[cfg(feature = "rustls")]
+pub mod crypto {
+    //! Crypto provider initialization for rustls.
+    pub use crate::oneio::crypto::*;
+}
 #[cfg(feature = "digest")]
 pub use crate::oneio::digest::*;
 #[cfg(any(feature = "http", feature = "ftp"))]