|
| 1 | +# s2n-quic-tests |
| 2 | + |
| 3 | +This crate contains integration tests for the s2n-quic implementation. These tests verify the behavior of the QUIC protocol implementation across various scenarios and edge cases. |
| 4 | + |
| 5 | +## Test Organization |
| 6 | + |
| 7 | +The test organization in this crate is inspired by the approach used in the [rust-lang/cargo](https://github.com/rust-lang/cargo) repository. This approach differs from the typical Rust integration test organization described in the [Rust book](https://doc.rust-lang.org/book/ch11-03-test-organization.html) for the reason highlighted in the [Cargo book](https://doc.rust-lang.org/cargo/reference/cargo-targets.html#integration-tests): |
| 8 | + |
| 9 | +>Each integration test results in a separate executable binary, and cargo test will run them serially. In some cases this can be inefficient, as it can take longer to compile, and may not make full use of multiple CPUs when running the tests. If you have a lot of integration tests, you may want to consider creating a single integration test, and split the tests into multiple modules. |
| 10 | +
|
| 11 | +To further increase performance, the tests are contained within the `src` folder to avoid having the tests wait for compilation and linking of the intermediate `s2n-quic-tests` lib. |
| 12 | + |
| 13 | +### Platform-specific Tests |
| 14 | + |
| 15 | +Some tests in this crate are platform-specific, particularly those that depend on s2n-tls, which is only available on Unix systems. These tests are conditionally compiled using `cfg[unix]` attributes. For example: |
| 16 | + |
| 17 | +```rust |
| 18 | +#[cfg(unix)] |
| 19 | +mod resumption; |
| 20 | + |
| 21 | +#[cfg(not(target_os = "windows"))] |
| 22 | +mod mtls; |
| 23 | +``` |
| 24 | + |
| 25 | +This approach ensures that tests only run on platforms where their dependencies are available. The Cargo.toml file also includes platform-specific dependencies to support this. |
| 26 | + |
| 27 | +## Test Structure |
| 28 | + |
| 29 | +The test suite is organized into several categories: |
| 30 | + |
| 31 | +- **Basic Connectivity Tests**: Verify that clients and servers can establish connections and exchange data. |
| 32 | +- **Error Handling Tests**: Ensure proper handling of protocol errors, malformed packets, and connection failures. |
| 33 | +- **Network Pathology Tests**: Test behavior under various network conditions (latency, packet loss, reordering). |
| 34 | +- **Edge Case Tests**: Verify correct behavior in unusual or extreme scenarios. |
| 35 | + |
| 36 | +### Directory Structure |
| 37 | + |
| 38 | +``` |
| 39 | +src/ |
| 40 | +├── lib.rs # Contains common test utilities and setup functions |
| 41 | +├── recorder.rs # Event recording utilities |
| 42 | +├── tests.rs # Main test module that imports all tests |
| 43 | +└── tests/ # Contains all the test files |
| 44 | + ├── blackhole.rs # Individual test files |
| 45 | + ├── ... # Other test files |
| 46 | + └── snapshots/ # Snapshot files for tests |
| 47 | +``` |
| 48 | + |
| 49 | +## Running Tests |
| 50 | + |
| 51 | +### Running All Tests |
| 52 | + |
| 53 | +To run all tests in the crate: |
| 54 | + |
| 55 | +```bash |
| 56 | +cargo test |
| 57 | +``` |
| 58 | + |
| 59 | +### Running Specific Tests |
| 60 | + |
| 61 | +To run tests in a specific module: |
| 62 | + |
| 63 | +```bash |
| 64 | +cargo test -- <module_name> |
| 65 | +``` |
| 66 | + |
| 67 | +For example, to run the blackhole tests: |
| 68 | + |
| 69 | +```bash |
| 70 | +cargo test -- blackhole |
| 71 | +``` |
| 72 | + |
| 73 | +To run a specific test: |
| 74 | + |
| 75 | +```bash |
| 76 | +cargo test -- <module_name>::<test_name> |
| 77 | +``` |
| 78 | + |
| 79 | +For example: |
| 80 | + |
| 81 | +```bash |
| 82 | +cargo test -- blackhole::blackhole_success_test |
| 83 | +``` |
| 84 | + |
| 85 | +### Running Tests with Logging |
| 86 | + |
| 87 | +To enable trace logging during test execution: |
| 88 | + |
| 89 | +```bash |
| 90 | +cargo test -- --nocapture |
| 91 | +``` |
| 92 | + |
| 93 | +## Testing Philosophy |
| 94 | + |
| 95 | +The s2n-quic-tests crate follows several key principles: |
| 96 | + |
| 97 | +1. **Comprehensive Coverage**: Tests cover both normal operation paths and error handling scenarios. |
| 98 | + |
| 99 | +2. **Deterministic Testing**: Tests are designed to be deterministic and reproducible, avoiding flaky tests that could pass or fail randomly. |
| 100 | + |
| 101 | +3. **Isolation**: Each test runs in isolation to prevent interference between tests. |
| 102 | + |
| 103 | +4. **Realistic Scenarios**: Tests simulate real-world network conditions including packet loss, reordering, and latency. |
| 104 | + |
| 105 | +## Test Utilities |
| 106 | + |
| 107 | +### Network Simulation |
| 108 | + |
| 109 | +The test suite uses the [Bach](https://github.com/camshaft/bach) async simulation framework for simulating various network conditions: |
| 110 | + |
| 111 | +- Packet loss |
| 112 | +- Packet reordering |
| 113 | +- Network latency |
| 114 | +- Bandwidth limitations |
| 115 | +- Connection blackholes |
| 116 | + |
| 117 | +### Event Recording |
| 118 | + |
| 119 | +The `recorder.rs` module provides utilities for recording and verifying events during test execution, allowing tests to assert on the sequence and content of events. |
| 120 | + |
| 121 | +### Packet Interceptor |
| 122 | + |
| 123 | +The test suite uses a packet interceptor utility that allows tests to inspect, modify, or drop datagrams or modify remote addresses on datagrams as they flow between the client and server. |
| 124 | + |
| 125 | +Implement the `s2n_quic_core::packet::interceptor::Interceptor` trait and configure it in tests like this: |
| 126 | + |
| 127 | +```rust |
| 128 | +let client = Client::builder() |
| 129 | + .with_io(handle.builder().build().unwrap())? |
| 130 | + .with_packet_interceptor(interceptor)? |
| 131 | + .start()?; |
| 132 | +``` |
| 133 | + |
| 134 | +### Common Setup |
| 135 | + |
| 136 | +`src/lib.rs` provides common utilities for setting up test clients and servers with various configurations. |
| 137 | + |
| 138 | +## Contributing New Tests |
| 139 | + |
| 140 | +When adding new tests: |
| 141 | + |
| 142 | +1. Place the test in an appropriate module based on what it's testing |
| 143 | +2. Use the common setup utilities when possible |
| 144 | +3. Make tests deterministic and avoid dependencies on external systems |
| 145 | +4. Document the purpose of the test and any non-obvious aspects |
| 146 | +5. Ensure tests run in a reasonable amount of time |
| 147 | + |
| 148 | +## Integration with CI |
| 149 | + |
| 150 | +These tests are automatically run as part of the CI pipeline for s2n-quic. The CI configuration can be found in the `.github/workflows/ci.yml` file in the root of the repository. |
| 151 | + |
0 commit comments