fix: resolve static linking failures with OpenSSL and duplicate symbols#438
fix: resolve static linking failures with OpenSSL and duplicate symbols#438dkropachev wants to merge 5 commits into
Conversation
97356ff to
3b91f17
Compare
|
I am not sure if I took correct way regarding |
No, it's not correct. The real problem in the first place is is that they are duplicated in the sources. I think a proper solution is to delete them from the Optionally, we could make the stubs present in |
Agreed.
Do we need to? Can't we use the stubs from integration.rs? |
I do propose using stubs from |
|
Sysexit or other abort method sounds reasonable. |
|
Guys, could you please tell me what to do with the pr |
|
Section "Fix: Duplicate symbol guards (src/testing_unimplemented.cpp)" should be reverted. The correct fix is to remove those symbols from testing_unimplemented.cpp |
wprzytula
left a comment
There was a problem hiding this comment.
Please improve commit messages. I can't read them.
5f4f201 to
621b995
Compare
What's the point of including this in the cover letter?
|
15203e1 to
391f3b5
Compare
Publish transitive dependencies for the static driver in generated pkg-config metadata. Teach the smoke app to consume static pkg-config output safely, and add package-level/static-integration checks that exercise the installed artifacts. This keeps installed static consumers aligned with the in-tree target. It also makes the Windows OpenSSL external-project path explicit during static integration builds.
The Rust integration layer already provides these test-only symbols when cpp_integration_testing is enabled. Drop the duplicate C++ stubs from testing_unimplemented.cpp so static integration builds link against a single definition.
Build the OPENSSL_LIBS environment variable with an explicit join instead of relying on PowerShell interpolation. This keeps openssl-sys from seeing a literal variable name in package and static-integration builds.
Split the static driver build from the static integration-test build and share the configured build directory setup. This keeps the static integration target focused on building and probing the test binary.
Add dedicated static-smoke make targets and call them directly from the package workflow. This keeps CI failures scoped to one smoke mode and avoids threading SCYLLA_SMOKE_BUILD_STATIC through the dynamic package checks.
391f3b5 to
015dd2c
Compare
|
@wprzytula , @Lorak-mmk , ping |
wprzytula
left a comment
There was a problem hiding this comment.
I have hard time grasping this PR, because some commits are bags of loosely-related changes. Please split into smaller commits, with proper exhaustive explanation of the changes in the commit messages.
Commit windows: assemble OPENSSL_LIBS from discovered libraries is a good example - there is a single change + an explanation why, what is done, and how it works now.
| @@ -384,9 +385,14 @@ test-app-package: .prepare-for-test | |||
| @pwsh.exe -NoProfile -Command "\ | |||
| \$$ErrorActionPreference = 'Stop'; \ | |||
| \$$composeFile = '${MAKEFILE_PATH}/../../tests/examples_cluster/docker-compose.yml'; \ | |||
| \$$smokePaths = @('$(SMOKE_TEST_INSTALL_PATH)\bin\scylla-cpp-driver-smoke-test.exe'); \ | |||
| \$$smokePaths = @(); \ | |||
| if ('$(SCYLLA_SMOKE_ONLY_STATIC)' -ne 'ON') { \ | |||
| \$$smokePaths += '$(SMOKE_TEST_INSTALL_PATH)\bin\scylla-cpp-driver-smoke-test.exe'; \ | |||
| }; \ | |||
| if ('$(SCYLLA_SMOKE_BUILD_STATIC)' -eq 'ON') { \ | |||
| \$$smokePaths += '$(SMOKE_TEST_INSTALL_PATH)\bin\scylla-cpp-driver-smoke-test-static.exe'; \ | |||
| } elseif ('$(SCYLLA_SMOKE_ONLY_STATIC)' -eq 'ON') { \ | |||
| throw 'SCYLLA_SMOKE_ONLY_STATIC requires SCYLLA_SMOKE_BUILD_STATIC=ON'; \ | |||
| }; \ | |||
There was a problem hiding this comment.
🔧 I dislike that building static smoke app is opt-in, but building dynamic smoke app is opt-out. Please use opt-in for both, e.g., SCYLLA_SMOKE_BUILD_STATIC and SCYLLA_SMOKE_BUILD_DYNAMIC.
| # TODO: Remove `build/libscylla-cpp-driver.*` after https://github.com/scylladb/cpp-rs-driver/issues/164 is fixed. | ||
| INTEGRATION_TEST_BIN: | | ||
| build/cassandra-integration-tests | ||
| build/libscylla-cpp-driver.* |
There was a problem hiding this comment.
🤔 You removed the TODO but didn't do what it was saying.
There was a problem hiding this comment.
Move it out ot a separate commit with explanation.
| @cd "${BUILD_DIR}" | ||
| cmake -DCASS_BUILD_INTEGRATION_TESTS=ON -DCMAKE_BUILD_TYPE=Release .. && (make -j 4 || make) | ||
|
|
||
| STATIC_BUILD_DIR := $(CURRENT_DIR)build-static |
There was a problem hiding this comment.
♻️ This should be put along other constants in this file (next to BUILD_DIR).
| build-static-integration-test-bin: | ||
| ifeq ($(OS_TYPE),windows) | ||
| $(MAKE) .package-build-prepare-windows | ||
| cmake -S . -B build-static -G "Visual Studio 17 2022" -A x64 -DCASS_BUILD_INTEGRATION_TESTS=ON -DCASS_USE_STATIC_LIBS=ON -DCMAKE_BUILD_TYPE=Release -DOPENSSL_VERSION=$(OPENSSL_WIN_VERSION) | ||
| @pwsh -NoProfile -Command "\ | ||
| $$useExternalOpenSSL = ((Select-String -Path 'build-static\\CMakeCache.txt' -Pattern '^SCYLLA_OPENSSL_EXTERNAL_PROJECT:BOOL=' -ErrorAction SilentlyContinue | Select-Object -First 1).Line -split '=', 2)[1]; \ | ||
| $$externalOpenSSLTarget = ((Select-String -Path 'build-static\\CMakeCache.txt' -Pattern '^SCYLLA_OPENSSL_EXTERNAL_TARGET:STRING=' -ErrorAction SilentlyContinue | Select-Object -First 1).Line -split '=', 2)[1]; \ | ||
| if ($$useExternalOpenSSL -eq 'ON' -and $$externalOpenSSLTarget) { \ | ||
| cmake --build build-static --config Release --target $$externalOpenSSLTarget; \ | ||
| if ($$LASTEXITCODE -ne 0) { exit $$LASTEXITCODE } \ | ||
| }; \ | ||
| $$opensslIncDir = ((Select-String -Path 'build-static\\CMakeCache.txt' -Pattern '^OPENSSL_INCLUDE_DIR:PATH=' | Select-Object -First 1).Line -split '=', 2)[1]; \ | ||
| $$opensslSslLibPath = ((Select-String -Path 'build-static\\CMakeCache.txt' -Pattern '^OPENSSL_SSL_LIBRARY(_RELEASE)?:FILEPATH=' -ErrorAction SilentlyContinue | Select-Object -First 1).Line -split '=', 2)[1]; \ | ||
| $$opensslCryptoLibPath = ((Select-String -Path 'build-static\\CMakeCache.txt' -Pattern '^OPENSSL_CRYPTO_LIBRARY(_RELEASE)?:FILEPATH=' -ErrorAction SilentlyContinue | Select-Object -First 1).Line -split '=', 2)[1]; \ | ||
| if ($$opensslIncDir) { \ | ||
| $$env:OPENSSL_DIR = (Split-Path $$opensslIncDir -Parent); \ | ||
| $$env:OPENSSL_INCLUDE_DIR = $$opensslIncDir; \ | ||
| if (-not $$opensslSslLibPath) { \ | ||
| $$opensslSslLibPath = (Get-ChildItem -Path $$env:OPENSSL_DIR -Recurse -Include 'libssl*.lib','ssleay32*.lib' -File -ErrorAction SilentlyContinue | Select-Object -First 1).FullName; \ | ||
| } \ | ||
| if (-not $$opensslCryptoLibPath) { \ | ||
| $$opensslCryptoLibPath = (Get-ChildItem -Path $$env:OPENSSL_DIR -Recurse -Include 'libcrypto*.lib','libeay32*.lib' -File -ErrorAction SilentlyContinue | Select-Object -First 1).FullName; \ | ||
| } \ | ||
| $$opensslLibPath = if ($$opensslSslLibPath) { $$opensslSslLibPath } elseif ($$opensslCryptoLibPath) { $$opensslCryptoLibPath } else { '' }; \ | ||
| if ($$opensslLibPath) { \ | ||
| $$env:OPENSSL_LIB_DIR = Split-Path $$opensslLibPath -Parent; \ | ||
| } else { \ | ||
| $$env:OPENSSL_LIB_DIR = Join-Path $$env:OPENSSL_DIR 'lib'; \ | ||
| } \ | ||
| if ($$opensslSslLibPath -and $$opensslCryptoLibPath) { \ | ||
| $$opensslSslLibName = [System.IO.Path]::GetFileNameWithoutExtension($$opensslSslLibPath); \ | ||
| $$opensslCryptoLibName = [System.IO.Path]::GetFileNameWithoutExtension($$opensslCryptoLibPath); \ | ||
| $$env:OPENSSL_LIBS = \"$$opensslSslLibName`:`$$opensslCryptoLibName\"; \ | ||
| } \ | ||
| }; \ | ||
| cmake --build build-static --config Release; \ | ||
| if ($$LASTEXITCODE -ne 0) { exit $$LASTEXITCODE } \ | ||
| " | ||
| build-static\Release\cassandra-integration-tests.exe --gtest_list_tests > NUL |
There was a problem hiding this comment.
🔧 This is some enormous gibberish, without a single comment explaining it.
| .package-configure: .package-build-prepare | ||
| ifeq ($(OS_TYPE),windows) | ||
| cmake -S . -B build -G "Visual Studio 17 2022" -A x64 -DCMAKE_BUILD_TYPE=$(CMAKE_BUILD_TYPE) -DOPENSSL_VERSION=1.1.1u $(CMAKE_FLAGS) | ||
| cmake -S . -B build -G "Visual Studio 17 2022" -A x64 -DCMAKE_BUILD_TYPE=$(CMAKE_BUILD_TYPE) -DOPENSSL_VERSION=$(OPENSSL_WIN_VERSION) $(CMAKE_FLAGS) |
There was a problem hiding this comment.
♻️ Extracting this should constitute a separate refactor commit, to reduce stress on the reviewers'/readers' context window.
| build-driver: .package-configure | ||
| ifeq ($(OS_TYPE),windows) | ||
| @pwsh -NoProfile -Command "$$opensslVersion = ((Select-String -Path 'build\\CMakeCache.txt' -Pattern '^OPENSSL_VERSION:STRING=' | Select-Object -First 1).Line -split '=', 2)[1]; $$opensslTarget = \"openssl-$${opensslVersion}-library\"; cmake --build build --config $(CMAKE_BUILD_TYPE) --target $$opensslTarget; $$env:OPENSSL_DIR = (Resolve-Path 'build\\libs\\openssl').Path; $$env:OPENSSL_INCLUDE_DIR = \"$$env:OPENSSL_DIR\\include\"; $$env:OPENSSL_LIB_DIR = \"$$env:OPENSSL_DIR\\lib\"; cmake --build build --config $(CMAKE_BUILD_TYPE)" | ||
| @pwsh -NoProfile -Command "\ | ||
| $$useExternalOpenSSL = ((Select-String -Path 'build\\CMakeCache.txt' -Pattern '^SCYLLA_OPENSSL_EXTERNAL_PROJECT:BOOL=' -ErrorAction SilentlyContinue | Select-Object -First 1).Line -split '=', 2)[1]; \ | ||
| $$externalOpenSSLTarget = ((Select-String -Path 'build\\CMakeCache.txt' -Pattern '^SCYLLA_OPENSSL_EXTERNAL_TARGET:STRING=' -ErrorAction SilentlyContinue | Select-Object -First 1).Line -split '=', 2)[1]; \ | ||
| if ($$useExternalOpenSSL -eq 'ON' -and $$externalOpenSSLTarget) { \ | ||
| cmake --build build --config $(CMAKE_BUILD_TYPE) --target $$externalOpenSSLTarget; \ | ||
| if ($$LASTEXITCODE -ne 0) { exit $$LASTEXITCODE } \ | ||
| }; \ | ||
| $$opensslIncDir = ((Select-String -Path 'build\\CMakeCache.txt' -Pattern '^OPENSSL_INCLUDE_DIR:PATH=' -ErrorAction SilentlyContinue | Select-Object -First 1).Line -split '=', 2)[1]; \ | ||
| $$opensslSslLibPath = ((Select-String -Path 'build\\CMakeCache.txt' -Pattern '^OPENSSL_SSL_LIBRARY(_RELEASE)?:FILEPATH=' -ErrorAction SilentlyContinue | Select-Object -First 1).Line -split '=', 2)[1]; \ | ||
| $$opensslCryptoLibPath = ((Select-String -Path 'build\\CMakeCache.txt' -Pattern '^OPENSSL_CRYPTO_LIBRARY(_RELEASE)?:FILEPATH=' -ErrorAction SilentlyContinue | Select-Object -First 1).Line -split '=', 2)[1]; \ | ||
| if ($$opensslIncDir) { \ | ||
| $$env:OPENSSL_DIR = (Split-Path $$opensslIncDir -Parent); \ | ||
| $$env:OPENSSL_INCLUDE_DIR = $$opensslIncDir; \ | ||
| if (-not $$opensslSslLibPath) { \ | ||
| $$opensslSslLibPath = (Get-ChildItem -Path $$env:OPENSSL_DIR -Recurse -Include 'libssl*.lib','ssleay32*.lib' -File -ErrorAction SilentlyContinue | Select-Object -First 1).FullName; \ | ||
| } \ | ||
| if (-not $$opensslCryptoLibPath) { \ | ||
| $$opensslCryptoLibPath = (Get-ChildItem -Path $$env:OPENSSL_DIR -Recurse -Include 'libcrypto*.lib','libeay32*.lib' -File -ErrorAction SilentlyContinue | Select-Object -First 1).FullName; \ | ||
| } \ | ||
| $$opensslLibPath = if ($$opensslSslLibPath) { $$opensslSslLibPath } elseif ($$opensslCryptoLibPath) { $$opensslCryptoLibPath } else { '' }; \ | ||
| if ($$opensslLibPath) { \ | ||
| $$env:OPENSSL_LIB_DIR = Split-Path $$opensslLibPath -Parent; \ | ||
| } else { \ | ||
| $$env:OPENSSL_LIB_DIR = Join-Path $$env:OPENSSL_DIR 'lib'; \ | ||
| } \ | ||
| if ($$opensslSslLibPath -and $$opensslCryptoLibPath) { \ | ||
| $$opensslSslLibName = [System.IO.Path]::GetFileNameWithoutExtension($$opensslSslLibPath); \ | ||
| $$opensslCryptoLibName = [System.IO.Path]::GetFileNameWithoutExtension($$opensslCryptoLibPath); \ | ||
| $$env:OPENSSL_LIBS = \"$$opensslSslLibName`:`$$opensslCryptoLibName\"; \ | ||
| } \ | ||
| }; \ | ||
| cmake --build build --config $(CMAKE_BUILD_TYPE); \ | ||
| if ($$LASTEXITCODE -ne 0) { exit $$LASTEXITCODE } \ | ||
| " |
There was a problem hiding this comment.
😕 Again enormous gibberish without explanation.
Is it about:
It also makes the Windows OpenSSL external-project path explicit during static integration builds.
?
If so, then PLEASE keep your commits logically separate. I struggle so much to understand the purpose and meaning of all these changes...
There was a problem hiding this comment.
I suppose it's feasible to keep Windows OpenSSL external-project path-related changes in a separate commit, as they are orthogonal to other changes.
| bash -c ' \ | ||
| set -euo pipefail; \ | ||
| dnf -y install make cmake gcc-c++ findutils rpm-build; \ | ||
| dnf -y install make cmake gcc-c++ findutils rpm-build zlib-devel; \ |
There was a problem hiding this comment.
❓ I don't see how adding this is related to this commit.
| if [ -z "$$smoke_bin" ]; then \ | ||
| echo "ERROR: smoke-test binary not found"; \ | ||
| exit 1; \ | ||
| fi; \ | ||
| "$$smoke_bin" 127.0.0.1 \ | ||
| "$$smoke_bin" 127.0.0.1; \ | ||
| if [ "$(SCYLLA_SMOKE_BUILD_STATIC)" = "ON" ]; then \ | ||
| smoke_static_bin=$$(find /usr -name "scylla-cpp-driver-smoke-test-static" -type f 2>/dev/null | head -1); \ | ||
| if [ -z "$$smoke_static_bin" ]; then \ | ||
| echo "ERROR: static smoke-test binary not found"; \ | ||
| exit 1; \ | ||
| fi; \ | ||
| "$$smoke_static_bin" 127.0.0.1; \ | ||
| fi \ |
There was a problem hiding this comment.
♻️ Instead of treating static build specially, I'd explicitly divide test targets into the dynamic and the static cases. This would be more explicit and intuitive.
| .vscode/ | ||
| .zed | ||
| build/ | ||
| build-static/ |
There was a problem hiding this comment.
🔧 This is added to .gitignore only in commit makefile: factor static build targets, but the first commit already mentions this path.
There was a problem hiding this comment.
Pull request overview
This PR fixes static-linking regressions (issue #164) by ensuring the driver’s static build publishes its transitive link requirements to installed consumers (pkg-config), aligning smoke/package checks with installed artifacts, and removing duplicate integration-test stubs that caused symbol clashes.
Changes:
- Publish static transitive dependencies via
scylla-cpp-driver_static.pcand wire the smoke app to consumepkg-config --staticmetadata (including macOS framework flags). - Remove duplicate C++ “unimplemented” stubs that are already provided by the Rust integration testing layer.
- Improve CI/build tooling for static linking (Windows OpenSSL env assembly, dedicated static smoke targets, and added static regression checks in workflows).
Reviewed changes
Copilot reviewed 12 out of 13 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tests/src/integration/CMakeLists.txt | Adjusts link order for integration tests and adds libuv ExternalProject dependency handling. |
| src/testing_unimplemented.cpp | Removes duplicate stub symbols to prevent duplicate-definition/link issues. |
| scylla-rust-wrapper/scylla-cpp-driver_static.pc.in | Switches static pkg-config metadata to use computed Requires.private / Libs.private. |
| scylla-rust-wrapper/CMakeLists.txt | Adds transitive link deps for the static imported target and populates pkg-config substitution variables. |
| packaging/smoke-test-app/Makefile | Extends package test runner to optionally run static smoke binaries and cleans up static artifacts. |
| packaging/smoke-test-app/CMakeLists.txt | Adds robust static pkg-config flag resolution/parsing to build a fully-static smoke binary. |
| Makefile | Introduces dedicated static integration build targets, improves Windows OpenSSL env handling, and adds static smoke/package targets. |
| cmake/FindOPENSSL.cmake | Expands Windows/MSVC OpenSSL search paths to cover arch/RT-mode subdirs. |
| cmake/ExternalProject-OpenSSL.cmake | Exposes cache metadata indicating when OpenSSL is built via ExternalProject and its target name. |
| cmake/Dependencies.cmake | Initializes OpenSSL ExternalProject indicator variables for consumers that inspect the cache. |
| .gitignore | Ignores the new build-static/ directory. |
| .github/workflows/build-lint-and-test.yml | Adds a static-link build regression step for issue #164. |
| .github/workflows/build-cpack-packages.yml | Adds static smoke verification and static-link regression steps across package workflows. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Summary
Fix the static-linking regressions behind
#164at the installed-package boundary, not just for the in-tree integration binary. The branch now publishes the static driver's transitive link requirements through pkg-config, teaches the package smoke app to consume that metadata safely, removes duplicate testing stubs from the C++ side, and keeps the package CI coverage readable with dedicated static-smoke targets.Fixes: #164
What Changed
scylla-cpp-driver_static.pcsrc/testing_unimplemented.cppthat are already provided by the Rust integration layer duringcpp_integration_testingOPENSSL_LIBSassemblyOPENSSL_LIBSwith an explicit join soopenssl-sysreceives real library names in package and static-integration buildsSCYLLA_SMOKE_BUILD_STATIC=ONthrough CI YAMLCommit Structure
c1eeee8static linking: support installed consumers and package checks61d6d5ftesting: remove duplicate static-integration stubs1dd2538windows: assemble OPENSSL_LIBS from discovered libraries08cec82ci: split static package smoke checks into dedicated targets5f4f201gitignore: ignore build_staticTesting
cmake -S . -B /tmp/cpp-rs-driver-cmake-check -G Ninja -DCMAKE_BUILD_TYPE=Releasemake checkmake run-test-unitcmake -S . -B /tmp/cpp-rs-driver-install-build -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/tmp/cpp-rs-driver-install-prefix && cmake --build /tmp/cpp-rs-driver-install-build -j 4 && cmake --install /tmp/cpp-rs-driver-install-buildPKG_CONFIG_PATH=/tmp/cpp-rs-driver-install-prefix/lib/pkgconfig:/usr/lib/x86_64-linux-gnu/pkgconfig:/usr/share/pkgconfig cmake -S packaging/smoke-test-app -B /tmp/cpp-rs-driver-smoke-build -G Ninja -DCMAKE_BUILD_TYPE=Release -DSCYLLA_SMOKE_BUILD_STATIC=ON && PKG_CONFIG_PATH=/tmp/cpp-rs-driver-install-prefix/lib/pkgconfig:/usr/lib/x86_64-linux-gnu/pkgconfig:/usr/share/pkgconfig cmake --build /tmp/cpp-rs-driver-smoke-build -j 4cmake --build build_static -j 4 --target cassandra-integration-testsmake -n OS=Windows_NT build-driver CMAKE_BUILD_TYPE=Release | rg "OPENSSL_LIBS|Join\(':', @\("make -n OS=Windows_NT build-static-integration-test-bin | rg "OPENSSL_LIBS|Join\(':', @\("make -n test-package-deb-static-smokemake -n test-package-rpm-native-static-smoke SCYLLA_HOST=scylla SKIP_DOCKER_COMPOSE=1make -n test-package-macos-static-smokePre-review checklist
Makefilein{SCYLLA,CASSANDRA}_(NO_VALGRIND_)TEST_FILTER.Fixes:annotations to PR description.