GH-51329: [C++][CI] Test static linking with S3 and fix Azure/GCS deps

[tadeja]

Rationale for this change

Additional downstream static-linking test either pre #50195 (or followup)

What changes are included in this PR?

Test Arrow/S3 static linking,
fix dependencies for Azure and bundled GCS.

Are these changes tested?

Tested by CI

Are there any user-facing changes?

No.

• GitHub Issue: [C++] arrow_static's exported CMake interface omits LibXml2, so static links fail on the bundled Azure SDK #51329

[github-actions]

Thanks for opening a pull request!

This pull request has been automatically converted to a draft because its title doesn't match Arrow's required format.

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

After updating the title, you can mark the pull request as ready for review.

See also:

• Other pull requests
• Contribution Guidelines - Contributing Overview

[raulcd]

@tadeja do you mean adding this as part of #50195
or is this just for testing and follow up on a new issue PR to increase coverage?
I am ok testing it here and potentially adding improvements for testing the different packages (as also suggested Azure).

[raulcd]

@github-actions crossbow submit test-conda-cpp

[github-actions]

Revision: b6b76f8

Submitted crossbow builds: ursacomputing/crossbow @ actions-c1f06c30ce

Task	Status
test-conda-cpp

[tadeja]

@tadeja do you mean adding this as part of #50195 or is this just for testing and follow up on a new issue PR to increase coverage?

I don't mind, whichever is neater and more convenient to implement?
Also, I missed adding the change to cpp_test.sh 🤦 Adding it now.

[tadeja]

@github-actions crossbow submit test-conda-cpp

[github-actions]

Revision: f03e760

Submitted crossbow builds: ursacomputing/crossbow @ actions-e4020eadf0

Task	Status
test-conda-cpp

[tadeja]

@github-actions crossbow submit test-conda-cpp

[github-actions]

Revision: e28fd84

Submitted crossbow builds: ursacomputing/crossbow @ actions-4282328564

Task	Status
test-conda-cpp

[tadeja]

yay, new static linking test shows further failures for bundled Azure and google-cloud-cpp:
job ARM64 macos 14 C++

[ 50%] Built target arrow-example
[ 75%] Building CXX object CMakeFiles/arrow-example-s3-static.dir/example.cc.o
[100%] Linking CXX executable arrow-example-s3-static
Undefined symbols for architecture arm64:
  "_curl_easy_cleanup", referenced from:
      Azure::Core::_internal::UniqueHandleDeleter<void, curl_easy_cleanup>::operator()(void*) in libarrow_bundled_dependencies.a[2](curl.cpp.o)
      google::cloud::rest_internal::v3_2::CurlPtrCleanup::operator()(void*) const in libarrow_bundled_dependencies.a[214](curl_handle.cc.o)

job ARM64 Ubuntu 22.04 C++

[ 75%] Building CXX object CMakeFiles/arrow-example-s3-static.dir/example.cc.o
[100%] Linking CXX executable arrow-example-s3-static
/usr/bin/ld: /usr/local/lib/libarrow_bundled_dependencies.a(xml_wrapper.cpp.o): in function `Azure::Storage::_internal::XmlReader::XmlReader(char const*, unsigned long)':
/build/cpp/_deps/azure_sdk-src/sdk/storage/azure-storage-common/src/xml_wrapper.cpp:416: undefined reference to `xmlReaderForMemory'
...

[raulcd]

In my opinion it is better if we tackle this separately on its own issue.

[tadeja]

@github-actions crossbow submit test-conda-cpp

[raulcd]

this won't do anything here, right? Doesn't this has to be executed on top of my branch?

[tadeja]

Yes, this is to be able to potentially merge it either before or after #50195. Or within #50195 :)

[github-actions]

Revision: 4134d2d

Submitted crossbow builds: ursacomputing/crossbow @ actions-e76cd70dbd

Task	Status
test-conda-cpp

[tadeja]

I've just temporarily added here workaround of #50949 (cherry-pick 76eea10) to get test-conda-cpp succeed (still without the fix of #50195).

[tadeja]

Now with temporarily added workaround of #50949 (cherry-pick 76eea10) to get rid of
137 - arrow-filesystem-test (Failed)
143 - arrow-s3fs-module-test (Failed)
we get job AMD64 Conda C++ AVX2 to succeed
and also Docker Test conda-cpp to succeed
and both show the new static linking test is OK for now (before testing with current changes of #50195)

[100%] Linking CXX executable arrow-example-s3-static
[100%] Built target arrow-example-s3-static
/arrow/cpp/examples/minimal_build /build/cpp /
+ pushd /arrow/cpp/examples/minimal_build
+ PATH=/opt/conda/envs/arrow/bin:...
+ /build/cpp/examples/minimal_build/arrow-example
...
* Verify S3 initializes and finalizes
...

[timosachsenberg]

In my opinion it is better if we tackle this separately on its own issue.

Filed as #51329 — it reproduces the libxml2 half against a released artifact (libarrow-dev 24.0.0-1 from the APT repo, where libarrow.so records NEEDED libxml2.so.2 but Arrow::arrow_static declares nothing), and it ends in a dlopen-time undefined symbol: xmlBufferFree for a downstream consumer rather than a link error, which is why it went unnoticed. Happy for it to be closed as a duplicate if you'd rather fold this into #50195.

Two observations from writing it up, in case they're useful for wherever the fix lands:

• The pkg-config side needs its own line. The diff here adds LibXml2 to ARROW_SYSTEM_DEPENDENCIES and LibXml2::LibXml2 to ARROW_STATIC_INSTALL_INTERFACE_LIBS, but ARROW_PC_REQUIRES_PRIVATE is only ever populated via resolve_dependency(... PC_PACKAGE_NAMES ...), so a bare find_package(LibXml2 REQUIRED) won't reach arrow.pc. The bundled-AWS branch already does this explicitly at ThirdpartyToolchain.cmake:4262 — string(APPEND ARROW_PC_REQUIRES_PRIVATE " libcurl") — and the analogous line for Azure would be libxml-2.0 (plus libcurl).

• There's existing precedent for the whole shape of this. cpp/src/arrow/CMakeLists.txt populates ARROW_STATIC_INSTALL_INTERFACE_LIBS from fifteen SYSTEM branches and exactly one BUNDLED branch — AWSSDK_SOURCE STREQUAL "BUNDLED" → CURL::libcurl. So "a vendored dependency's own external requirements still have to be declared" is already an established pattern; Azure just never got it, which is also why the macOS _curl_easy_cleanup failure above is the same bug rather than a separate one.

Worth noting this is a different axis from #50753: that one is about SYSTEM declarations being easy to forget because they sit apart from resolve_dependency, whereas libxml2 isn't a dependency Arrow resolves at all, so co-locating wouldn't have surfaced it.

[github-actions]

⚠️ GitHub issue #51329 has been automatically assigned in GitHub to PR creator.

[Copilot]

🟡 Changes recommended

Resolve the Windows bundled-GCS curl linkage issue and align the CI S3 flag with effective build settings.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds static S3 linking coverage and exports dependencies needed by bundled Azure/GCS consumers.

Changes:

• Adds Azure/GCS static-link dependencies.
• Adds S3 initialization to the minimal example.
• Runs a static S3 smoke test in C++ CI.

File summaries

File	Reviewed changes
cpp/src/arrow/CMakeLists.txt	Adds static Azure/GCS dependency interfaces; bundled GCS curl linkage remains incomplete on Windows.
cpp/examples/minimal_build/example.cc	Exercises S3 initialization and finalization.
cpp/examples/minimal_build/CMakeLists.txt	Builds the S3-enabled static example.
cpp/cmake_modules/ThirdpartyToolchain.cmake	Updates bundled curl and libxml2 dependency discovery.
ci/scripts/cpp_test.sh	Builds and runs the S3 smoke test; its flag may diverge from effective build settings when threading is disabled.

Review details

Suppressed comments (2)

ci/scripts/cpp_test.sh:149

• cpp_build.sh force-disables ARROW_S3 when ARROW_ENABLE_THREADING=OFF (its lines 69-80), but this flag is derived from the original environment here. In that supported configuration with ARROW_S3=ON, Arrow is built without S3 and this test then tries to link the S3 example, producing a false failure. Please derive the example flag from the effective build settings or carry the override into the test step.

    -DARROW_EXAMPLE_S3="${ARROW_S3:-OFF}" \

cpp/src/arrow/CMakeLists.txt:52

• This only updates the exported CMake target. arrow.pc gets Requires.private/Libs.private from ARROW_PC_REQUIRES_PRIVATE/ARROW_PC_LIBS_PRIVATE, but neither this Azure branch nor the new bundled-GCS branch adds libxml-2.0 or libcurl there (the bundled AWS path explicitly adds libcurl at cpp/cmake_modules/ThirdpartyToolchain.cmake:4262-4264). Consequently pkg-config --static --libs arrow still omits the libraries needed by bundled Azure/GCS consumers. Please add the corresponding private pkg-config requirements for both dependencies.

  elseif(Azure_SOURCE STREQUAL "BUNDLED")
    if(UNIX)
      list(APPEND ARROW_STATIC_INSTALL_INTERFACE_LIBS CURL::libcurl LibXml2::LibXml2)
    endif()

• Files reviewed: 5/5 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.