Skip to content

[Python][C++] PyArrow fails to build trying to find a system abseil when Arrow C++ was built with bundled abseil #49764

@raulcd

Description

@raulcd

Describe the bug, including details regarding any error messages, version, and platform.

After the following PR was merged:

If I build Arrow C++ with bundled abseil, example of my build locally:

export ARROW_HOME=~/code/dist
export LD_LIBRARY_PATH=~/code/dist/lib:$LD_LIBRARY_PATH
export CMAKE_PREFIX_PATH=$ARROW_HOME:$CMAKE_PREFIX_PATH
export CMAKE_BUILD_PARALLEL_LEVEL=7
cmake -S arrow/cpp -B arrow/cpp/build \
        -DCMAKE_INSTALL_PREFIX=$ARROW_HOME \
        --preset ninja-debug-python-maximal
cmake --build arrow/cpp/build --target install

And then try to build PyArrow with the following:

export ARROW_HOME=~/code/dist
export LD_LIBRARY_PATH=~/code/dist/lib:$LD_LIBRARY_PATH
export CMAKE_PREFIX_PATH=$ARROW_HOME:$CMAKE_PREFIX_PATH
export PYARROW_BUNDLE_CYTHON_CPP=1
export CMAKE_BUILD_PARALLEL_LEVEL=4

cd arrow/python
time python -m pip install -C cmake.build-type=Debug -C build.verbose=true --no-deps --no-build-isolation -vv --config-settings=build-dir=build -e .

It fails with:

$ time python -m pip install -C cmake.build-type=Debug -C build.verbose=true --no-deps --no-build-isolation -vv --config-settings=build-dir=build -e .
Using pip 25.2 from /home/raulcd/code/pyarrow-dev/lib/python3.13/site-packages/pip (python 3.13)
Non-user install because user site-packages disabled
Created temporary directory: /tmp/pip-build-tracker-srsdn7we
Initialized build tracking at /tmp/pip-build-tracker-srsdn7we
Created build tracker: /tmp/pip-build-tracker-srsdn7we
Entered build tracker: /tmp/pip-build-tracker-srsdn7we
Created temporary directory: /tmp/pip-install-8ctg2cdg
Created temporary directory: /tmp/pip-ephem-wheel-cache-kyr1sawn
Obtaining file:///home/raulcd/code/arrow/python
  Added file:///home/raulcd/code/arrow/python to build tracker '/tmp/pip-build-tracker-srsdn7we'
  Running command Checking if build backend supports build_editable
  Checking if build backend supports build_editable ... done
  Created temporary directory: /tmp/pip-modern-metadata-pxffjlmo
  Running command Preparing editable metadata (pyproject.toml)
  *** scikit-build-core 0.11.6 using CMake 4.2.3 (metadata_editable)
  Preparing editable metadata (pyproject.toml) ... done
  Source in /home/raulcd/code/arrow/python has version 24.0.0.dev309+gb95645d21.d20260415, which satisfies requirement pyarrow==24.0.0.dev309+gb95645d21.d20260415 from file:///home/raulcd/code/arrow/python
  Removed pyarrow==24.0.0.dev309+gb95645d21.d20260415 from file:///home/raulcd/code/arrow/python from build tracker '/tmp/pip-build-tracker-srsdn7we'
Created temporary directory: /tmp/pip-unpack-8ddqeqnt
Building wheels for collected packages: pyarrow
  Created temporary directory: /tmp/pip-wheel-2uujcf0x
  Destination directory: /tmp/pip-wheel-2uujcf0x
  Running command Building editable for pyarrow (pyproject.toml)
  *** scikit-build-core 0.11.6 using CMake 4.2.3 (editable)
  *** Configuring CMake...
  loading initial cache file build/CMakeInit.txt
  -- System processor: x86_64
  -- Arrow build warning level: PRODUCTION
  -- Build Type: Debug
  -- CMAKE_C_FLAGS:  -Wall -fno-semantic-interposition -msse4.2  -fdiagnostics-color=always  -fno-omit-frame-pointer -Wno-unused-variable -Wno-maybe-uninitialized
  -- CMAKE_CXX_FLAGS:  -Wno-noexcept-type -Wno-self-move  -Wall -fno-semantic-interposition -msse4.2  -fdiagnostics-color=always  -fno-omit-frame-pointer -Wno-unused-variable -Wno-maybe-uninitialized
  -- Generator: Ninja
  -- Build output directory: /home/raulcd/code/arrow/python/build/debug
  -- Found NumPy version: 2.3.4
  -- NumPy include dir: /home/raulcd/code/pyarrow-dev/lib/python3.13/site-packages/numpy/_core/include
  -- Found Cython version: 3.2.4
  CMake Error at /home/raulcd/code/dist/lib/cmake/Arrow/FindabslAlt.cmake:31 (find_package):
    By not providing "Findabsl.cmake" in CMAKE_MODULE_PATH this project has
    asked CMake to find a package configuration file provided by "absl", but
    CMake did not find one.

    Could not find a package configuration file provided by "absl" with any of
    the following names:

      abslConfig.cmake
      absl-config.cmake

    Add the installation prefix of "absl" to CMAKE_PREFIX_PATH or set
    "absl_DIR" to a directory containing one of the above files.  If "absl"
    provides a separate development package or SDK, be sure it has been
    installed.
  Call Stack (most recent call first):
    /usr/share/cmake-4.2/Modules/CMakeFindDependencyMacro.cmake:93 (find_package)
    /usr/share/cmake-4.2/Modules/CMakeFindDependencyMacro.cmake:125 (__find_dependency_common)
    /home/raulcd/code/dist/lib/cmake/Arrow/ArrowConfig.cmake:101 (find_dependency)
    /home/raulcd/code/dist/lib/cmake/Arrow/ArrowConfig.cmake:124 (arrow_find_dependencies)
    CMakeLists.txt:294 (find_package)


  -- Configuring incomplete, errors occurred!

  *** CMake configuration failed
  error: subprocess-exited-with-error
  
  × Building editable for pyarrow (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/raulcd/code/pyarrow-dev/bin/python /home/raulcd/code/pyarrow-dev/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_editable /tmp/tmpf_0c3_78
  cwd: /home/raulcd/code/arrow/python
  Building editable for pyarrow (pyproject.toml) ... error
  ERROR: Failed building editable for pyarrow
Failed to build pyarrow
error: failed-wheel-build-for-install

× Failed to build installable wheels for some pyproject.toml based projects
╰─> pyarrow
Exception information:
Traceback (most recent call last):
  File "/home/raulcd/code/pyarrow-dev/lib/python3.13/site-packages/pip/_internal/cli/base_command.py", line 107, in _run_wrapper
    status = _inner_run()
  File "/home/raulcd/code/pyarrow-dev/lib/python3.13/site-packages/pip/_internal/cli/base_command.py", line 98, in _inner_run
    return self.run(options, args)
           ~~~~~~~~^^^^^^^^^^^^^^^
  File "/home/raulcd/code/pyarrow-dev/lib/python3.13/site-packages/pip/_internal/cli/req_command.py", line 71, in wrapper
    return func(self, options, args)
  File "/home/raulcd/code/pyarrow-dev/lib/python3.13/site-packages/pip/_internal/commands/install.py", line 442, in run
    raise InstallWheelBuildError(build_failures)
pip._internal.exceptions.InstallWheelBuildError: <InstallWheelBuildError: failed-wheel-build-for-install>
Removed build tracker: '/tmp/pip-build-tracker-srsdn7we'

real	0m2.883s
user	0m4.272s
sys	0m0.567s

Basically when ARROW_BUILD_STATIC=ON, somehow absl is in ARROW_SYSTEM_DEPENDENCIES and fails to be found.

Component(s)

C++, Python

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions