[arrow] Update to 17.0.0 by raulcd · Pull Request #39948 · microsoft/vcpkg

raulcd · 2024-07-16T11:02:38Z

Changes comply with the maintainer guide.
SHA512s are updated for each updated download.
The "supports" clause reflects platforms that may be fixed by this new version.
Any fixed CI baseline entries are removed from that file.
Any patches that are no longer applied are deleted from the port's directory.
The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
Only one version is added to each modified port's versions file.

MonicaLiu0311 · 2024-07-18T05:56:27Z

I got the following error when testing the cuda feature:

CMake Error at scripts/cmake/vcpkg_fixup_pkgconfig.cmake:134 (message):
  G:/arrow/downloads/tools/msys2/1e74ca60daa10104/mingw64/bin/pkg-config.exe
  --exists arrow-cuda failed with error code: 1

      ENV{PKG_CONFIG_PATH}: "G:/arrow/packages/arrow_x64-windows/lib/pkgconfig;G:/arrow/packages/arrow_x64-windows/share/pkgconfig;G:/arrow/installed/x64-windows/lib/pkgconfig;G:/arrow/installed/x64-windows/share/pkgconfig"
      output: Package cuda was not found in the pkg-config search path.

  Perhaps you should add the directory containing `cuda.pc'

  to the PKG_CONFIG_PATH environment variable

  Package 'cuda', required by 'arrow-cuda', not found
Call Stack (most recent call first):
  scripts/cmake/vcpkg_fixup_pkgconfig.cmake:196 (z_vcpkg_fixup_pkgconfig_check_files)
  ports/arrow/portfile.cmake:71 (vcpkg_fixup_pkgconfig)
  scripts/ports.cmake:192 (include)

raulcd · 2024-07-18T10:40:58Z

@kou any idea?

raulcd · 2024-07-18T14:21:06Z

ok, it seems the cuda dependency:

vcpkg/ports/cuda/portfile.cmake

Line 1 in 492d1cc

    
           # This package doesn't install CUDA. It instead verifies that CUDA is installed.

doesn't really install cuda.
So cuda should be installed as a system dependency.
At this point, we might want to remove the arrow-cuda feature, thoughts?

kou

@MonicaLiu0311 Can CUDA be found with other port? I want to know that the CUDA not found error is only caused with this port.

FYI: Apache Arrow uses the standard find_package(CUDAToolkit REQUIRED): https://github.com/apache/arrow/blob/apache-arrow-17.0.0/cpp/src/arrow/gpu/CMakeLists.txt#L44
FindCUDAToolkit.cmake is included in CMake.

If it doesn't work with vcpkg, we may want to improve the cuda port.

kou · 2024-07-19T00:45:41Z

It seems that we can also remove this: https://github.com/apache/arrow/blob/ec58e4de9e08e4398e293cde70a7fed5bfb3ba5c/cpp/src/arrow/CMakeLists.txt#L166

@kou I think this is not the URL you wanted to share here :) so I am not sure what you wanted us to remove here

Oh... Sorry... I've fixed the URL...
I wanted to show the latest code about this...

I did update the patch to apply, it is still required because:

-if(NOT WIN32 AND NOT APPLE) +if(NOT WIN32 AND NOT APPLE AND NOT ANDROID)

is necessary

this is just the update to apply, not something that we will modify (because it was modified on Arrow):

list(APPEND ARROW_SYSTEM_LINK_LIBS "ws2_32.dll") list(APPEND ARROW_SYSTEM_LINK_LIBS "ws2_32")

Ah, I see. I was wrong. Diff's diff is difficult to review...

Anyway, let's upstream and remove this patch in a future release: apache/arrow#43390

dg0yt · 2024-07-19T03:29:31Z

@kou The error is from vcpkg_fixup_pkgconfig after the actual build. It points to the fact that arrow installs arrow-cuda.pc which refers to cuda.pc file even if such a file doesn't exist. That's why this error occurs only for arrow.

Obviously, cuda.pc should come with CUDA. Maybe CUDA does not provide pc files for Windows?

Possible fixes and workarounds.

Make port cuda add a cuda.pc if it is missing.
Remove cuda from Requires[.private] in arrow-cuda.pc (but ensure proper link Libs[.private]).
Remove arrow-cuda.pc for Windows.

kou · 2024-07-19T08:57:06Z

Ah, I got it.

I downloaded the CUDA package for Windows https://developer.nvidia.com/cuda-downloads?target_os=Windows&target_arch=x86_64&target_version=11&target_type=exe_local and checked the content. It doesn't provide cuda.pc...

Can we keep the current configuration as-is for now? We will stop installing arrow-cuda.pc on Windows in upstream. So the cuda feature will work on Windows with Apache Arrow 18.0.0.

MonicaLiu0311 · 2024-07-19T10:02:56Z

@MonicaLiu0311 Can CUDA be found with other port? I want to know that the CUDA not found error is only caused with this port.

PS G:\vcpkg> ./vcpkg install pcl[cuda] --recurse
Computing installation plan...
The following packages are already installed:
    pcl[core,cuda]:x64-windows@1.14.1
pcl:x64-windows is already installed
Total install time: 1.74 ms
The package pcl provides CMake targets:

    find_package(PCL CONFIG REQUIRED)
    target_link_libraries(main PRIVATE ${PCL_LIBRARIES})

kou · 2024-07-22T20:58:03Z

Thanks.
It also shows that #39948 (comment) is right.
And #39948 (comment) is my suggestion to move forward this.

MonicaLiu0311 · 2024-08-06T08:18:20Z

Convert this PR to draft since there is no progress. Please ping us if this PR is ready for review again.

raulcd · 2024-08-06T08:49:44Z

Convert this PR to draft since there is no progress. Please ping us if this PR is ready for review again.

@MonicaLiu0311 from my understanding @kou was asking you if we could keep the PR as is as this will get solved on 18.0.0, so I was just waiting for your feedback.

MonicaLiu0311 · 2024-08-06T10:03:39Z

Convert this PR to draft since there is no progress. Please ping us if this PR is ready for review again.

@MonicaLiu0311 from my understanding @kou was asking you if we could keep the PR as is as this will get solved on 18.0.0, so I was just waiting for your feedback.

Sorry I missed this message. Please resolve the conflict and I will continue reviewing.

raulcd · 2024-08-07T08:57:39Z

ok, it seems a PR (with exactly the same changes as this original PR) was merged last week: #40164
I am closing this one now.

raulcd force-pushed the arrow-17.0.0 branch 3 times, most recently from 442474f to 99fe30f Compare July 16, 2024 17:01

[arrow] Update to 17.0.0

f081d1f

raulcd force-pushed the arrow-17.0.0 branch from 99fe30f to f081d1f Compare July 16, 2024 17:33

Update versions

492d1cc

FrankXie05 assigned MonicaLiu0311 Jul 17, 2024

MonicaLiu0311 added the category:port-update The issue is with a library, which is requesting update new revision label Jul 17, 2024

kou reviewed Jul 19, 2024

View reviewed changes

MonicaLiu0311 marked this pull request as draft August 6, 2024 08:17

kou mentioned this pull request Aug 6, 2024

[C++] Don't install arrow-cuda.pc on Windows apache/arrow#43591

Closed

raulcd closed this Aug 7, 2024

raulcd deleted the arrow-17.0.0 branch August 7, 2024 08:58

dg0yt mentioned this pull request Aug 7, 2024

[arrow] update to 17.0 #40119

Closed

Conversation

raulcd commented Jul 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MonicaLiu0311 commented Jul 18, 2024

Uh oh!

raulcd commented Jul 18, 2024

Uh oh!

raulcd commented Jul 18, 2024

Uh oh!

kou left a comment

Choose a reason for hiding this comment

Uh oh!

kou Jul 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

raulcd Jul 22, 2024

Choose a reason for hiding this comment

Uh oh!

kou Jul 22, 2024

Choose a reason for hiding this comment

Uh oh!

raulcd Jul 23, 2024

Choose a reason for hiding this comment

Uh oh!

raulcd Jul 23, 2024

Choose a reason for hiding this comment

Uh oh!

kou Jul 23, 2024

Choose a reason for hiding this comment

Uh oh!

dg0yt commented Jul 19, 2024

Uh oh!

kou commented Jul 19, 2024

Uh oh!

MonicaLiu0311 commented Jul 19, 2024

Uh oh!

kou commented Jul 22, 2024

Uh oh!

MonicaLiu0311 commented Aug 6, 2024

Uh oh!

raulcd commented Aug 6, 2024

Uh oh!

MonicaLiu0311 commented Aug 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

raulcd commented Aug 7, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

raulcd commented Jul 16, 2024 •

edited

Loading

kou Jul 19, 2024 •

edited

Loading

MonicaLiu0311 commented Aug 6, 2024 •

edited

Loading