Skip to content

Conversation

@assignUser
Copy link
Member

No description provided.

@github-actions
Copy link

@github-actions
Copy link

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

@assignUser
Copy link
Member Author

@kszucs could you take a look if this is correct?

@kszucs
Copy link
Member

kszucs commented Apr 14, 2022

@github-actions crossbow submit wheel

@github-actions
Copy link

Revision: 52fc416eacde6de48f550c297709d396efdef0e8

Submitted crossbow builds: ursacomputing/crossbow @ actions-1862

Task Status
verify-rc-binaries-wheels-linux-almalinux-8-amd64 Github Actions
verify-rc-binaries-wheels-linux-conda-latest-amd64 Github Actions
verify-rc-binaries-wheels-linux-ubuntu-18.04-amd64 Github Actions
verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64 Github Actions
verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64 Github Actions
verify-rc-binaries-wheels-macos-10.15-amd64 Github Actions
verify-rc-binaries-wheels-macos-10.15-amd64-conda Github Actions
verify-rc-binaries-wheels-macos-11-amd64 Github Actions
verify-rc-binaries-wheels-macos-11-arm64 Github Actions
verify-rc-binaries-wheels-windows Github Actions
wheel-macos-big-sur-cp310-arm64 Github Actions
wheel-macos-big-sur-cp310-universal2 Github Actions
wheel-macos-big-sur-cp38-arm64 Github Actions
wheel-macos-big-sur-cp39-arm64 Github Actions
wheel-macos-big-sur-cp39-universal2 Github Actions
wheel-macos-high-sierra-cp310-amd64 Github Actions
wheel-macos-high-sierra-cp37-amd64 Github Actions
wheel-macos-high-sierra-cp38-amd64 Github Actions
wheel-macos-high-sierra-cp39-amd64 Github Actions
wheel-macos-mavericks-cp310-amd64 Github Actions
wheel-macos-mavericks-cp37-amd64 Github Actions
wheel-macos-mavericks-cp38-amd64 Github Actions
wheel-macos-mavericks-cp39-amd64 Github Actions
wheel-manylinux2010-cp310-amd64 Github Actions
wheel-manylinux2010-cp37-amd64 Github Actions
wheel-manylinux2010-cp38-amd64 Github Actions
wheel-manylinux2010-cp39-amd64 Github Actions
wheel-manylinux2014-cp310-amd64 Github Actions
wheel-manylinux2014-cp310-arm64 TravisCI
wheel-manylinux2014-cp37-amd64 Github Actions
wheel-manylinux2014-cp37-arm64 TravisCI
wheel-manylinux2014-cp38-amd64 Github Actions
wheel-manylinux2014-cp38-arm64 TravisCI
wheel-manylinux2014-cp39-amd64 Github Actions
wheel-manylinux2014-cp39-arm64 TravisCI
wheel-windows-cp310-amd64 Github Actions
wheel-windows-cp37-amd64 Github Actions
wheel-windows-cp38-amd64 Github Actions
wheel-windows-cp39-amd64 Github Actions

@assignUser
Copy link
Member Author

@kszucs I think most of those fails are caused by the vcpkg image not being up to date. I saw that the action in the corssbow repo for that runs on cron and can not be manually triggered for cases like this. I'll open a pr,

@kszucs
Copy link
Member

kszucs commented Apr 19, 2022

@assignUser that should only apply for the windows images, but the linux images get rebuilt in each run if they are not up to date.

@assignUser
Copy link
Member Author

@github-actions crossbow submit python-wheel-manylinux-2014

@github-actions
Copy link

Unable to match any tasks for `python-wheel-manylinux-2014`
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/2189254246```

@assignUser
Copy link
Member Author

@github-actions crossbow submit -g wheel

@github-actions
Copy link

Revision: 7664672c186bc6086ea48d6a3b78af5ec49a1171

Submitted crossbow builds: ursacomputing/crossbow @ actions-1873

Task Status
wheel-macos-big-sur-cp310-arm64 Github Actions
wheel-macos-big-sur-cp310-universal2 Github Actions
wheel-macos-big-sur-cp38-arm64 Github Actions
wheel-macos-big-sur-cp39-arm64 Github Actions
wheel-macos-big-sur-cp39-universal2 Github Actions
wheel-macos-high-sierra-cp310-amd64 Github Actions
wheel-macos-high-sierra-cp37-amd64 Github Actions
wheel-macos-high-sierra-cp38-amd64 Github Actions
wheel-macos-high-sierra-cp39-amd64 Github Actions
wheel-macos-mavericks-cp310-amd64 Github Actions
wheel-macos-mavericks-cp37-amd64 Github Actions
wheel-macos-mavericks-cp38-amd64 Github Actions
wheel-macos-mavericks-cp39-amd64 Github Actions
wheel-manylinux2010-cp310-amd64 Github Actions
wheel-manylinux2010-cp37-amd64 Github Actions
wheel-manylinux2010-cp38-amd64 Github Actions
wheel-manylinux2010-cp39-amd64 Github Actions
wheel-manylinux2014-cp310-amd64 Github Actions
wheel-manylinux2014-cp310-arm64 TravisCI
wheel-manylinux2014-cp37-amd64 Github Actions
wheel-manylinux2014-cp37-arm64 TravisCI
wheel-manylinux2014-cp38-amd64 Github Actions
wheel-manylinux2014-cp38-arm64 TravisCI
wheel-manylinux2014-cp39-amd64 Github Actions
wheel-manylinux2014-cp39-arm64 TravisCI
wheel-windows-cp310-amd64 Github Actions
wheel-windows-cp37-amd64 Github Actions
wheel-windows-cp38-amd64 Github Actions
wheel-windows-cp39-amd64 Github Actions

@assignUser
Copy link
Member Author

The linux-headers thing was a red hering the actual issue was: Can't locate IPC/Cmd.pm in @INC (@INC contains:
Which seems to be fixed by installing perl-IPC-Cmd via yum.
I'll push once the build finishes on my machine.

@kszucs I don't really know why the macos build would fail due to permission issues (e.g. https://github.com/ursacomputing/crossbow/runs/6077317725?check_suite_focus=true#step:8:27 ) do you have any insights on what could cause that?

@assignUser
Copy link
Member Author

@github-actions crossbow submit -g wheel

@github-actions
Copy link

Revision: 75c22be833155169837d661c38734f4656410a9a

Submitted crossbow builds: ursacomputing/crossbow @ actions-1875

Task Status
wheel-macos-big-sur-cp310-arm64 Github Actions
wheel-macos-big-sur-cp310-universal2 Github Actions
wheel-macos-big-sur-cp38-arm64 Github Actions
wheel-macos-big-sur-cp39-arm64 Github Actions
wheel-macos-big-sur-cp39-universal2 Github Actions
wheel-macos-high-sierra-cp310-amd64 Github Actions
wheel-macos-high-sierra-cp37-amd64 Github Actions
wheel-macos-high-sierra-cp38-amd64 Github Actions
wheel-macos-high-sierra-cp39-amd64 Github Actions
wheel-macos-mavericks-cp310-amd64 Github Actions
wheel-macos-mavericks-cp37-amd64 Github Actions
wheel-macos-mavericks-cp38-amd64 Github Actions
wheel-macos-mavericks-cp39-amd64 Github Actions
wheel-manylinux2010-cp310-amd64 Github Actions
wheel-manylinux2010-cp37-amd64 Github Actions
wheel-manylinux2010-cp38-amd64 Github Actions
wheel-manylinux2010-cp39-amd64 Github Actions
wheel-manylinux2014-cp310-amd64 Github Actions
wheel-manylinux2014-cp310-arm64 TravisCI
wheel-manylinux2014-cp37-amd64 Github Actions
wheel-manylinux2014-cp37-arm64 TravisCI
wheel-manylinux2014-cp38-amd64 Github Actions
wheel-manylinux2014-cp38-arm64 TravisCI
wheel-manylinux2014-cp39-amd64 Github Actions
wheel-manylinux2014-cp39-arm64 TravisCI
wheel-windows-cp310-amd64 Github Actions
wheel-windows-cp37-amd64 Github Actions
wheel-windows-cp38-amd64 Github Actions
wheel-windows-cp39-amd64 Github Actions

@kszucs
Copy link
Member

kszucs commented Apr 20, 2022

@kszucs I don't really know why the macos build would fail due to permission issues (e.g. https://github.com/ursacomputing/crossbow/runs/6077317725?check_suite_focus=true#step:8:27 ) do you have any insights on what could cause that?

The github actions service has stopped on our self-hosted machine and when I applied a workaround I started it with the wrong user. It should be fixed now.

@assignUser
Copy link
Member Author

@github-actions crossbow submit -g wheel

@github-actions
Copy link

Revision: e334dfe6fe179738ae0e7f4e61d238c786f2e7d7

Submitted crossbow builds: ursacomputing/crossbow @ actions-1885

Task Status
wheel-macos-big-sur-cp310-arm64 Github Actions
wheel-macos-big-sur-cp310-universal2 Github Actions
wheel-macos-big-sur-cp38-arm64 Github Actions
wheel-macos-big-sur-cp39-arm64 Github Actions
wheel-macos-big-sur-cp39-universal2 Github Actions
wheel-macos-high-sierra-cp310-amd64 Github Actions
wheel-macos-high-sierra-cp37-amd64 Github Actions
wheel-macos-high-sierra-cp38-amd64 Github Actions
wheel-macos-high-sierra-cp39-amd64 Github Actions
wheel-macos-mavericks-cp310-amd64 Github Actions
wheel-macos-mavericks-cp37-amd64 Github Actions
wheel-macos-mavericks-cp38-amd64 Github Actions
wheel-macos-mavericks-cp39-amd64 Github Actions
wheel-manylinux2010-cp310-amd64 Github Actions
wheel-manylinux2010-cp37-amd64 Github Actions
wheel-manylinux2010-cp38-amd64 Github Actions
wheel-manylinux2010-cp39-amd64 Github Actions
wheel-manylinux2014-cp310-amd64 Github Actions
wheel-manylinux2014-cp310-arm64 TravisCI
wheel-manylinux2014-cp37-amd64 Github Actions
wheel-manylinux2014-cp37-arm64 TravisCI
wheel-manylinux2014-cp38-amd64 Github Actions
wheel-manylinux2014-cp38-arm64 TravisCI
wheel-manylinux2014-cp39-amd64 Github Actions
wheel-manylinux2014-cp39-arm64 TravisCI
wheel-windows-cp310-amd64 Github Actions
wheel-windows-cp37-amd64 Github Actions
wheel-windows-cp38-amd64 Github Actions
wheel-windows-cp39-amd64 Github Actions

@kszucs
Copy link
Member

kszucs commented Apr 20, 2022

@assignUser rebase on master in order to resolve the big-sur test errors

@kszucs
Copy link
Member

kszucs commented Apr 20, 2022

Can you build and push the windows images using the following?

# build
REPO=ghcr.io/ursacomputing/arrow PYTHON=3.7 archery docker build --using-docker-cli --no-pull python-wheel-windows-vs2017
REPO=ghcr.io/ursacomputing/arrow PYTHON=3.8 archery docker build --using-docker-cli --no-pull python-wheel-windows-vs2017
REPO=ghcr.io/ursacomputing/arrow PYTHON=3.9 archery docker build --using-docker-cli --no-pull python-wheel-windows-vs2017
REPO=ghcr.io/ursacomputing/arrow PYTHON=3.10 archery docker build --using-docker-cli --no-pull python-wheel-windows-vs2017

# push
REPO=ghcr.io/ursacomputing/arrow PYTHON=3.7 archery docker push --using-docker-cli python-wheel-windows-vs2017
REPO=ghcr.io/ursacomputing/arrow PYTHON=3.8 archery docker push --using-docker-cli python-wheel-windows-vs2017
REPO=ghcr.io/ursacomputing/arrow PYTHON=3.9 archery docker push --using-docker-cli python-wheel-windows-vs2017
REPO=ghcr.io/ursacomputing/arrow PYTHON=3.10 archery docker push --using-docker-cli python-wheel-windows-vs2017

@assignUser assignUser force-pushed the ARROW-16198-bump-vcpkg branch from 89fa1f7 to 9176c16 Compare April 20, 2022 16:03
@assignUser
Copy link
Member Author

@github-actions crossbow submit -g wheel

@github-actions
Copy link

Revision: 9176c1615bdeb872dd29837169eea36e7db8e9b2

Submitted crossbow builds: ursacomputing/crossbow @ actions-1887

Task Status
wheel-macos-big-sur-cp310-arm64 Github Actions
wheel-macos-big-sur-cp310-universal2 Github Actions
wheel-macos-big-sur-cp38-arm64 Github Actions
wheel-macos-big-sur-cp39-arm64 Github Actions
wheel-macos-big-sur-cp39-universal2 Github Actions
wheel-macos-high-sierra-cp310-amd64 Github Actions
wheel-macos-high-sierra-cp37-amd64 Github Actions
wheel-macos-high-sierra-cp38-amd64 Github Actions
wheel-macos-high-sierra-cp39-amd64 Github Actions
wheel-macos-mavericks-cp310-amd64 Github Actions
wheel-macos-mavericks-cp37-amd64 Github Actions
wheel-macos-mavericks-cp38-amd64 Github Actions
wheel-macos-mavericks-cp39-amd64 Github Actions
wheel-manylinux2010-cp310-amd64 Github Actions
wheel-manylinux2010-cp37-amd64 Github Actions
wheel-manylinux2010-cp38-amd64 Github Actions
wheel-manylinux2010-cp39-amd64 Github Actions
wheel-manylinux2014-cp310-amd64 Github Actions
wheel-manylinux2014-cp310-arm64 TravisCI
wheel-manylinux2014-cp37-amd64 Github Actions
wheel-manylinux2014-cp37-arm64 TravisCI
wheel-manylinux2014-cp38-amd64 Github Actions
wheel-manylinux2014-cp38-arm64 TravisCI
wheel-manylinux2014-cp39-amd64 Github Actions
wheel-manylinux2014-cp39-arm64 TravisCI
wheel-windows-cp310-amd64 Github Actions
wheel-windows-cp37-amd64 Github Actions
wheel-windows-cp38-amd64 Github Actions
wheel-windows-cp39-amd64 Github Actions

@assignUser
Copy link
Member Author

assignUser commented Apr 21, 2022

Found the issue with the windows builds, due to #7388 cmake was looking for zstd_static.lib even though the version installed via vcpkg is called zstd.lib (since microsoft/vcpkg@d878305). I fixed this by manually setting the suffix to "", not sure if modifying the logic in Findzstd.cmake is necessary?

@assignUser
Copy link
Member Author

I created an issue for adding the automatic change to the documentation: https://issues.apache.org/jira/browse/ARROW-16265

@kszucs
Copy link
Member

kszucs commented Apr 21, 2022

Found the issue with the windows builds, due to #7388 cmake was looking for zstd_static.lib even though the version installed via vcpkg is called zstd.lib (since microsoft/vcpkg@d878305). I fixed this by manually setting the suffix to "", not sure if modifying the logic in Findzstd.cmake is necessary?

Nice catch! Could you set that cmake variable in Usevcpkg.cmake?

@kszucs
Copy link
Member

kszucs commented Apr 21, 2022

I'm going to check locally what's the issue with absl in the universal2 builds.

@assignUser
Copy link
Member Author

assignUser commented Apr 21, 2022

I created a job to dump the logs but that hasn't been run for over 2h ... Githubs Macs seem to be busy xD self-hosted jobs will wait for ever if you don't have a runner 🤦‍♂️

@github-actions
Copy link

Revision: 2f2c28a

Submitted crossbow builds: ursacomputing/crossbow @ actions-1908

Task Status
wheel-macos-big-sur-cp310-universal2 Github Actions

@kszucs
Copy link
Member

kszucs commented Apr 22, 2022

#12959 should resolve the universal2 issues, wait for that to be merged first then rebase this PR

@kszucs kszucs force-pushed the ARROW-16198-bump-vcpkg branch from 2f2c28a to 1459be8 Compare April 22, 2022 17:26
@kszucs
Copy link
Member

kszucs commented Apr 22, 2022

@github-actions crossbow submit -g wheel

@github-actions
Copy link

Revision: 5085fd2

Submitted crossbow builds: ursacomputing/crossbow @ actions-1916

Task Status
wheel-macos-big-sur-cp310-arm64 Github Actions
wheel-macos-big-sur-cp310-universal2 Github Actions
wheel-macos-big-sur-cp38-arm64 Github Actions
wheel-macos-big-sur-cp39-arm64 Github Actions
wheel-macos-big-sur-cp39-universal2 Github Actions
wheel-macos-high-sierra-cp310-amd64 Github Actions
wheel-macos-high-sierra-cp37-amd64 Github Actions
wheel-macos-high-sierra-cp38-amd64 Github Actions
wheel-macos-high-sierra-cp39-amd64 Github Actions
wheel-macos-mavericks-cp310-amd64 Github Actions
wheel-macos-mavericks-cp37-amd64 Github Actions
wheel-macos-mavericks-cp38-amd64 Github Actions
wheel-macos-mavericks-cp39-amd64 Github Actions
wheel-manylinux2010-cp310-amd64 Github Actions
wheel-manylinux2010-cp37-amd64 Github Actions
wheel-manylinux2010-cp38-amd64 Github Actions
wheel-manylinux2010-cp39-amd64 Github Actions
wheel-manylinux2014-cp310-amd64 Github Actions
wheel-manylinux2014-cp310-arm64 TravisCI
wheel-manylinux2014-cp37-amd64 Github Actions
wheel-manylinux2014-cp37-arm64 TravisCI
wheel-manylinux2014-cp38-amd64 Github Actions
wheel-manylinux2014-cp38-arm64 TravisCI
wheel-manylinux2014-cp39-amd64 Github Actions
wheel-manylinux2014-cp39-arm64 TravisCI
wheel-windows-cp310-amd64 Github Actions
wheel-windows-cp37-amd64 Github Actions
wheel-windows-cp38-amd64 Github Actions
wheel-windows-cp39-amd64 Github Actions

@kszucs
Copy link
Member

kszucs commented Apr 22, 2022

@github-actions crossbow submit wheel-macos-big-sur-cp310-universal2

@github-actions
Copy link

Revision: cb83048

Submitted crossbow builds: ursacomputing/crossbow @ actions-1921

Task Status
wheel-macos-big-sur-cp310-universal2 Github Actions

@kszucs
Copy link
Member

kszucs commented Apr 22, 2022

@github-actions crossbow submit wheel-macos-high-sierra-cp310-amd64

@github-actions
Copy link

Revision: cb83048

Submitted crossbow builds: ursacomputing/crossbow @ actions-1923

Task Status
wheel-macos-high-sierra-cp310-amd64 Github Actions

@kszucs
Copy link
Member

kszucs commented Apr 22, 2022

@github-actions crossbow submit test-build-vcpkg-win

@github-actions
Copy link

Revision: cb83048

Submitted crossbow builds: ursacomputing/crossbow @ actions-1924

Task Status
test-build-vcpkg-win Github Actions

@kszucs
Copy link
Member

kszucs commented Apr 23, 2022

test-build-vcpkg-win failure is unrelated to this PR

Copy link
Member

@kszucs kszucs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @assignUser!

@kszucs kszucs closed this in 40c1226 Apr 23, 2022
@ursabot
Copy link

ursabot commented Apr 28, 2022

Benchmark runs are scheduled for baseline = f3df0cb and contender = 40c1226. 40c1226 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed] test-mac-arm
[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.08% ⬆️0.0%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/608| 40c1226f ec2-t3-xlarge-us-east-2>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/596| 40c1226f test-mac-arm>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/595| 40c1226f ursa-i9-9960x>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/606| 40c1226f ursa-thinkcentre-m75q>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/607| f3df0cb6 ec2-t3-xlarge-us-east-2>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/595| f3df0cb6 test-mac-arm>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/594| f3df0cb6 ursa-i9-9960x>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/605| f3df0cb6 ursa-thinkcentre-m75q>
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@assignUser assignUser deleted the ARROW-16198-bump-vcpkg branch June 21, 2022 07:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants