Skip to content

Conversation

@kszucs
Copy link
Member

@kszucs kszucs commented Oct 18, 2021

No description provided.

@github-actions
Copy link

@github-actions
Copy link

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

@kszucs
Copy link
Member Author

kszucs commented Oct 18, 2021

The LLVM 13 package on MinGW might be faulty since for example the macOS build with LLVM 13 works as expected.

There is no easy way to downgrade LLVM to 12 in MinGW so I'm going to disable gandiva in those builds for now.

CMake Error at D:/a/_temp/msys64/mingw32/lib/cmake/llvm/LLVMExports.cmake:1254 (message):
  The imported target "mlir-tblgen" references the file

     "D:/a/_temp/msys64/mingw32/bin/mlir-tblgen.exe"

  but this file does not exist.  Possible reasons include:

  * The file was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and contained

     "D:/a/_temp/msys64/mingw32/lib/cmake/llvm/LLVMExports.cmake"

  but not all the files it references.

Call Stack (most recent call first):
  D:/a/_temp/msys64/mingw32/lib/cmake/llvm/LLVMConfig.cmake:263 (include)
  cmake_modules/FindLLVMAlt.cmake:27 (find_package)
  src/gandiva/CMakeLists.txt:28 (find_package)

@kszucs kszucs requested a review from bkietz October 18, 2021 14:35
@kszucs
Copy link
Member Author

kszucs commented Oct 18, 2021

@github-actions crossbow submit -g nightly

@github-actions
Copy link

Revision: b941060ac1f5b82b120192a05868fd29a32a5c59

Submitted crossbow builds: ursacomputing/crossbow @ actions-983

Task Status
almalinux-8-amd64 Github Actions
almalinux-8-arm64 TravisCI
amazon-linux-2-amd64 Github Actions
centos-7-amd64 Github Actions
centos-8-amd64 Github Actions
centos-8-arm64 TravisCI
conda-clean Azure
conda-linux-gcc-py36-arm64 Azure
conda-linux-gcc-py36-cpu-r40 Azure
conda-linux-gcc-py36-cuda Azure
conda-linux-gcc-py37-arm64 Azure
conda-linux-gcc-py37-cpu-r41 Azure
conda-linux-gcc-py37-cuda Azure
conda-linux-gcc-py38-arm64 Azure
conda-linux-gcc-py38-cpu Azure
conda-linux-gcc-py38-cuda Azure
conda-linux-gcc-py39-arm64 Azure
conda-linux-gcc-py39-cpu Azure
conda-linux-gcc-py39-cuda Azure
conda-osx-arm64-clang-py38 Azure
conda-osx-arm64-clang-py39 Azure
conda-osx-clang-py36-r40 Azure
conda-osx-clang-py37-r41 Azure
conda-osx-clang-py38 Azure
conda-osx-clang-py39 Azure
conda-win-vs2017-py36-r40 Azure
conda-win-vs2017-py37-r41 Azure
conda-win-vs2017-py38 Azure
conda-win-vs2017-py39 Azure
debian-bookworm-amd64 Github Actions
debian-bookworm-arm64 TravisCI
debian-bullseye-amd64 Github Actions
debian-bullseye-arm64 TravisCI
debian-buster-amd64 Github Actions
debian-buster-arm64 TravisCI
example-cpp-minimal-build-static Github Actions
example-cpp-minimal-build-static-system-dependency Github Actions
homebrew-cpp Github Actions
homebrew-r-autobrew Github Actions
java-jars Github Actions
nuget Github Actions
python-sdist Github Actions
test-build-vcpkg-win Github Actions
test-conda-cpp Github Actions
test-conda-cpp-valgrind Azure
test-conda-python-3.10 Github Actions
test-conda-python-3.6 Github Actions
test-conda-python-3.6-pandas-0.23 Github Actions
test-conda-python-3.7 Github Actions
test-conda-python-3.7-dask-latest Github Actions
test-conda-python-3.7-dask-master Github Actions
test-conda-python-3.7-hdfs-2.9.2 Github Actions
test-conda-python-3.7-hdfs-3.2.1 Github Actions
test-conda-python-3.7-kartothek-latest Github Actions
test-conda-python-3.7-kartothek-master Github Actions
test-conda-python-3.7-pandas-0.24 Github Actions
test-conda-python-3.7-pandas-latest Github Actions
test-conda-python-3.7-spark-branch-3.0 Github Actions
test-conda-python-3.8 Github Actions
test-conda-python-3.8-hypothesis Github Actions
test-conda-python-3.8-pandas-latest Github Actions
test-conda-python-3.8-pandas-nightly Github Actions
test-conda-python-3.8-spark-master Github Actions
test-conda-python-3.9 Github Actions
test-conda-python-3.9-pandas-master Github Actions
test-debian-11-cpp Github Actions
test-debian-11-go-1.15 Azure
test-debian-11-python-3 Azure
test-debian-c-glib Github Actions
test-debian-ruby Github Actions
test-fedora-33-cpp Github Actions
test-fedora-33-python-3 Azure
test-r-arrow-backwards-compatibility Github Actions
test-r-depsource-auto Azure
test-r-depsource-system Github Actions
test-r-devdocs Github Actions
test-r-gcc-11 Github Actions
test-r-install-local Github Actions
test-r-linux-as-cran Github Actions
test-r-linux-rchk Github Actions
test-r-linux-valgrind Azure
test-r-minimal-build Azure
test-r-offline-maximal Github Actions
test-r-offline-minimal Azure
test-r-rhub-debian-gcc-devel-lto-latest Azure
test-r-rhub-ubuntu-gcc-release-latest Azure
test-r-rocker-r-base-latest Azure
test-r-rstudio-r-base-3.6-bionic Azure
test-r-rstudio-r-base-3.6-centos7-devtoolset-8 Azure
test-r-rstudio-r-base-3.6-centos8 Azure
test-r-rstudio-r-base-3.6-opensuse15 Azure
test-r-rstudio-r-base-3.6-opensuse42 Azure
test-r-ubuntu-21.04 Github Actions
test-r-versions Github Actions
test-ubuntu-18.04-cpp Github Actions
test-ubuntu-18.04-cpp-release Github Actions
test-ubuntu-18.04-cpp-static Github Actions
test-ubuntu-18.04-python-3 Azure
test-ubuntu-18.04-r-sanitizer Azure
test-ubuntu-20.04-cpp Github Actions
test-ubuntu-20.04-cpp-14 Github Actions
test-ubuntu-20.04-cpp-17 Github Actions
test-ubuntu-20.04-cpp-thread-sanitizer Github Actions
test-ubuntu-20.10-docs Azure
test-ubuntu-c-glib Github Actions
test-ubuntu-default-docs Azure
test-ubuntu-ruby Github Actions
ubuntu-bionic-amd64 Github Actions
ubuntu-bionic-arm64 TravisCI
ubuntu-focal-amd64 Github Actions
ubuntu-focal-arm64 TravisCI
ubuntu-hirsute-amd64 Github Actions
ubuntu-hirsute-arm64 TravisCI
wheel-macos-big-sur-cp310-arm64 Github Actions
wheel-macos-big-sur-cp310-universal2 Github Actions
wheel-macos-big-sur-cp38-arm64 Github Actions
wheel-macos-big-sur-cp39-arm64 Github Actions
wheel-macos-big-sur-cp39-universal2 Github Actions
wheel-macos-high-sierra-cp310-amd64 Github Actions
wheel-macos-high-sierra-cp36-amd64 Github Actions
wheel-macos-high-sierra-cp37-amd64 Github Actions
wheel-macos-high-sierra-cp38-amd64 Github Actions
wheel-macos-high-sierra-cp39-amd64 Github Actions
wheel-macos-mavericks-cp310-amd64 Github Actions
wheel-macos-mavericks-cp36-amd64 Github Actions
wheel-macos-mavericks-cp37-amd64 Github Actions
wheel-macos-mavericks-cp38-amd64 Github Actions
wheel-macos-mavericks-cp39-amd64 Github Actions
wheel-manylinux2010-cp310-amd64 Github Actions
wheel-manylinux2010-cp36-amd64 Github Actions
wheel-manylinux2010-cp37-amd64 Github Actions
wheel-manylinux2010-cp38-amd64 Github Actions
wheel-manylinux2010-cp39-amd64 Github Actions
wheel-manylinux2014-cp310-amd64 Github Actions
wheel-manylinux2014-cp310-arm64 TravisCI
wheel-manylinux2014-cp36-amd64 Github Actions
wheel-manylinux2014-cp36-arm64 TravisCI
wheel-manylinux2014-cp37-amd64 Github Actions
wheel-manylinux2014-cp37-arm64 TravisCI
wheel-manylinux2014-cp38-amd64 Github Actions
wheel-manylinux2014-cp38-arm64 TravisCI
wheel-manylinux2014-cp39-amd64 Github Actions
wheel-manylinux2014-cp39-arm64 TravisCI
wheel-windows-cp310-amd64 Github Actions
wheel-windows-cp36-amd64 Github Actions
wheel-windows-cp37-amd64 Github Actions
wheel-windows-cp38-amd64 Github Actions
wheel-windows-cp39-amd64 Github Actions

@kou
Copy link
Member

kou commented Oct 18, 2021

I don't know why centos-7-amd64 build is failed with this change...:

https://github.com/ursacomputing/crossbow/runs/3928114316?check_suite_focus=true#step:8:913

FAILED: src/arrow/python/CMakeFiles/arrow_python_objlib.dir/helpers.cc.o 
/usr/lib64/ccache/c++  -DARROW_HAVE_RUNTIME_SSE4_2 -DARROW_HAVE_SSE4_2 -DARROW_HDFS -DARROW_JEMALLOC -DARROW_JEMALLOC_INCLUDE_DIR="" -DARROW_PYTHON_EXPORTING -DARROW_WITH_BROTLI -DARROW_WITH_BZ2 -DARROW_WITH_LZ4 -DARROW_WITH_RE2 -DARROW_WITH_SNAPPY -DARROW_WITH_TIMING_TESTS -DARROW_WITH_UTF8PROC -DARROW_WITH_ZLIB -DARROW_WITH_ZSTD -DURI_STATIC_BUILD -DUTF8PROC_STATIC -Isrc -I../src -I../src/generated -isystem ../thirdparty/flatbuffers/include -isystem /usr/include/boost169 -isystem thrift_ep-install/include -isystem protobuf_ep-install/include -isystem jemalloc_ep-prefix/src -isystem xsimd_ep/src/xsimd_ep-install/include -isystem re2_ep-install/include -isystem utf8proc_ep-install/include -isystem ../thirdparty/hadoop/include -isystem orc_ep-install/include -isystem /usr/lib64/python3.6/site-packages/numpy/core/include -isystem /usr/include/python3.6m -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -O3 -DNDEBUG  -Wall -Wno-attributes -msse4.2  -DNDEBUG -fPIC   -std=c++11 -MD -MT src/arrow/python/CMakeFiles/arrow_python_objlib.dir/helpers.cc.o -MF src/arrow/python/CMakeFiles/arrow_python_objlib.dir/helpers.cc.o.d -o src/arrow/python/CMakeFiles/arrow_python_objlib.dir/helpers.cc.o -c ../src/arrow/python/helpers.cc
../src/arrow/python/helpers.cc: In function 'arrow::Result<arrow::py::OwnedRef> arrow::py::internal::{anonymous}::PyObjectToPyInt(PyObject*)':
../src/arrow/python/helpers.cc:170:12: error: cannot bind 'arrow::py::OwnedRef' lvalue to 'arrow::py::OwnedRef&&'
     return ref;
            ^
In file included from ../src/arrow/type.h:30:0,
                 from ../src/arrow/python/helpers.h:32,
                 from ../src/arrow/python/helpers.cc:21:
../src/arrow/result.h:177:3: error:   initializing argument 1 of 'arrow::Result<T>::Result(T&&) [with T = arrow::py::OwnedRef]'
   Result(T&& value) noexcept {  // NOLINT(runtime/explicit)
   ^
../src/arrow/python/helpers.cc:180:12: error: cannot bind 'arrow::py::OwnedRef' lvalue to 'arrow::py::OwnedRef&&'
     return ref;
            ^
In file included from ../src/arrow/type.h:30:0,
                 from ../src/arrow/python/helpers.h:32,
                 from ../src/arrow/python/helpers.cc:21:
../src/arrow/result.h:177:3: error:   initializing argument 1 of 'arrow::Result<T>::Result(T&&) [with T = arrow::py::OwnedRef]'
   Result(T&& value) noexcept {  // NOLINT(runtime/explicit)
   ^

@kszucs
Copy link
Member Author

kszucs commented Oct 18, 2021

@kou that's related to 15f4e56

@kou
Copy link
Member

kou commented Oct 18, 2021

OK. We can work on it in a separated pull request.

@kou
Copy link
Member

kou commented Oct 19, 2021

The centos-7-amd64 failure: ARROW-14369

@kou
Copy link
Member

kou commented Oct 19, 2021

Canceled Azure Pipelines jobs: #11453

@kou
Copy link
Member

kou commented Oct 19, 2021

@kszucs Can we remove empty dev/tasks/conda-recipes/azure.yml ? 15f4e56#diff-59bfcc47236440735af1377f7ba0a9c942972c6996872f33743a7046c9f86835
Is it needed?

@kou
Copy link
Member

kou commented Oct 19, 2021

@github-actions crossbow submit -g nightly

@github-actions
Copy link

Revision: 68c49b7

Submitted crossbow builds: ursacomputing/crossbow @ actions-999

Task Status
almalinux-8-amd64 Github Actions
almalinux-8-arm64 TravisCI
amazon-linux-2-amd64 Github Actions
centos-7-amd64 Github Actions
centos-8-amd64 Github Actions
centos-8-arm64 TravisCI
conda-clean Azure
conda-linux-gcc-py36-arm64 Azure
conda-linux-gcc-py36-cpu-r40 Azure
conda-linux-gcc-py36-cuda Azure
conda-linux-gcc-py37-arm64 Azure
conda-linux-gcc-py37-cpu-r41 Azure
conda-linux-gcc-py37-cuda Azure
conda-linux-gcc-py38-arm64 Azure
conda-linux-gcc-py38-cpu Azure
conda-linux-gcc-py38-cuda Azure
conda-linux-gcc-py39-arm64 Azure
conda-linux-gcc-py39-cpu Azure
conda-linux-gcc-py39-cuda Azure
conda-osx-arm64-clang-py38 Azure
conda-osx-arm64-clang-py39 Azure
conda-osx-clang-py36-r40 Azure
conda-osx-clang-py37-r41 Azure
conda-osx-clang-py38 Azure
conda-osx-clang-py39 Azure
conda-win-vs2017-py36-r40 Azure
conda-win-vs2017-py37-r41 Azure
conda-win-vs2017-py38 Azure
conda-win-vs2017-py39 Azure
debian-bookworm-amd64 Github Actions
debian-bookworm-arm64 TravisCI
debian-bullseye-amd64 Github Actions
debian-bullseye-arm64 TravisCI
debian-buster-amd64 Github Actions
debian-buster-arm64 TravisCI
example-cpp-minimal-build-static Github Actions
example-cpp-minimal-build-static-system-dependency Github Actions
homebrew-cpp Github Actions
homebrew-r-autobrew Github Actions
java-jars Github Actions
nuget Github Actions
python-sdist Github Actions
test-build-vcpkg-win Github Actions
test-conda-cpp Github Actions
test-conda-cpp-valgrind Azure
test-conda-python-3.10 Github Actions
test-conda-python-3.6 Github Actions
test-conda-python-3.6-pandas-0.23 Github Actions
test-conda-python-3.7 Github Actions
test-conda-python-3.7-dask-latest Github Actions
test-conda-python-3.7-dask-master Github Actions
test-conda-python-3.7-hdfs-2.9.2 Github Actions
test-conda-python-3.7-hdfs-3.2.1 Github Actions
test-conda-python-3.7-kartothek-latest Github Actions
test-conda-python-3.7-kartothek-master Github Actions
test-conda-python-3.7-pandas-0.24 Github Actions
test-conda-python-3.7-pandas-latest Github Actions
test-conda-python-3.7-spark-branch-3.0 Github Actions
test-conda-python-3.8 Github Actions
test-conda-python-3.8-hypothesis Github Actions
test-conda-python-3.8-pandas-latest Github Actions
test-conda-python-3.8-pandas-nightly Github Actions
test-conda-python-3.8-spark-master Github Actions
test-conda-python-3.9 Github Actions
test-conda-python-3.9-pandas-master Github Actions
test-debian-11-cpp Github Actions
test-debian-11-go-1.15 Azure
test-debian-11-python-3 Azure
test-debian-c-glib Github Actions
test-debian-ruby Github Actions
test-fedora-33-cpp Github Actions
test-fedora-33-python-3 Azure
test-r-arrow-backwards-compatibility Github Actions
test-r-depsource-auto Azure
test-r-depsource-system Github Actions
test-r-devdocs Github Actions
test-r-gcc-11 Github Actions
test-r-install-local Github Actions
test-r-linux-as-cran Github Actions
test-r-linux-rchk Github Actions
test-r-linux-valgrind Azure
test-r-minimal-build Azure
test-r-offline-maximal Github Actions
test-r-offline-minimal Azure
test-r-rhub-debian-gcc-devel-lto-latest Azure
test-r-rhub-ubuntu-gcc-release-latest Azure
test-r-rocker-r-base-latest Azure
test-r-rstudio-r-base-3.6-bionic Azure
test-r-rstudio-r-base-3.6-centos7-devtoolset-8 Azure
test-r-rstudio-r-base-3.6-centos8 Azure
test-r-rstudio-r-base-3.6-opensuse15 Azure
test-r-rstudio-r-base-3.6-opensuse42 Azure
test-r-ubuntu-21.04 Github Actions
test-r-versions Github Actions
test-ubuntu-18.04-cpp Github Actions
test-ubuntu-18.04-cpp-release Github Actions
test-ubuntu-18.04-cpp-static Github Actions
test-ubuntu-18.04-python-3 Azure
test-ubuntu-18.04-r-sanitizer Azure
test-ubuntu-20.04-cpp Github Actions
test-ubuntu-20.04-cpp-14 Github Actions
test-ubuntu-20.04-cpp-17 Github Actions
test-ubuntu-20.04-cpp-thread-sanitizer Github Actions
test-ubuntu-20.10-docs Azure
test-ubuntu-c-glib Github Actions
test-ubuntu-default-docs Azure
test-ubuntu-ruby Github Actions
ubuntu-bionic-amd64 Github Actions
ubuntu-bionic-arm64 TravisCI
ubuntu-focal-amd64 Github Actions
ubuntu-focal-arm64 TravisCI
ubuntu-hirsute-amd64 Github Actions
ubuntu-hirsute-arm64 TravisCI
wheel-macos-big-sur-cp310-arm64 Github Actions
wheel-macos-big-sur-cp310-universal2 Github Actions
wheel-macos-big-sur-cp38-arm64 Github Actions
wheel-macos-big-sur-cp39-arm64 Github Actions
wheel-macos-big-sur-cp39-universal2 Github Actions
wheel-macos-high-sierra-cp310-amd64 Github Actions
wheel-macos-high-sierra-cp36-amd64 Github Actions
wheel-macos-high-sierra-cp37-amd64 Github Actions
wheel-macos-high-sierra-cp38-amd64 Github Actions
wheel-macos-high-sierra-cp39-amd64 Github Actions
wheel-macos-mavericks-cp310-amd64 Github Actions
wheel-macos-mavericks-cp36-amd64 Github Actions
wheel-macos-mavericks-cp37-amd64 Github Actions
wheel-macos-mavericks-cp38-amd64 Github Actions
wheel-macos-mavericks-cp39-amd64 Github Actions
wheel-manylinux2010-cp310-amd64 Github Actions
wheel-manylinux2010-cp36-amd64 Github Actions
wheel-manylinux2010-cp37-amd64 Github Actions
wheel-manylinux2010-cp38-amd64 Github Actions
wheel-manylinux2010-cp39-amd64 Github Actions
wheel-manylinux2014-cp310-amd64 Github Actions
wheel-manylinux2014-cp310-arm64 TravisCI
wheel-manylinux2014-cp36-amd64 Github Actions
wheel-manylinux2014-cp36-arm64 TravisCI
wheel-manylinux2014-cp37-amd64 Github Actions
wheel-manylinux2014-cp37-arm64 TravisCI
wheel-manylinux2014-cp38-amd64 Github Actions
wheel-manylinux2014-cp38-arm64 TravisCI
wheel-manylinux2014-cp39-amd64 Github Actions
wheel-manylinux2014-cp39-arm64 TravisCI
wheel-windows-cp310-amd64 Github Actions
wheel-windows-cp36-amd64 Github Actions
wheel-windows-cp37-amd64 Github Actions
wheel-windows-cp38-amd64 Github Actions
wheel-windows-cp39-amd64 Github Actions

Copy link
Member

@kou kou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@kou kou closed this in 55a371c Oct 19, 2021
@ursabot
Copy link

ursabot commented Oct 19, 2021

Benchmark runs are scheduled for baseline = b3767f7 and contender = 55a371c. 55a371c is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️0.51% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.4% ⬆️0.18%] ursa-thinkcentre-m75q
Supported benchmarks:
ursa-i9-9960x: langs = Python, R, JavaScript
ursa-thinkcentre-m75q: langs = C++, Java
ec2-t3-xlarge-us-east-2: cloud = True

@kou
Copy link
Member

kou commented Oct 19, 2021

Forgot to update LLVM in .env...

@kszucs
Copy link
Member Author

kszucs commented Oct 19, 2021

@kou thanks for pushing this through!

Forgot to update LLVM in .env...

I think it's fine to use LLVM 12 in the docker images and LLVM 13 on MinGW and macOS so we test more LLVM versions.

projjal pushed a commit to projjal/arrow that referenced this pull request Oct 28, 2021
Closes apache#11448 from kszucs/llvm13

Lead-authored-by: Krisztián Szűcs <szucs.krisztian@gmail.com>
Co-authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
projjal pushed a commit to projjal/arrow that referenced this pull request Dec 8, 2021
Closes apache#11448 from kszucs/llvm13

Lead-authored-by: Krisztián Szűcs <szucs.krisztian@gmail.com>
Co-authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
projjal pushed a commit to projjal/arrow that referenced this pull request Jan 11, 2022
Closes apache#11448 from kszucs/llvm13

Lead-authored-by: Krisztián Szűcs <szucs.krisztian@gmail.com>
Co-authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants