Skip to content

Add arm64-build-checks github action#17989

Merged
alalek merged 1 commit intoopencv:masterfrom
Yikun:arm64-check-action
Aug 18, 2020
Merged

Add arm64-build-checks github action#17989
alalek merged 1 commit intoopencv:masterfrom
Yikun:arm64-check-action

Conversation

@Yikun
Copy link
Copy Markdown
Contributor

@Yikun Yikun commented Jul 30, 2020

This patch adds github action workflow to enable the arm64 build checks.

Related: #17951

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under OpenCV (BSD) License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
  • The PR is proposed to proper branch
  • There is reference to original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@Yikun Yikun mentioned this pull request Jul 30, 2020
Closed
dkurt
dkurt reviewed Jul 30, 2020
View reviewed changes
dkurt
dkurt reviewed Jul 30, 2020
View reviewed changes
@Yikun Yikun force-pushed the arm64-check-action branch from 1249c2a to 758e0ce Compare July 30, 2020 07:51
@dkurt
Copy link
Copy Markdown
Member

dkurt commented Jul 30, 2020

Please take a look at openvinotoolkit/openvino#164. It's not enough to just install native libraries - you need foreign architecture packages.

@Yikun
Copy link
Copy Markdown
Contributor Author

Yikun commented Jul 30, 2020

Please take a look at openvinotoolkit/openvino#164. It's not enough to just install native libraries - you need foreign architecture packages.

Thanks for your info, I propose to change it in next patchset as below:

sed -i -E 's|^deb ([^ ]+) (.*)$|deb [arch=amd64] \1 \2\ndeb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ \2|' /etc/apt/sources.list 
dpkg --add-architecture arm64
apt-get update
apt-get install -y --no-install-recommends \
        crossbuild-essential-arm64 \
        git \
        cmake \
        ca-certificates \
        libpython-dev:arm64 \
        libpython3-dev:arm64 \
        libusb-1.0-0-dev:arm64 \
        python-numpy \
        python3-numpy \
        libgstreamer1.0-dev:arm64 \
        libgstreamer-plugins-base1.0-dev:arm64

mkdir build
cd build
cmake -DBUILD_SHARED_LIBS=ON \
      -DBUILD_PERF_TESTS=ON \
      -DBUILD_TESTS=ON \
      -DWITH_OPENNI2=ON \
      -DWITH_GDAL=ON \
      -DBUILD_EXAMPLES=OFF \
      -DPYTHON_DEFAULT_EXECUTABLE=/usr/bin/python3 \
      -DWITH_OPENCL=OFF \
      -DBUILD_DOCS=OFF \
      -DCMAKE_TOOLCHAIN_FILE=../platforms/linux/aarch64-gnu.toolchain.cmake \
      -DWITH_CUDA=OFF ../

cd build
cmake --build . --config release -- -j4

Any idea?

@mshabunin
Copy link
Copy Markdown
Contributor

mshabunin commented Jul 30, 2020

@Yikun , is is possible to test patches to multiple repositories together (opencv + opencv_contrib + opencv_extra)?

@asmorkalov
Copy link
Copy Markdown
Contributor

asmorkalov commented Jul 30, 2020

@alalek @mshabunin @vpisarev Do we plan to use Github actions?

@Yikun
Copy link
Copy Markdown
Contributor Author

Yikun commented Jul 31, 2020

@Yikun , is is possible to test patches to multiple repositories together (opencv + opencv_contrib + opencv_extra)?

The original thoughts of adding this is for the the build checking(for test, we have the nightly build), so I think opencv + opencv_contrib is ok (build opencv with contrib module).

So I'm not sure should we add some opencv_extra test in here. Could you give some suggestion?

@Yikun
Copy link
Copy Markdown
Contributor Author

Yikun commented Jul 31, 2020 •
edited
Loading

@alalek @mshabunin @vpisarev Do we plan to use Github actions?

@asmorkalov If the community need other multiarch job in here, I'd like to help to add them step by step. : )

@Yikun Yikun force-pushed the arm64-check-action branch 2 times, most recently from ba934cf to a568e3c Compare July 31, 2020 06:48
dkurt
dkurt reviewed Jul 31, 2020
View reviewed changes
dkurt
dkurt reviewed Jul 31, 2020
View reviewed changes
dkurt
dkurt reviewed Jul 31, 2020
View reviewed changes
dkurt
dkurt reviewed Jul 31, 2020
View reviewed changes
@Yikun Yikun force-pushed the arm64-check-action branch 2 times, most recently from b3c8801 to df4123d Compare August 3, 2020 01:52
@Yikun Yikun requested a review from dkurt August 3, 2020 08:33
@Yikun
Copy link
Copy Markdown
Contributor Author

Yikun commented Aug 6, 2020

Any more suggestions?

dkurt
dkurt reviewed Aug 6, 2020
View reviewed changes
dkurt
dkurt reviewed Aug 6, 2020
View reviewed changes
@Yikun Yikun force-pushed the arm64-check-action branch 2 times, most recently from 6655662 to aa019b8 Compare August 6, 2020 11:35
alalek
alalek reviewed Aug 10, 2020
View reviewed changes
.github/workflows/arm64-build-checks.yml Outdated
libpython-dev:arm64 \
libpython3-dev:arm64 \
python-numpy \
python3-numpy \
Copy link
Copy Markdown
Member

@alalek alalek Aug 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

python-numpy
python3-numpy

Python binding is not enabled in cross-compile build (special arm64 variants should be used or extra CMake options should be added).

Copy link
Copy Markdown
Contributor Author

@Yikun Yikun Aug 11, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I try to install libpython-dev:arm64 libpython3-dev:arm64 python-numpy:arm64 python3-numpy:arm64 first,

-DENABLE_PYTHON=ON \
-DPYTHON_EXECUTABLE=/usr/bin/python3.6 \
-DPYTHON_INCLUDE_DIR=/usr/include/python3.6 \
-DPYTHON_LIBRARY=/usr/lib/aarch64-linux-gnu/libpython3.6m.so

And I try to build with above extra options, but looks like the python2/3 binding is still unavailable, did I miss something?

Copy link
Copy Markdown
Member

@dkurt dkurt Aug 11, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are not OpenCV's flags. Use

        -DPYTHON3_INCLUDE_PATH="/usr/include/python3.6" \
        -DPYTHON3_NUMPY_INCLUDE_DIRS="/usr/local/lib/python3.6/dist-packages/numpy/core/include" \
        -DPYTHON3_CVPY_SUFFIX=".cpython-36m-aarch64-linux-gnu.so" \

(make sure you have installed libpython3-dev:armhf and python3-numpy)

Copy link
Copy Markdown
Contributor Author

@Yikun Yikun Aug 11, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dkurt Thanks! it works:

apt-get install -y libpython-dev:arm64 libpython3-dev:arm64 python-numpy python3-numpy
cmake -DPYTHON2_INCLUDE_PATH=/usr/include/python2.7/ \
      -DPYTHON2_NUMPY_INCLUDE_DIRS=/usr/lib/python2.7/dist-packages/numpy/core/include \
      -DPYTHON3_INCLUDE_PATH=/usr/include/python3.6m/ \
      -DPYTHON3_CVPY_SUFFIX=.cpython-36m-aarch64-linux-gnu.so \
      -DPYTHON3_NUMPY_INCLUDE_DIRS=/usr/lib/python3/dist-packages/numpy/core/include \
      -DCMAKE_TOOLCHAIN_FILE=../platforms/linux/aarch64-gnu.toolchain.cmake \
      -DOPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules ../

and python info as below:

--   Python 2:
--     Interpreter:                 /usr/bin/python2.7 (ver 2.7.17)
--     Libraries:
--     numpy:                       /usr/lib/python2.7/dist-packages/numpy/core/include (ver undefined - cannot be probed because of the cross-compilation)
--     install path:                lib/python2.7/dist-packages/cv2/python-2.7
--
--   Python 3:
--     Interpreter:                 /usr/bin/python3 (ver 3.6.9)
--     Libraries:
--     numpy:                       /usr/lib/python3/dist-packages/numpy/core/include (ver undefined - cannot be probed because of the cross-compilation)
--     install path:                lib/python3.6/dist-packages/cv2/python-3.6
--
--   Python (for build):            /usr/bin/python2.7

Looks like we still need the PYTHON2_LIBRARIES=/usr/lib/aarch64-linux-gnu/libpython2.7.so and PYTHON3_LIBRARIES=/usr/lib/aarch64-linux-gnu/libpython3.6m.so, right?

Copy link
Copy Markdown
Member

@dkurt dkurt Aug 11, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cannot say exactly. Can you try to build and check? Which modules are in "to be built"?

Copy link
Copy Markdown
Contributor Author

@Yikun Yikun Aug 11, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

--     To be built:                 aruco bgsegm bioinspired calib3d ccalib core datasets dnn dnn_objdetect dnn_superres dpm face features2d flann fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python2 python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab xfeatures2d ximgproc xobjdetect xphoto
--     Disabled:                    world
--     Disabled by dependency:      -
--     Unavailable:                 alphamat cnn_3dobj cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev cvv freetype hdf java js julia matlab ovis sfm viz

the "to be built" info are same as above(with lib flag or without lib flag), and py2 and py3 is included.

@Yikun Yikun force-pushed the arm64-check-action branch from aa019b8 to 170fda5 Compare August 11, 2020 08:53
dkurt
dkurt reviewed Aug 11, 2020
View reviewed changes
dkurt
dkurt reviewed Aug 11, 2020
View reviewed changes
dkurt
dkurt reviewed Aug 11, 2020
View reviewed changes
dkurt
dkurt reviewed Aug 11, 2020
View reviewed changes
.github/workflows/arm64-build-checks.yml
-DPYTHON3_NUMPY_INCLUDE_DIRS=/usr/lib/python3/dist-packages/numpy/core/include \
-DCMAKE_TOOLCHAIN_FILE=../platforms/linux/aarch64-gnu.toolchain.cmake \
-DOPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules \
../
Copy link
Copy Markdown
Member

@dkurt dkurt Aug 11, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alalek, @mshabunin, does it make sense to enable Actions to test this PR?

Copy link
Copy Markdown
Contributor Author

@Yikun Yikun Aug 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any more idea on it?

Copy link
Copy Markdown
Member

@alalek alalek Aug 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing of PR scripts was performed by @Yikun here:
https://github.com/Yikun/opencv/actions

(Actions can be enabled in repository forks separately)

There are no plans to enable Actions for OpenCV repository or maintain them due to limited resources.

Copy link
Copy Markdown
Contributor Author

@Yikun Yikun Aug 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alalek Thanks for your reply. : )

There are no plans to enable Actions for OpenCV repository or maintain them due to limited resources.

If the "limited resources" refers to the CI resources (build machines), actually, Github provides free resources (container, x86, ubuntu) for the projects which are using github action service. So, we did not need to provide additional resources.

@Yikun Yikun force-pushed the arm64-check-action branch from 170fda5 to a37f7cc Compare August 11, 2020 10:07
@Yikun Yikun requested a review from alalek August 14, 2020 00:54
@alalek alalek merged commit d696d7f into opencv:master Aug 18, 2020
@alalek alalek mentioned this pull request Dec 3, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dkurt dkurt dkurt left review comments

@alalek alalek alalek approved these changes

Assignees

@alalek alalek

Labels

category: build/install platform: arm ARM boards related issues: RPi, NVIDIA TK/TX, etc

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Yikun @dkurt @mshabunin @asmorkalov @alalek