[baremetal][aarch64] Build a baremetal OpenCV for AArch64. by fpetrogalli · Pull Request #19921 · opencv/opencv

fpetrogalli · 2021-04-16T10:21:21Z

This patch provide a toolchain file that allows to build the library
for baremetal applications. Minimal changes have been applied to the
code to be able to compile with a baremetal toolchain.

The option CV_BAREMETAL is used to guard the bits in the code that
are baremetal-specific.

The toolchain file requires the path to the baremetal
toolchain. Example cmake invocation:

cmake ../opencv/ \
-DCMAKE_TOOLCHAIN_FILE=../opencv/platforms/baremetal/aarch64.toolchain.cmake \
-DBAREMETAL_TOOLCHAIN_PATH=/absolute/path/to/gcc-baremetal-toolchain/bin/ \
-DBUILD_EXAMPLES=ON

A barematel toolchain for targeting aarch64 can be found at [1], under
aarch64-none-elf.

[1] https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-a/downloads

The current patch covers the following modules

-- OpenCV modules:
-- To be built: core features2d highgui imgcodecs imgproc ml photo video
-- Disabled: flann ts videoio world
-- Disabled by dependency: calib3d objdetect stitching
-- Unavailable: dnn gapi java python2 python3
-- Applications: examples apps

The folder samples/baremetal provides two example baremetal applications.

Acknowledgment

The patch is based on original code written by @l-oneil and @joshsowerby.

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
The PR is proposed to proper branch
There is reference to original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

fpetrogalli · 2021-04-16T10:25:50Z

@vpisarev @alalek, this is a patch we are using to build some components of OpenCV for baremetal execution. We would like to make it available to anyone who needs to deploy baremetal OpenCV. The toolchain file is aarch64 specific, but I believe it could be easily modified to target other architectures. I am posting this as a PR to be able to discuss it with the developers community.

alalek

The patch is really massive.
This build configuration would probably broken too quickly.
Also it is unclear which limitations should be handled (looks like they are related to file-system, threads, mutexes, anything else?)

Consider writing dedicate "port" file for this configuration with stubs for missing functionality. Such port files can be included here:

opencv/modules/core/include/opencv2/core/cvdef.h

Lines 53 to 55 in 29fb4f9

    
           #ifdef OPENCV_INCLUDE_PORT_FILE  // User-provided header file with custom platform configuration 
        
           #include OPENCV_INCLUDE_PORT_FILE 
        
           #endif

Add include directory and name of this file in CMake toolchain.

alalek · 2021-04-20T18:10:53Z

cmake/templates/cvconfig.h.in

+#cmakedefine CV_BAREMETAL
+


Please remove this change.
It is enough to have CMake's add_definitions("-DCV_BAREMETAL")

BTW, There are plans to deprecate cvconfig.h.

modules/core/CMakeLists.txt

modules/core/include/opencv2/core/base.hpp

modules/core/include/opencv2/core/utils/filesystem.private.hpp

modules/core/src/async.cpp

fpetrogalli · 2021-04-21T08:49:38Z

OPENCV_INCLUDE_PORT_FILE

Thank you @alalek - this is very useful information, I didn't know about this possibility.

If I get this right, it means that I can remove all the conditional compilation around CV_BAREMETAL by providing stubs of things like or example std::std::mutex and std::condition_variable, right? I agree this will be a much better solution.

One additional question. I would like to make the port file available upstream. Is there a place in the project where I should store it? If yes and once it is there, how can I guard it from regressions?

alalek · 2021-04-21T15:21:42Z

I would like to make the port file available upstream. Is there a place in the project where I should store it?

I believe it can be stored near CMake toolchain file (toolchain can add extra rules for handling that):

platforms/baremetal/aarch64.toolchain.cmake

e.g. here:

platforms/baremetal/include/aarch64_baremetal_port.hpp

fpetrogalli · 2021-04-21T22:49:21Z

I have marked some comments as resolved, just to keep track of what I am doing locally. The changes will show up when I am done with the modifications.

vpisarev · 2021-04-24T14:25:24Z

@fpetrogalli, sorry for delay.

As I described in our e-mail thread, in order to move forward, we would like to ask you to split the patch into 2, and correspondingly replace CV_BAREMETAL with a combination of specific flags, like CV_DISABLE_MULTI_THREADING, OPENCV_HAVE_FILESYSTEM_SUPPORT=OFF etc.

the first patch, which is the biggest part of your current pull request, should just introduce a way to disable multi-threading.
the second patch can contain everything else.

PR: opencv#19921 Changes: 1. Add a baremetal port headerfile. 2. Removed the conditional compilation of: - cubic root; - CV_CXX11 specific code; - pthread code in system.cpp; - `mutex` code in logtagmanager.hpp; - `std::thread` code in parallel.cpp. 3. Move the setting OPENCV_HAVE_FILESYSTEM_SUPPORT to zero in the header file of the port.

fpetrogalli · 2021-04-26T10:02:20Z

@alalek / @vpisarev - I have updated the PR with the code I was working on last week. I'll use this last version as the starting point for the split you have requested in the email thread and in the last comment because it already addresses some of the issues that @alalek reported. Please consider the current status of this PR as a WIP, it does not need to be reviewed.

fpetrogalli · 2021-04-26T16:09:20Z

Hello - I have a question about the CV_DISABLE_MULTI_THREAD option I am working on.

There are situations in which disabling #include <thread> is straightforward, as for example in the CV_CXX11-guarded code in [1].

[1] https://github.com/opencv/opencv/blob/master/modules/core/src/parallel.cpp.

In other places [2] the code seems to be designed to be multi threaded (please correct me if I am wrong). For places like these, would it make sense to just disable the module (videoio in case of [2]), instead of coming up with a version of the code that does not use threads? At the moment we are happy to have baremetal compilation of some core parts of OpenCV, not the whole library. We could disable multi-threaded code in these conditionally disabled modules on a per need basis in the future. Would that be OK?

[2] https://github.com/opencv/opencv/blob/master/modules/videoio/src/cap_gstreamer.cpp

alalek · 2021-04-26T19:43:59Z

I believe stubs should be created in the #else branch to handle these unsupported cases (which throws CV_Error(Error::StsNotImplemented, ...))

Disabling videoio should be OK for this mode.

PR: opencv#19921 Changes: 1. Add a baremetal port headerfile. 2. Removed the conditional compilation of: - cubic root; - CV_CXX11 specific code; - pthread code in system.cpp; - `mutex` code in logtagmanager.hpp; - `std::thread` code in parallel.cpp. 3. Move the setting OPENCV_HAVE_FILESYSTEM_SUPPORT to zero in the header file of the port.

fpetrogalli · 2021-04-28T17:17:26Z

Please notice that I have force-rebased this PR on top of a1d7e90, which is the tip of the first feature I factored out from the original patch (the one that adds CV_DISABLE_MULTI_THREAD, IN #19985 )

I will add more commits to make use of the macro at #19985 and remove CV_BAREMETAL.

Please don't review this patch in its totality, as it is a WIP while I work on #19985 .

Please let me know if recommend a different approach instead of this workflow (rebase on top of #19985).

PR: opencv#19921 Changes: 1. Add a baremetal port headerfile. 2. Removed the conditional compilation of: - cubic root; - CV_CXX11 specific code; - pthread code in system.cpp; - `mutex` code in logtagmanager.hpp; - `std::thread` code in parallel.cpp. 3. Move the setting OPENCV_HAVE_FILESYSTEM_SUPPORT to zero in the header file of the port.

fpetrogalli · 2021-04-28T17:40:43Z

Once again, a rebase on top of 0c5842e (current HEAD of #19985).

PR: opencv#19921 Changes: 1. Add a baremetal port headerfile. 2. Removed the conditional compilation of: - cubic root; - CV_CXX11 specific code; - pthread code in system.cpp; - `mutex` code in logtagmanager.hpp; - `std::thread` code in parallel.cpp. 3. Move the setting OPENCV_HAVE_FILESYSTEM_SUPPORT to zero in the header file of the port.

modules/core/src/parallel.cpp

fpetrogalli · 2021-05-11T06:54:55Z

modules/core/src/utils/logtagmanager.hpp

+#else // CV_DISABLE_MULTI_THREAD
+    // Custom (failing) implementation of `std::mutex`.
+    struct MutexType {
+        void lock(){


@alalek @vpisarev - I am seeing failures like the following in my baremetal runs. I have poked around a bit to see if there is a configure option that allows turning off logging, but I haven't found one so far. Should I come up with a "turn off logging" cmake option? Or is there already one and am I simply missing it?

Francesco

terminate called after throwing an instance of 'cv::Exception' what(): OpenCV(4.5.2-dev) /home/frapet01/projects/opencv/opencv/modules/core/src/utils/logtagmanager.hpp:46: error: (-213:The function/feature is not implemented) cv::Mutex are disabled by CV_DISABLE_MULTI_THREAD=ON in function 'lock'

Never mind, I found the macro that needs disabling: #define CV_LOG_WITH_TAG(tag, msgLevel, extra_check0, extra_check1, ...)

Should I disable it under the PR that disables filesystem support, at #20010 , or should I create a CV_DISABLE_LOGGING option? Please let me know what you want me to do, for now I'll just push an extra commit in this PR that disables it under OPENCV_HAVE_FILESYSTEM_SUPPORT.

Disabling logging is not a best option.

Consider replacing this part:

- using LockType = std::lock_guard<MutexType>; + using LockType = cv::AutoLock;

Using cv::AutoLock would require to implement stubs for cv::AutoLock, which seems to raise issues similar to implementing stub for things like mutexes and threads.

I think it would make sense to have a separate option that disable logging if invoked at command line configuration. That would be turned on only manually when targeting baremetal. I agree that disabling logging might not be ideal in the general case, but for baremetal there is no filesystem to log to.

Alternatively, if you really think we should use cv-AutoLock, I need some help in figuring out what to do. In #19985 the AutoLock/Mutex stubs are defined as follows:

#ifndef CV_DISABLE_MULTI_THREAD typedef std::recursive_mutex Mutex; typedef std::lock_guard<cv::Mutex> AutoLock; #else // CV_DISABLE_MULTI_THREAD // Custom (failing) implementation of `std::recursive_mutex`. struct Mutex { void lock(){ CV_Error(cv::Error::StsNotImplemented, "cv::Mutex are disabled by CV_DISABLE_MULTI_THREAD=ON"); } void unlock(){ CV_Error(cv::Error::StsNotImplemented, "cv::Mutex are disabled by CV_DISABLE_MULTI_THREAD=ON"); } }; // Stub for cv::AutoLock when threads are disabled. struct AutoLock { AutoLock(Mutex &) { } }; #endif // CV_DISABLE_MULTI_THREAD

What should I based the implementation of AutoLock on? As you can see, even if I use cv::AutoLock, I would still be getting exceptions from the stubs of cv::Mutex.

Right, "no-op" AutoLock stub is required for single-threaded mode ( #19985 (comment) ).

Ah, right! Thanks @alalek . I forgot about the stubs for AutoLock. With this change, it works, no more exceptions :) I have updated the patch for the single thread execution with the suggestion, see #19985

I have also rebased this one to use that new commit.

One thing that is not clear to me is the reason why we have to raise exceptions for the method of cv::Mutex, and just use stubs for cv::AutoLock. I am asking because I would like to add a comment in the code of #19985 explaining this, as other people my ask the same question when seeing the stub and the exception.

The changes in this patch are needed to have the code at opencv#19921 build again. * Move the changes in cmake/OpenCVUtils.cmake into the main CmakeLists.txt file. It seems that the setting underneath the option `OPENCV_DISABLE_THREAD_SUPPORT` need to be set after all the options have been configured, otherwise they won't be picked up. This happens only in combination with the toolchain file needed in the baremetal builds. * Replace `ocv_update` with `ocv_force_update`. The variables need to be re-written in case they have already been set by other configuration steps. The previous invocation `ocv_update` was leaving the variables untouched if already set.

The option forces the library to build without thread support.

- reduce amount of #if conditions

This patch provide a toolchain file that allows to build the library for baremetal applications. Minimal changes have been applied to the code to be able to compile with a baremetal toolchain. The option `CV_BAREMETAL` is used to guard the bits in the code that are baremetal-specific. The toolchain file requires the path to the baremetal toolchain. Example cmake invocation: cmake ../opencv/ \ -DCMAKE_TOOLCHAIN_FILE=../opencv/platforms/baremetal/aarch64.toolchain.cmake \ -DBAREMETAL_TOOLCHAIN_PATH=/absolute/path/to/gcc-baremetal-toolchain/bin/ \ -DBUILD_EXAMPLES=ON A barematel toolchain for targeting aarch64 can be found at [1], under `aarch64-none-elf`. [1] https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-a/downloads The folder `samples/baremetal` provides two example baremetal applications.

1. Disable threads in openjpeg from the toolchain file. 2. Have the perf tests to build.

…when the filesystem is disabled. This reverts commit fafcd25.

fpetrogalli · 2021-07-12T13:08:52Z

Replaced by #20392

alalek reviewed Apr 20, 2021

View reviewed changes

fpetrogalli force-pushed the aarch64-baremetal branch from f2e7320 to e4230c4 Compare April 26, 2021 09:59

fpetrogalli marked this pull request as draft April 26, 2021 10:02

fpetrogalli mentioned this pull request Apr 26, 2021

[build][option] Introduce OPENCV_DISABLE_THREAD_SUPPORT option. #19985

Merged

6 tasks

fpetrogalli force-pushed the aarch64-baremetal branch from e4230c4 to 55ffded Compare April 28, 2021 17:03

fpetrogalli force-pushed the aarch64-baremetal branch from 55ffded to c86ae05 Compare April 28, 2021 17:11

fpetrogalli force-pushed the aarch64-baremetal branch from c86ae05 to 0e5ef17 Compare April 28, 2021 17:39

fpetrogalli force-pushed the aarch64-baremetal branch from 0e5ef17 to dae9254 Compare April 29, 2021 15:39

fpetrogalli mentioned this pull request Apr 29, 2021

[build][option] Build option to disable filesystem support. #20010

Merged

6 tasks

fpetrogalli force-pushed the aarch64-baremetal branch from dae9254 to ae6b6f2 Compare April 29, 2021 22:31

fpetrogalli force-pushed the aarch64-baremetal branch from ae6b6f2 to 9c363cd Compare April 30, 2021 08:38

fpetrogalli force-pushed the aarch64-baremetal branch from 9c363cd to 33c4c73 Compare April 30, 2021 11:01

fpetrogalli force-pushed the aarch64-baremetal branch from 905f088 to 3f50e14 Compare May 7, 2021 09:48

fpetrogalli commented May 7, 2021

View reviewed changes

modules/core/src/parallel.cpp Show resolved Hide resolved

fpetrogalli force-pushed the aarch64-baremetal branch from 3f50e14 to 4af249d Compare May 7, 2021 16:57

fpetrogalli commented May 11, 2021

View reviewed changes

fpetrogalli force-pushed the aarch64-baremetal branch 3 times, most recently from 2028358 to 2967c8f Compare May 17, 2021 16:59

fpetrogalli force-pushed the aarch64-baremetal branch 3 times, most recently from 90d5f9d to 1100ece Compare May 27, 2021 17:02

fpetrogalli force-pushed the aarch64-baremetal branch 4 times, most recently from d7da79a to a85dadd Compare June 21, 2021 10:50

fpetrogalli force-pushed the aarch64-baremetal branch from a85dadd to 2f0e78e Compare July 7, 2021 16:40

Francesco Petrogalli and others added 3 commits July 8, 2021 10:07

[build][option] Introduce OPENCV_DISABLE_THREAD_SUPPORT option.

01049be

The option forces the library to build without thread support.

update handling of OPENCV_DISABLE_THREAD_SUPPORT

8630492

- reduce amount of #if conditions

[to squash] cmake: apply mode vars in toolchains too

0d819b1

fpetrogalli force-pushed the aarch64-baremetal branch from 2f0e78e to fe56fbf Compare July 8, 2021 10:11

Francesco Petrogalli added 5 commits July 8, 2021 14:06

[to squash] Fix up on the original patch.

4a31bfe

1. Disable threads in openjpeg from the toolchain file. 2. Have the perf tests to build.

[to_squash] Prevent filesystem checks that assert in the test system …

e0f4b2f

…when the filesystem is disabled. This reverts commit fafcd25.

[to squash] Remove patch file erroneously tracked in git

58fd968

[to squash] Fix build issue on non-baremetal.

174b6b2

fpetrogalli force-pushed the aarch64-baremetal branch from fe56fbf to 174b6b2 Compare July 8, 2021 14:10

alalek mentioned this pull request Jul 12, 2021

AArch64 semihosting #20392

Merged

6 tasks

fpetrogalli closed this Jul 12, 2021

fpetrogalli deleted the aarch64-baremetal branch July 19, 2021 14:48

	#ifdef OPENCV_INCLUDE_PORT_FILE // User-provided header file with custom platform configuration
	#include OPENCV_INCLUDE_PORT_FILE
	#endif

		#cmakedefine CV_BAREMETAL

Uh oh!

Conversation

fpetrogalli commented Apr 16, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Acknowledgment

Pull Request Readiness Checklist

Uh oh!

fpetrogalli commented Apr 16, 2021

Uh oh!

alalek left a comment

Choose a reason for hiding this comment

Uh oh!

alalek Apr 20, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fpetrogalli commented Apr 21, 2021

Uh oh!

alalek commented Apr 21, 2021

Uh oh!

fpetrogalli commented Apr 21, 2021

Uh oh!

vpisarev commented Apr 24, 2021

Uh oh!

fpetrogalli commented Apr 26, 2021

Uh oh!

fpetrogalli commented Apr 26, 2021

Uh oh!

alalek commented Apr 26, 2021

Uh oh!

fpetrogalli commented Apr 28, 2021

Uh oh!

fpetrogalli commented Apr 28, 2021

Uh oh!

Uh oh!

fpetrogalli May 11, 2021

Choose a reason for hiding this comment

Uh oh!

fpetrogalli May 11, 2021

Choose a reason for hiding this comment

Uh oh!

alalek May 11, 2021

Choose a reason for hiding this comment

Uh oh!

fpetrogalli May 11, 2021

Choose a reason for hiding this comment

Uh oh!

alalek May 11, 2021

Choose a reason for hiding this comment

Uh oh!

fpetrogalli May 12, 2021

Choose a reason for hiding this comment

Uh oh!

fpetrogalli commented Jul 12, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

fpetrogalli commented Apr 16, 2021 •

edited

Loading