Skip to content

Registering user-defined logging callback. OE-11 see #11003#27140

Closed
kinchungwong wants to merge 1 commit intoopencv:4.xfrom
kinchungwong:logging_20250324
Closed

Registering user-defined logging callback. OE-11 see #11003#27140
kinchungwong wants to merge 1 commit intoopencv:4.xfrom
kinchungwong:logging_20250324

Conversation

@kinchungwong
Copy link
Copy Markdown
Contributor

@kinchungwong kinchungwong commented Mar 24, 2025

This pullrequest changes

Allow OpenCV user register for logging callback.
Proposal: OE-11 (https://github.com/opencv/opencv/wiki/OE-11.-Logging)
Design discussions: #11003
PR: #27140

Summary:

  • LoggingCallbackHandler added (abstract base class)
    • Allow users to subclass and implement
  • LogCallbackManager (internal singleton)
    • Thread-safety via cv::Mutex
    • add(), remove() are idempotent
  • Wrapper methods for creating a handler
    • Static LoggingCallbackHandler functions
      • createFromFunction (wraps std::function)
      • createFromStaticFuncPtr (wraps raw pointer to static function)
  • Tests
    • core/test/test_logcallbackmanager.cpp
  • Docs
    • Added two Doxygen subgroups under core_logging
      • core_logging_callback_interface
      • core_logging_callback
  • Changes to legacy code
    • TODO: okay to remove from PR
    • LoggingCallbackPtrType (C-style raw pointer, which does not accept file/line/func args)
    • addLoggingCallback(LoggingCallbackPtrType)
    • removeLoggingCallback(LoggingCallbackPtrType)

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • The commits to this PR is up-to-date and squashed
  • The commits to this PR does not contain any code that is still being debated or disagreed with
    Currently, it does. Surrounded with ALLOW_DEPRECATED_CODE and ALLOW_CONTROVERSIAL_CODE

Checklist

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@mshabunin
Copy link
Copy Markdown
Contributor

Maybe we don't need multiple callbacks? Just allowing to set one callback would also allow to customize it according to user's needs: write to multiple locations, filter by level, etc.

I was looking at the logging facilities in Qt: https://doc.qt.io/qt-6/qtlogging.html#qInstallMessageHandler - it is quite simple and customizable.

@kinchungwong
Copy link
Copy Markdown
Contributor Author

Maybe we don't need multiple callbacks? Just allowing to set one callback would also allow to customize it according to user's needs: write to multiple locations, filter by level, etc.

From my experience, libraries and applications that integrate OpenCV often do so across many levels and involving many components. Take torchvision as example.

It is for this reason that I think, each downstream library, and the user application itself, should be entitled to one (and only one) callback registration, so that if they have an urgent need to be self-aware (e.g. to know that something went wrong deep inside OpenCV or one of its third-party dependencies, e.g. video), they have the means to do that, and it wouldn't cause other layers' callback registration to become clobbered.

I agree that any single layer (library or app) that wants multiple callbacks for themselves should handle those details within their own callback handler.

If we are concerned about mutex and unordered_set (and pitfalls of locking / blocking), I have an idea of allowing for a fixed-size hash table (say, 16 entries), and use atomics to get around locking. I can share some code in the next few days. (Not as a PR, I'll probably just share a link to my personal branch.)

Another major question is, whether the callback should receive a single fully-formatted (concatenated) string, or should it be entitled to receive the individual fields. Not receiving the individual fields makes filtering quite difficult. However, I'm also concerned about ABI breakage.

@patrikhuber
Copy link
Copy Markdown
Contributor

Thank you for picking up this work @kinchungwong!

Another major question is, whether the callback should receive a single fully-formatted (concatenated) string, or should it be entitled to receive the individual fields. Not receiving the individual fields makes filtering quite difficult.

I'm not sure what constitutes a "field", but the loglevel should definitely be part of the callback. Basically as a user I'd ideally want to write something simple like this:

void myLogCallback(cv::utils::logging::LogLevel level, std::string msg,
                   const char* func, const char* file, int line)
{
    switch(level)
    {
        case cv::utils::logging::LOG_LEVEL_ERROR:
            spdlog::error("[{}:{}] {}", file, line, msg);
            break;
        case cv::utils::logging::LOG_LEVEL_WARNING:
            spdlog::warn("[{}:{}] {}", file, line, msg);
            break;
        case cv::utils::logging::LOG_LEVEL_INFO:
            spdlog::info("[{}:{}] {}", file, line, msg);
            break;
        default:
            spdlog::debug("[{}:{}] {}", file, line, msg);
            break;
    }
}

int main()
{
    // ...
    cv::utils::logging::setLogCallback(myLogCallback);
    // ...
}

If the func/file/line make things more complicated, then I don't need that - not sure if OpenCV even has those "fields". Just the loglevel and message is fine too.

@kinchungwong
Copy link
Copy Markdown
Contributor Author

kinchungwong commented Mar 25, 2025

void myLogCallback(cv::utils::logging::LogLevel level, std::string msg,
const char* func, const char* file, int line)

@patrikhuber : It does, refer to writeLogMessageEx in logger.hpp

Update: I have reworked this PR into the new design. Refer to PR description or commit comment. I believe this design resolves all of the concerns I know of.

IMHO, other simpler designs have fatal pitfalls, it's not worth pursuing.

Proposal: OE-11
Design discussions: opencv#11003
PR: opencv#27140

Summary:
- LoggingCallbackHandler added (abstract base class)
  - Allow users to subclass and implement
- LogCallbackManager (internal singleton)
  - Thread-safety via cv::Mutex
  - add(), remove() are idempotent
- Wrapper methods for creating a handler
  - Static LoggingCallbackHandler functions
    - createFromFunction (wraps std::function)
    - createFromStaticFuncPtr (wraps raw pointer to static function)
- Tests
  - core/test/test_logcallbackmanager.cpp
- Docs
  - Added two Doxygen subgroups under core_logging
    - core_logging_callback_interface
    - core_logging_callback
- Changes to legacy code
  - TODO: okay to remove from PR
  - LoggingCallbackPtrType (C-style raw pointer, which does not accept file/line/func args)
  - addLoggingCallback(LoggingCallbackPtrType)
  - removeLoggingCallback(LoggingCallbackPtrType)
@kinchungwong kinchungwong marked this pull request as ready for review March 26, 2025 04:15
@kinchungwong
Copy link
Copy Markdown
Contributor Author

kinchungwong commented Mar 26, 2025

I created a PR #27154 for the simple C-style approach as @mshabunin suggested. Since that PR competes with this one, I'd like to hear opinions from everyone.

In particular, I see that there are competing interests from primarily C++ users and primarily C users. C++ wants to use objects, shared pointers, STL containers, and mutexes, while C wants nothing of that. Personally I value correctness and safety above anything else, but I also understands that these values are already shared by both C++ and C users - they ask for idiomatic designs in their language of use, because of code safety concerns.

We probably need some battle-hardened developers who have seen things that we haven't. Particular of interest are: libraries that diamond-aggregate OpenCV (DeepStream?), any platforms not desktop Windows/Linux/Mac, mobile platforms, embedded platforms, novel architecture i.e. Loongson MIPS, risc-v etc.

@kinchungwong kinchungwong changed the title WIP: core logging callback (early draft) OE-11 see #11003 WIP: registering user-defined logging callback. OE-11 see #11003 Mar 26, 2025
@kinchungwong kinchungwong changed the title WIP: registering user-defined logging callback. OE-11 see #11003 Registering user-defined logging callback. OE-11 see #11003 Mar 26, 2025
asmorkalov pushed a commit that referenced this pull request Mar 30, 2025
User-defined logger callback, C-style. #27154

This is a competing PR, an alternative to #27140 

Both functions accept C-style pointer to static functions. Both functions allow restoring the OpenCV built-in implementation by passing in a nullptr.
- replaceWriteLogMessage
- replaceWriteLogMessageEx

This implementation is not compatible with C++ log handler objects.

This implementation has minimal thread safety, in the sense that the function pointer are stored and read atomically. But otherwise, the user-defined static functions must accept calls at all times, even after having been deregistered, because some log calls may have started before deregistering.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
@asmorkalov asmorkalov closed this Apr 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants