Skip to content

Let call site know if decoding to a supplied memory buffer succeeded or not#24800

Closed
reunanen wants to merge 1 commit intoopencv:4.xfrom
reunanen:let-call-site-know-if-decoding-to-a-supplied-memory-buffer-succeeded-or-not
Closed

Let call site know if decoding to a supplied memory buffer succeeded or not#24800
reunanen wants to merge 1 commit intoopencv:4.xfrom
reunanen:let-call-site-know-if-decoding-to-a-supplied-memory-buffer-succeeded-or-not

Conversation

@reunanen
Copy link
Copy Markdown
Contributor

Problem: when using the version of imdecode that caters for avoiding memory reallocations, there appears to be no (reliable) way to know if decoding succeeded, and the image data in the supplied memory buffer is valid.

Proposed solution: add a new wrapper around imdecode_: one that takes a required (and not optional) placeholder for the decoded matrix, and returns a bool value indicating if the decoding succeeded, and the image data in the supplied memory buffer is usable to begin with.

Alternative solutions considered:

  • Add a new optional output argument (e.g., having type bool* defaulting to nullptr) to the relevant existing version of imdecode, and set it to the return value of imdecode_. Doing so would however change the function's signature, possibly causing ABI compatibility issues.
  • Change the existing version of imdecode to return an empty Mat (and not *dst) in case the call to imdecode_ returns false (while keeping dst intact). However, doing so might conceivably break some existing applications out there, if they rely on the current behavior.
  • Change imdecode_ to release() its mat argument in all error cases (in particular, when no decoder is found). However, doing so appears to be wasteful; remember that we wanted to avoid memory reallocations in the first place. (Although admittedly this is already happening in case a decoder is found, but it fails to decode the data.)

NB: If any of the above alternative solutions sounds more appealing to the maintainer(s), I am fine switching to one of them. Or if there is a better solution, please let me know.

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
    it looks like 4.x is still the default branch, so it sounds appropriate to me – but if not, then feel free to advise
  • There is a reference to the original bug report and related work
    the problem is described in the PR itself
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
    there are some tests, but no performance tests for example, because arguably not really applicable in this case
  • The feature is well documented and sample code can be built with the project CMake
    there is no sample code, but the feature is kind of trivial

…g memory reallocations, there appears to be no (reliable) way to know if decoding succeeded, and the image data in the supplied memory buffer is valid.

Proposed solution: add a new wrapper around `imdecode_`: one that takes a _required_ (and not optional) placeholder for the decoded matrix, and returns a `bool` value indicating if the decoding succeeded, and the image data in the supplied memory buffer is usable to begin with.

Alternative solutions considered:
- Add a new optional output argument (e.g., having type `bool*` defaulting to `nullptr`) to the relevant existing version of `imdecode`, and set it to the return value of `imdecode_`. Doing so would however change the function's signature, possibly causing ABI compatibility issues.
- Change the existing version of `imdecode` to `return` an empty `Mat` (and not `*dst`) in case the call to `imdecode_` returns `false` (while keeping `dst` intact). However, doing so might conceivably break some existing applications out there, if they rely on the current behavior.
- Change `imdecode_` to `release()` its `mat` argument in _all_ error cases (in particular, when no decoder is found). However, doing so appears to be wasteful; remember that we wanted to avoid memory reallocations in the first place. (Although admittedly this is already happening in case a decoder is found, but it fails to decode the data.)

NB: If any of the above alternative solutions sounds more appealing to the maintainer(s), I am fine switching to one of them. Or if there is a better solution, please let me know.
@reunanen reunanen force-pushed the let-call-site-know-if-decoding-to-a-supplied-memory-buffer-succeeded-or-not branch from 679a346 to d97e4c2 Compare December 31, 2023 10:08
@Kumataro
Copy link
Copy Markdown
Contributor

Kumataro commented Jan 1, 2024

Hello, I believe there is a way to know if decoding succeeded or not.

there appears to be no (reliable) way to know if decoding succeeded,

Is this specification not enough to this requirements ?

https://docs.opencv.org/4.x/d4/da8/group__imgcodecs.html#ga26a67788faa58ade337f8d28ba0eb19e

The function imdecode reads an image from the specified buffer in the memory. If the buffer is too short or contains invalid data, the function returns an empty matrix ( Mat::data==NULL ).

In source code, following lines drops destination mat if decoding is failed,

if (!success)
{
mat.release();
return false;
}

Maybe we can determine it with mat.empty().

Mat src = imdecode(Mat(buf));
if(src.empty()){
  // decoding is failed.
}else{
  // decoding is succeeded.
}    

@reunanen
Copy link
Copy Markdown
Contributor Author

reunanen commented Jan 1, 2024

Hello @Kumataro, thank you for your comments.

Is this specification not enough to this requirements ?

That specification alone seems to be accurate, but do note that I'm interested in the overload of imdecode that allows me to pass dst, so that I can re-use memory that has already been allocated.

In source code, following lines drops destination mat if decoding is failed,

Yes, but not if we return from here or here or here.

Also do note that in my original post, I already mentioned as a third alternative solution that we do mat.release() in all these cases (and not only in the case that you point out).

Mat src = imdecode(Mat(buf));

Yes, this would work, but again do note that I'm interested in the overload of imdecode that allows me to pass dst, so that I can re-use memory that has already been allocated.

@Kumataro
Copy link
Copy Markdown
Contributor

Kumataro commented Jan 1, 2024

@reunanen Thank you for your comment, it is cleared.
I worry that more function overloading may confuse OpenCV users.
To keep back-compatibility, I suggests new idea to use cv::ImreadModes.(Extend 2nd idea).
I would be happy if you could include this as one of your options.

Mat imdecode( InputArray _buf, int flags, Mat* dst )
{
    CV_TRACE_FUNCTION();

    Mat buf = _buf.getMat(), img;
    dst = dst ? dst : &img;
-   imdecode_( buf, flags, *dst );
+   bool result = imdecode_( buf, flags, *dst );
+   if ( (result == false) && ( (flags & IMREAD_XXXX) != 0 )) 
+   {
+     return Mat() ;
+   }
    return *dst;
}

@Kumataro
Copy link
Copy Markdown
Contributor

Kumataro commented Jan 2, 2024

Hello, @reunanen

imdecode() extends with default-argument

We can extend imdecode() function using default-argument to keep backward compatibility for currently users.

diff --git a/modules/imgcodecs/include/opencv2/imgcodecs.hpp b/modules/imgcodecs/include/opencv2/imgcodecs.hpp
index 89bd6e1c1b..132df213c9 100644
--- a/modules/imgcodecs/include/opencv2/imgcodecs.hpp
+++ b/modules/imgcodecs/include/opencv2/imgcodecs.hpp
@@ -323,8 +323,10 @@ CV_EXPORTS_W Mat imdecode( InputArray buf, int flags );
 @param flags The same flags as in cv::imread, see cv::ImreadModes.
 @param dst The optional output placeholder for the decoded matrix. It can save the image
 reallocations when the function is called repeatedly for images of the same size.
+@param result if non-null, the decode result.
+@note When dst is not empty, dst will not be empty if failed. Refer result arguments.
 */
-CV_EXPORTS Mat imdecode( InputArray buf, int flags, Mat* dst);
+CV_EXPORTS Mat imdecode( InputArray buf, int flags, Mat* dst, bool* result = nullptr);

 /** @brief Reads a multi-page image from a buffer in memory.

diff --git a/modules/imgcodecs/src/loadsave.cpp b/modules/imgcodecs/src/loadsave.cpp
index 734f7b516c..93b6ffcde6 100644
--- a/modules/imgcodecs/src/loadsave.cpp
+++ b/modules/imgcodecs/src/loadsave.cpp
@@ -941,13 +941,17 @@ Mat imdecode( InputArray _buf, int flags )
     return img;
 }

-Mat imdecode( InputArray _buf, int flags, Mat* dst )
+Mat imdecode( InputArray _buf, int flags, Mat* dst, bool* result)
 {
     CV_TRACE_FUNCTION();

     Mat buf = _buf.getMat(), img;
     dst = dst ? dst : &img;
-    imdecode_( buf, flags, *dst );
+    bool decode_result = imdecode_( buf, flags, *dst );
+
+    if(result != nullptr) {
+        *result = decode_result;
+    }

     return *dst;
 }

Is this backward compatibility is necessary ?

Conventional users may use the previous image in memory because it does not become empty when decoding fails.

However, since no problems occur when decoding is successful, the impact is limited and has not been considered a problem until now.

These "backward compatibility" are "implementations that do not match the documentation", One aspect, it may not need to be maintained. Rather, it can be said that specifications and implementation should match.

I think the option to always return an empty Mat if decoding fails is less likely to cause problems.

I would like to leave the decision up to the maintainer.

@reunanen
Copy link
Copy Markdown
Contributor Author

reunanen commented Jan 2, 2024

Hello @Kumataro, thank you for your additional comments.

CV_EXPORTS Mat imdecode( InputArray buf, int flags, Mat* dst, bool* result = nullptr);

This is what I was thinking too (basically the first alternative solution in my original post), but I was worried about ABI compatibility. It's not an issue for me, though, but might be for some users? I do not know, so I decided to err on the side of caution.

@asmorkalov
Copy link
Copy Markdown
Contributor

@reunanen OpenCV core team discussed your issue and implemented alternative solution. The existing imdecode implementation was modified to not release user-provided buffer, if decoder was not successful. See #24929. See the test in mentioned PR. Closing the PR.

@asmorkalov asmorkalov closed this Jan 29, 2024
@reunanen
Copy link
Copy Markdown
Contributor Author

@asmorkalov : thank you, works for me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants