Add AVIF support through libavif. by vrabaud · Pull Request #23596 · opencv/opencv

vrabaud · 2023-05-09T13:28:38Z

This is to fix #19271
Extra: opencv/opencv_extra#1069

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
There is a reference to the original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

vrabaud · 2023-05-11T11:59:57Z

I fixed the EXIF that was not handled and I set the output buffers to 16 bit when reading 10/12 bit. We lose the bit depth information though.

modules/imgcodecs/test/test_avif.cpp

cmake/OpenCVFindLibsGrfmt.cmake

CMakeLists.txt

asmorkalov · 2023-05-22T13:13:53Z

Build with default Ubuntu 22.04 libavif package fails with compilation error:

[ 90%] Building CXX object modules/imgcodecs/CMakeFiles/opencv_imgcodecs.dir/src/grfmt_avif.cpp.o
/home/arina/Projects/opencv/modules/imgcodecs/src/grfmt_avif.cpp: In constructor ‘cv::AvifDecoder::AvifDecoder()’:
/home/arina/Projects/opencv/modules/imgcodecs/src/grfmt_avif.cpp:139:27: error: ‘avifDecoder’ {aka ‘struct avifDecoder’} has no member named ‘allowIncremental’
  139 |   if (decoder_) decoder_->allowIncremental = AVIF_TRUE;
      |                           ^~~~~~~~~~~~~~~~
make[3]: *** [modules/imgcodecs/CMakeFiles/opencv_imgcodecs.dir/build.make:104: modules/imgcodecs/CMakeFiles/opencv_imgcodecs.dir/src/grfmt_avif.cpp.o] Error 1

Version info:

arina@arina-pc:~$ apt show libavif-dev
Package: libavif-dev
Version: 0.9.3-3
Priority: optional
Section: universe/libdevel
Source: libavif
Origin: Ubuntu
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Debian Multimedia Maintainers <debian-multimedia@lists.debian.org>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 87,0 kB
Depends: libavif13 (= 0.9.3-3)
Homepage: https://github.com/AOMediaCodec/libavif
Download-Size: 21,0 kB
APT-Manual-Installed: yes
APT-Sources: http://ru.archive.ubuntu.com/ubuntu jammy/universe amd64 Packages
Description: Library for handling .avif files (development files)
 This library aims to be a friendly, portable C implementation of the AV1
 Image File Format as described in https://aomediacodec.github.io/av1-avif/.
 .
 This package provides the development files.

vrabaud · 2023-06-01T14:12:18Z

I split the tests and fixed the compilation on Ubuntu (that API is also useless for us). I added a test for EXIF (that only works with libavif HEAD).

asmorkalov · 2023-06-02T12:43:34Z

@vrabaud Thanks a lot for the update. Looks like you forgot about opencv_extra patch:

[ RUN      ] Imgcodecs_Hdr.regression
imread_('/home/arina/Projects/opencv_extra/testdata/highgui//readwrite/rle.hdr'): can't read header: OpenCV(4.7.0-dev) /home/arina/Projects/opencv/modules/imgcodecs/src/grfmt_avif.cpp:183: error: (-212:Parsing error)  BMFF parsing failed in function 'readHeader'

/home/arina/Projects/opencv/modules/imgcodecs/test/test_grfmt.cpp:373: Failure
Value of: img_rle.empty()
  Actual: true
Expected: false
Could not open /home/arina/Projects/opencv_extra/testdata/highgui//readwrite/rle.hdr
[  FAILED  ] Imgcodecs_Hdr.regression (0 ms)
[ RUN      ] Imgcodecs_Hdr.regression_imencode
imread_('/home/arina/Projects/opencv_extra/testdata/highgui//readwrite/rle.hdr'): can't read header: OpenCV(4.7.0-dev) /home/arina/Projects/opencv/modules/imgcodecs/src/grfmt_avif.cpp:183: error: (-212:Parsing error)  BMFF parsing failed in function 'readHeader'

/home/arina/Projects/opencv/modules/imgcodecs/test/test_grfmt.cpp:396: Failure
Value of: img_ref.empty()
  Actual: true
Expected: false
Could not open /home/arina/Projects/opencv_extra/testdata/highgui//readwrite/rle.hdr
[  FAILED  ] Imgcodecs_Hdr.regression_imencode (0 ms)
....
[ RUN      ] All/Imgcodecs_FileMode.regression/55, where GetParam() = ("readwrite/rle.hdr", -1)
imread_('/home/arina/Projects/opencv_extra/testdata/highgui/readwrite/rle.hdr'): can't read header: OpenCV(4.7.0-dev) /home/arina/Projects/opencv/modules/imgcodecs/src/grfmt_avif.cpp:183: error: (-212:Parsing error)  BMFF parsing failed in function 'readHeader'

/home/arina/Projects/opencv/modules/imgcodecs/test/test_grfmt.cpp:57: Failure
Value of: single.empty()
  Actual: true
Expected: false
[  FAILED  ] All/Imgcodecs_FileMode.regression/55, where GetParam() = ("readwrite/rle.hdr", -1) (0 ms)
[ RUN      ] All/Imgcodecs_FileMode.regression/56, where GetParam() = ("readwrite/rle.hdr", 0)
imread_('/home/arina/Projects/opencv_extra/testdata/highgui/readwrite/rle.hdr'): can't read header: OpenCV(4.7.0-dev) /home/arina/Projects/opencv/modules/imgcodecs/src/grfmt_avif.cpp:183: error: (-212:Parsing error)  BMFF parsing failed in function 'readHeader'

/home/arina/Projects/opencv/modules/imgcodecs/test/test_grfmt.cpp:57: Failure
Value of: single.empty()
  Actual: true
Expected: false
[  FAILED  ] All/Imgcodecs_FileMode.regression/56, where GetParam() = ("readwrite/rle.hdr", 0) (0 ms)
[ RUN      ] All/Imgcodecs_FileMode.regression/57, where GetParam() = ("readwrite/rle.hdr", 1)
imread_('/home/arina/Projects/opencv_extra/testdata/highgui/readwrite/rle.hdr'): can't read header: OpenCV(4.7.0-dev) /home/arina/Projects/opencv/modules/imgcodecs/src/grfmt_avif.cpp:183: error: (-212:Parsing error)  BMFF parsing failed in function 'readHeader'

/home/arina/Projects/opencv/modules/imgcodecs/test/test_grfmt.cpp:57: Failure
Value of: single.empty()
  Actual: true
Expected: false
[  FAILED  ] All/Imgcodecs_FileMode.regression/57, where GetParam() = ("readwrite/rle.hdr", 1) (0 ms)
[ RUN      ] All/Imgcodecs_FileMode.regression/58, where GetParam() = ("readwrite/rle.hdr", 2)
imread_('/home/arina/Projects/opencv_extra/testdata/highgui/readwrite/rle.hdr'): can't read header: OpenCV(4.7.0-dev) /home/arina/Projects/opencv/modules/imgcodecs/src/grfmt_avif.cpp:183: error: (-212:Parsing error)  BMFF parsing failed in function 'readHeader'

/home/arina/Projects/opencv/modules/imgcodecs/test/test_grfmt.cpp:57: Failure
Value of: single.empty()
  Actual: true
Expected: false
[  FAILED  ] All/Imgcodecs_FileMode.regression/58, where GetParam() = ("readwrite/rle.hdr", 2) (1 ms)
[ RUN      ] All/Imgcodecs_FileMode.regression/59, where GetParam() = ("readwrite/rle.hdr", 4)
imread_('/home/arina/Projects/opencv_extra/testdata/highgui/readwrite/rle.hdr'): can't read header: OpenCV(4.7.0-dev) /home/arina/Projects/opencv/modules/imgcodecs/src/grfmt_avif.cpp:183: error: (-212:Parsing error)  BMFF parsing failed in function 'readHeader'

/home/arina/Projects/opencv/modules/imgcodecs/test/test_grfmt.cpp:57: Failure
Value of: single.empty()
  Actual: true
Expected: false
[  FAILED  ] All/Imgcodecs_FileMode.regression/59, where GetParam() = ("readwrite/rle.hdr", 4) (0 ms)
[----------] 60 tests from All/Imgcodecs_FileMode (36845 ms total)

vrabaud · 2023-06-02T13:48:14Z

All imgcodecs tests now pass.

asmorkalov · 2023-06-02T15:55:01Z

All tests passed on my side with Ubuntu 22.04. Thanks a lot! I'll try to look into code in details for the next several days and provide feedback. I also added Ubuntu 22.04 config with libavif on board to CI in opencv/ci-gha-workflow#101 and opencv-infrastructure/opencv-gha-dockerfile#18

cmake/OpenCVFindAVIF.cmake

CMakeLists.txt

modules/imgcodecs/src/grfmt_avif.cpp

modules/imgcodecs/test/test_avif.cpp

vrabaud · 2023-06-08T11:21:00Z

(the CI error seems unrelated: DNN with CUDA error)

asmorkalov

👍

modules/imgcodecs/test/test_avif.cpp

asmorkalov · 2023-06-09T10:45:24Z

@vrabaud We discussed the solution on core team meeting and decided to disable libavif by default for now and treat it's support as experimental till 4.9.0. Could you disable the option by default? I'll add CI configuration a bit later after the release.

vrabaud · 2023-06-09T11:04:10Z

@asmorkalov: sure, done. WITH_AVIF is set to OFF by default.

umbertov · 2023-10-26T08:06:16Z

@asmorkalov @vrabaud Hey, I was reading cmake files and noticed that AVIF support relies on headers & libraries either being present on the system, or built separately and pointed at by the user, while many other 3rd party components like jpeg, png, webp and more have an option (like BUILD_JPEG for libjpeg, for instance) to be built together with opencv and being statically linked into the libopencv_*.so libraries. I assume that they're being statically linked since ldd shows no references to e.g. libjpeg.

Is a BUILD_AVIF option not implemented just because of time & resources reasons, or because of weird licensing issues? Because if it is just due to the former, i might try to work on implementing it.
Thanks in advance for any answer!

asmorkalov · 2023-10-26T10:33:59Z

Major issue is mostly engineering. AVIF has a lot of own dependencies that we do not want to bring into OpenCV. Licenses for AVIF dependence sub-tree is open issue too.

vrabaud · 2023-10-26T12:43:58Z

Licenses are fine: libavif is BSD 2 clause.
Now, it is a thin wrapper around any encoding and/or decoding library.
If we pull in libaom, it is 400 files, dav1d (only for decoding) 120. That is a lot to add to the third_party directory: that was the only rational not to include it.

Add AVIF support through libavif. opencv#23596 This is to fix opencv#19271 Extra: opencv/opencv_extra#1069 ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake

Add AVIF support through libavif. opencv#23596 This is to fix opencv#19271 Extra: opencv/opencv_extra#1069 See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake

vrabaud force-pushed the libavif branch from 6740377 to cce5661 Compare May 9, 2023 13:29

asmorkalov added feature category: imgcodecs labels May 10, 2023

vrabaud force-pushed the libavif branch 2 times, most recently from 39ca51b to 1c4cebf Compare May 10, 2023 13:43

Jenrykster mentioned this pull request May 13, 2023

Avif format support kha-white/mokuro#57

Open

asmorkalov self-requested a review May 22, 2023 10:35

asmorkalov requested changes May 22, 2023

View reviewed changes

asmorkalov reviewed May 22, 2023

View reviewed changes

CMakeLists.txt Outdated Show resolved Hide resolved

asmorkalov added this to the 4.9.0 milestone May 30, 2023

vrabaud force-pushed the libavif branch from 8efcb6d to e55cfcd Compare May 31, 2023 12:42

vrabaud force-pushed the libavif branch from e55cfcd to 3ae5f35 Compare June 1, 2023 14:14

asmorkalov self-assigned this Jun 2, 2023

vrabaud force-pushed the libavif branch 2 times, most recently from 2423ae9 to bd40e00 Compare June 2, 2023 13:47

asmorkalov reviewed Jun 6, 2023

View reviewed changes

vrabaud added 8 commits June 8, 2023 11:40

Add AVIF support through libavif.

734f736

Make find_package QUIET

a7d9d9b

Remove avifinfo

73c2a2c

Add support for EXIF.

3c8104c

Set 10/12 bit buffers to CV_16U

7f5b172

Split tests and fix animation test.

ffd4387

Fix CMake and compilation with old libavif.

20462f7

Add an EXIF test.

125fec7

Fix test.

8d02091

vrabaud force-pushed the libavif branch from bd40e00 to c0faa4d Compare June 8, 2023 09:41

Fix comments.

5ad8410

vrabaud force-pushed the libavif branch from c0faa4d to 5ad8410 Compare June 8, 2023 09:42

asmorkalov approved these changes Jun 8, 2023

View reviewed changes

modules/imgcodecs/test/test_avif.cpp Outdated Show resolved Hide resolved

asmorkalov modified the milestones: 4.9.0, 4.8.0 Jun 8, 2023

Fix identical dimensions in tests.

9ed2f6c

Disable AVIF support by default

f11b0c2

asmorkalov merged commit 472aad4 into opencv:4.x Jun 9, 2023

asmorkalov mentioned this pull request Jul 12, 2023

(5.x) Merge 4.x #23979

Merged

roticv mentioned this pull request Feb 14, 2024

HEIF format support in OpenCV #14534

Open

dlguswo333 mentioned this pull request May 29, 2024

Support more image file formats dlguswo333/find-similar-images#20

Closed

brimoor mentioned this pull request Feb 14, 2025

[FR] Support more image codecs in fiftyone.utils.image.read voxel51/fiftyone#5492

Open

6 tasks

Uh oh!

Conversation

vrabaud commented May 9, 2023 • edited by asmorkalov Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Readiness Checklist

Uh oh!

vrabaud commented May 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

asmorkalov commented May 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vrabaud commented Jun 1, 2023

Uh oh!

asmorkalov commented Jun 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vrabaud commented Jun 2, 2023

Uh oh!

asmorkalov commented Jun 2, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vrabaud commented Jun 8, 2023

Uh oh!

asmorkalov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

asmorkalov commented Jun 9, 2023

Uh oh!

vrabaud commented Jun 9, 2023

Uh oh!

umbertov commented Oct 26, 2023

Uh oh!

asmorkalov commented Oct 26, 2023

Uh oh!

vrabaud commented Oct 26, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vrabaud commented May 9, 2023 •

edited by asmorkalov

Loading

vrabaud commented May 11, 2023 •

edited

Loading

asmorkalov commented May 22, 2023 •

edited

Loading

asmorkalov commented Jun 2, 2023 •

edited

Loading