Import and inference INT8 quantized TFLite model#23409
Merged
asmorkalov merged 10 commits intoopencv:4.xfrom Apr 24, 2023
Merged
Import and inference INT8 quantized TFLite model#23409asmorkalov merged 10 commits intoopencv:4.xfrom
asmorkalov merged 10 commits intoopencv:4.xfrom
Conversation
Contributor
|
Ubuntu 20.04: |
8a10784 to
166890b
Compare
| Mat parseTensor(const Tensor& tensor); | ||
|
|
||
| typedef void (TFLiteImporter::*TFLiteImporterNodeParser)(const Operator&, const std::string&, LayerParams&); | ||
| typedef void (TFLiteImporter::*TFLiteImporterNodeParser)(const Operator&, const std::string&, LayerParams&, LayerParams&); |
Contributor
There was a problem hiding this comment.
This should not be a common change.
Many handlers just don't use activParams at all.
Compiler warnings are disabled in dnn module, many warnings are really generated here for that change.
| // Connect layer to inputs | ||
| int i = 0; | ||
| for (int idx : layerInputs) { | ||
| std::vector<DataLayout> inpLayouts; |
Contributor
There was a problem hiding this comment.
Problem of handlers API is here.
We should not have any heavy postprocessing after the handler call.
Post-processing should be done in the handler itself (because handler defines post-processing steps).
Provide a method for registration of DNN layers to be called from handlers.
See ONNX importer implementation for reference.
Member
Author
There was a problem hiding this comment.
Thanks! I reworked both fused activations logic and created an addLayer method. Here is a graph for EfficientDet I used to verify that activations were added correctly: efficientdet.dot (efficientdet.pdf). Unfortunately, for this model they work as identity function and even with unconnected activations, test passes.
opencv-alalek
approved these changes
Apr 24, 2023
Contributor
opencv-alalek
left a comment
There was a problem hiding this comment.
Well done 👍 Thank you!
Member
Author
|
@opencv-alalek, thanks for the review! |
Merged
thewoz
pushed a commit
to thewoz/opencv
that referenced
this pull request
Jan 4, 2024
Import and inference INT8 quantized TFLite model opencv#23409 ### Pull Request Readiness Checklist * Support quantized TFLite models * Enable fused activations (FP32, INT8) **Merge with extra**: opencv/opencv_extra#1048  on the image, green boxes are from TFLite and red boxes from OpenCV See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake
thewoz
pushed a commit
to thewoz/opencv
that referenced
this pull request
May 29, 2024
Import and inference INT8 quantized TFLite model opencv#23409 ### Pull Request Readiness Checklist * Support quantized TFLite models * Enable fused activations (FP32, INT8) **Merge with extra**: opencv/opencv_extra#1048  on the image, green boxes are from TFLite and red boxes from OpenCV See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Readiness Checklist
Merge with extra: opencv/opencv_extra#1048
on the image, green boxes are from TFLite and red boxes from OpenCV
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
Patch to opencv_extra has the same branch name.