Parsing quantized nodes does not rely on names#22531
Conversation
|
@zihaomu I propose to touch some of quantized models (per-layer test) to cover new case with test too. |
Hi @asmorkalov, thanks for your code reviewing. This patch wants to stop rely on the node name when we parse the quantized node. And it doesn't introduce new features. In my opinion, the existing test cases are enough. |
rogday
left a comment
There was a problem hiding this comment.
Thank you for looking into this!
| CV_Error(Error::StsNotImplemented, "Per-channel scales/zeropoints are not supported"); | ||
| if (hasVariableInps) | ||
| { | ||
| LayerInfo layerInfo = layer_id.find(node_proto.input(i))->second; |
There was a problem hiding this comment.
So, you look for any real input layer and check if we created internal layer *Int8? I mean, someone could rename the layer or add a new one that doesn't follow that convention. Maybe we're better off with layerParams.set("depth", CV_8S) in each layer, or we could create a set of all integer layers. I think this should be discussed further with the rest of the team.
There was a problem hiding this comment.
Thanks for your code review. OpenCV DNN needs to allocate output blob according to depth type (for example the depth of CV_8S will be allocated as int8 Mat.)
someone could rename the layer or add a new one that doesn't follow that convention.
It is possible. I agree with create a set of all integer layers. I think a better way is for us to have both Int8 name endings and keep sets. The set only stores some special layers that do not follow the rules like Quantize. For other layer type ending with Int8, we can keep the current implementation.
There was a problem hiding this comment.
I'm not sure relying on internal names is a good idea. Let's discuss this on Friday, maybe someone will come up with a better solution.
There was a problem hiding this comment.
Hi @rogday, please take a look. I have refactored the code.
b3276e6 to
358edb7
Compare
| Ptr<Layer> preLayer = dstNet.getLayer(layerInfo.layerId); | ||
|
|
||
| for (int i = 0; i < node_proto.output_size(); i++) | ||
| if (layerInfo.depth == CV_8S && hasInt8Input(preLayer->type)) |
There was a problem hiding this comment.
So if we insert unsupported at the moment layer(Gather, for example) after integer node, it will set the depth to CV_8S and corrupt the memory at runtime? (reading int8 as float in this case).
There was a problem hiding this comment.
What if we have a set of ONNX layers instead? e.g. in this set we would put QuantizeLinear, DequantizeLinear, QLinearConv, Transpose, Reshape, MaxPool, etc.
And check that if we have integer inputs, current layer should be in this set, otherwise we throw an error, what do you think?
There was a problem hiding this comment.
So if we insert unsupported at the moment layer(Gather, for example) after integer node, it will set the depth to CV_8S and corrupt the memory at runtime? (reading int8 as float in this case).
Yes, if we do this, it will corrupt. But this should not happen, Gather-like layer should output the same format as the input. So, if a Gather layer gets int8 input, it should be GatherInt8. And if the type of GatherInt8 layer is Gather, we can add the Gather to the list.
There was a problem hiding this comment.
What if we have a set of ONNX layers instead? e.g. in this set we would put QuantizeLinear, DequantizeLinear, QLinearConv, Transpose, Reshape, MaxPool, etc.
And check that if we have integer inputs, the current layer should be in this set, otherwise we throw an error, what do you think?
I also considered a list of ONNX operators. Functionally, there is no difference between the two lists in essence, one is the original version of ONNX and the other is the mapped version of OpenCV.
Considering the future maintainability, indeed using the original version of ONNX operator names is better.
|
@zihaomu friendly reminder. |
|
@asmorkalov, Sorry for the late reply, I'm on vacation recently and will be back at work tomorrow. |
358edb7 to
d9eff7d
Compare
|
Discussed testing offline. Decided to merge without test. |
Related issues #22509 and #22442.
Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
Patch to opencv_extra has the same branch name.