Changing data_layout for pack in tf_importer#22249
Conversation
|
Hi @fengyuentau, you can use generate_tf_models.py or generate_tf2_models.py to export specific pb model. |
| if (dim != 0) | ||
| CV_Error(Error::StsNotImplemented, "Unsupported mode of pack operation."); | ||
|
|
||
| data_layouts[name] = DATA_LAYOUT_UNKNOWN; |
There was a problem hiding this comment.
In tf_importer, some layers are sensitive to the input's data layout (like Transpose). And these layers may error if they are after the Pack layer.
Maybe we can judge the output data layout by the input data layout.
For example, the input is 4 dim and NHWC data layout, which means the output should be the 5 dims.
But I'm not sure if this is the right way to do it.
There was a problem hiding this comment.
We can consider cases that have defined data layout in tf_importer, for example stacking hwc can have nhwc data layout output. The problem is we cannot take every single data layout into account since theoratically there is countless.
I wonder in which case pack is followed by transpose though.
There was a problem hiding this comment.
I think we keep it unknown for the output data layout of pack since there are only 4 data layouts defined:
enum DataLayout
{
DATA_LAYOUT_NHWC,
DATA_LAYOUT_NCHW,
DATA_LAYOUT_NDHWC,
DATA_LAYOUT_UNKNOWN,
DATA_LAYOUT_PLANAR // 2-dimensional outputs (matmul, flatten, reshape to 2d)
};These data layouts do not satisfy our need if we want to describe the output data layout of pack unless more are added.
There was a problem hiding this comment.
A complete test example would be helpful.
From stack API, we can find that the axis is [-(R+1), R+1).
The following code generates all possible cases of tf.stack.
for d in range(1, 7):
for axis in range(-d - 1, d + 1):
inp0 = tf.placeholder(tf.float32, list(range(1, d + 1)), 'input0')
constant = tf.random_normal(list(range(1, d + 1)))
pack_model = tf.stack([inp0, constant])
print(pack_model)
# save(inp0, pack_model, f'pack_dim{d}_{axis}', optimize=False)
Maybe in tf.stackcase, we should use two placeholders as model input. I'm not sure the save function in generate_tf_models.py can do this.
There was a problem hiding this comment.
Testing a single Pack operator does not make any sense in this pull request. This pull request is changing the output data layout, and with or without this change, such tests will always pass.
Fixes #22221.
In this case, the
packoperator stacks three 4-dimensional tensors into a 5-dimensional tensor. However, the data layout of the output of thepackoperator is still recognized as 4-dimensional (DATA_LAYOUT_NHWC) in tf_importer, which will lead to some problems like #22221 . Changing the data layout toDATA_LAYOUT_UNKNOWNfixes the problem.For testing, I doubt whether we should put the model in opencv_extra since it is ~432MB.
Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
Patch to opencv_extra has the same branch name.