Skip to content

CodeCamp #102: support SOLO deployment with OpenVINO#1454

Merged
lvhan028 merged 14 commits intoopen-mmlab:dev-1.xfrom
DDGRCF:solo-openvino-mmdet3.x
Dec 21, 2022
Merged

CodeCamp #102: support SOLO deployment with OpenVINO#1454
lvhan028 merged 14 commits intoopen-mmlab:dev-1.xfrom
DDGRCF:solo-openvino-mmdet3.x

Conversation

@DDGRCF
Copy link
Copy Markdown
Contributor

@DDGRCF DDGRCF commented Nov 28, 2022 •
edited
Loading

Motivation

支持SOLO mmdet3.x Openvino导出

Modification

  • support SOLO mmdet3.x Openvino export

Use cases (Optional)

python tools/deploy.py configs/mmdet/instance-seg/instance-seg_openvino_dynamic-800x1344.py \ 
                                    ${MMDETECTION_DIR}/configs/solo/solo_r50_fpn_1x_coco.py  \
                                    ${MMDETECTION_DIR}/demo/demo.jpg \
                                    --work-dir work_dirs/solo/instance-seg_openvino_dynamic-800x1344 \
                                    --device cpu

Checklist

  1. Pre-commit or other linting tools are used to fix the potential lint issues.
  2. The modification is covered by complete unit tests. If not, please add more unit tests to ensure the correctness.
  3. If the modification has a dependency on downstream projects of a newer version, this PR should be tested with all supported versions of downstream projects.
  4. The documentation has been modified accordingly, like docstring or example tutorials.

@mm-assistant
Copy link
Copy Markdown

mm-assistant bot commented Nov 28, 2022

We recommend using English or English & Chinese for pull requests so that we could have broader discussion.

hanrui1sensetime
hanrui1sensetime reviewed Nov 29, 2022
View reviewed changes
hanrui1sensetime
hanrui1sensetime reviewed Nov 29, 2022
View reviewed changes
@DDGRCF DDGRCF changed the title [超级极视营] 支持SOLO mmdet3.x OpenVINO导出 [超级极客营] 支持SOLO mmdet3.x OpenVINO导出 Nov 29, 2022
@hanrui1sensetime hanrui1sensetime changed the title [超级极客营] 支持SOLO mmdet3.x OpenVINO导出 [超级视客营] 支持SOLO mmdet3.x OpenVINO导出 Nov 30, 2022
@hanrui1sensetime
Copy link
Copy Markdown
Collaborator

hanrui1sensetime commented Nov 30, 2022

Please add unit test for the rewriter, test the performance using tools/test.py, and update the docs.

@DDGRCF DDGRCF changed the title [超级视客营] 支持SOLO mmdet3.x OpenVINO导出 CodeCamp #102 Nov 30, 2022
@hanrui1sensetime
Copy link
Copy Markdown
Collaborator

hanrui1sensetime commented Dec 2, 2022

The ut in the CI is failed, may fix it.

grimoire
grimoire reviewed Dec 6, 2022
View reviewed changes
grimoire
grimoire reviewed Dec 6, 2022
View reviewed changes
grimoire
grimoire reviewed Dec 6, 2022
View reviewed changes
grimoire
grimoire reviewed Dec 6, 2022
View reviewed changes
grimoire
grimoire reviewed Dec 6, 2022
View reviewed changes
grimoire
grimoire reviewed Dec 6, 2022
View reviewed changes
grimoire
grimoire reviewed Dec 6, 2022
View reviewed changes
grimoire
grimoire reviewed Dec 6, 2022
View reviewed changes
grimoire
grimoire reviewed Dec 6, 2022
View reviewed changes
@DDGRCF DDGRCF force-pushed the solo-openvino-mmdet3.x branch 3 times, most recently from eab235c to a7ba988 Compare December 9, 2022 03:10
@DDGRCF DDGRCF force-pushed the solo-openvino-mmdet3.x branch from a7ba988 to 1a2b94e Compare December 9, 2022 03:19
grimoire
grimoire reviewed Dec 9, 2022
View reviewed changes
grimoire
grimoire reviewed Dec 9, 2022
View reviewed changes
grimoire
grimoire reviewed Dec 9, 2022
View reviewed changes
@DDGRCF DDGRCF force-pushed the solo-openvino-mmdet3.x branch from f4a9511 to fbe6376 Compare December 9, 2022 10:39
@DDGRCF DDGRCF force-pushed the solo-openvino-mmdet3.x branch from fbe6376 to 7f956e6 Compare December 9, 2022 10:57
grimoire
grimoire reviewed Dec 12, 2022
View reviewed changes
grimoire
grimoire reviewed Dec 12, 2022
View reviewed changes
grimoire
grimoire reviewed Dec 12, 2022
View reviewed changes
grimoire
grimoire reviewed Dec 12, 2022
View reviewed changes
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Dec 12, 2022 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@grimoire
Copy link
Copy Markdown
Member

grimoire commented Dec 14, 2022

Hi, We have refactor the rewriter. ctx has been move out of the arguments. You can still use ctx with FUNCTION_REWRITER.get_context() in the rewrite function.
And please sign the cla.

grimoire
grimoire approved these changes Dec 16, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@grimoire grimoire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@lvhan028 lvhan028 requested a review from AllentDan December 16, 2022 03:02
@lvhan028 lvhan028 changed the title CodeCamp #102 CodeCamp #102: support SOLO deployment with OpenVINO Dec 16, 2022
AllentDan
AllentDan reviewed Dec 16, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@AllentDan AllentDan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my testing, it failed at onnx2openvino stage. The error log goes like this:

subprocess.CalledProcessError: Command 'mo --input_model="work_dirs/openvino/det/solo/end2end.onnx" --output_dir="work_dirs/openvino/det/solo" --output="dets,labels,masks" --input="input" --input_shape="[1, 3, 800, 1344]" ' returned non-zero exit status 1.

The pipeline was okay for MRCNN FYI.

@DDGRCF
Copy link
Copy Markdown
Contributor Author

DDGRCF commented Dec 17, 2022 •
edited
Loading

In my testing, it failed at onnx2openvino stage. The error log goes like this:

subprocess.CalledProcessError: Command 'mo --input_model="work_dirs/openvino/det/solo/end2end.onnx" --output_dir="work_dirs/openvino/det/solo" --output="dets,labels,masks" --input="input" --input_shape="[1, 3, 800, 1344]" ' returned non-zero exit status 1.

The pipeline was okay for MRCNN FYI.

Strangely! I success in my server. And here is my export information.

12/17 19:54:44 - mmengine - INFO - Args for Model Optimizer: mo --input_model="work_dirs/solo/instance-seg_openvino_dynamic-800x1344/end2end.onnx" --output_dir="work_dirs/solo/instance-seg_openvino_dynamic-800x1344" --output="dets,labels,masks" --input="input" --input_shape="[1, 3, 800, 1344]" 
12/17 19:54:48 - mmengine - INFO - Model Optimizer arguments:
Common parameters:
        - Path to the Input Model:      mmdeploy/work_dirs/solo/instance-seg_openvino_dynamic-800x1344/end2end.onnx
        - Path for generated IR:         mmdeploy/work_dirs/solo/instance-seg_openvino_dynamic-800x1344
        - IR output name:       end2end
        - Log level:    ERROR
        - Batch:        Not specified, inherited from the model
        - Input layers:         input
        - Output layers:        dets,labels,masks
        - Input shapes:         [1, 3, 800, 1344]
        - Source layout:        Not specified
        - Target layout:        Not specified
        - Layout:       Not specified
        - Mean values:  Not specified
        - Scale values:         Not specified
        - Scale factor:         Not specified
        - Precision of IR:      FP32
        - Enable fusing:        True
        - User transformations:         Not specified
        - Reverse input channels:       False
        - Enable IR generation for fixed input shape:   False
        - Use the transformations config file:  None
Advanced parameters:
        - Force the usage of legacy Frontend of Model Optimizer for model conversion into IR:   False
        - Force the usage of new Frontend of Model Optimizer for model conversion into IR:      False
OpenVINO runtime found in:      openvino_2022.2.0.7713/python/python3.7/openvino
OpenVINO runtime version:       2022.2.0-7713-af16ea1d79a-releases/2022/2
Model Optimizer version:        2022.2.0-7713-af16ea1d79a-releases/2022/2
[ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: mmdeploy/work_dirs/solo/instance-seg_openvino_dynamic-800x1344/end2end.xml
[ SUCCESS ] BIN file: mmdeploy/work_dirs/solo/instance-seg_openvino_dynamic-800x1344/end2end.bin
[ SUCCESS ] Total execution time: 2.25 seconds. 
[ SUCCESS ] Memory consumed: 364 MB.

I think 800x1344 may be too large for some computer. Can you test in 300x300 openvino deploy config(I don't have other computer).

@DDGRCF DDGRCF closed this Dec 17, 2022
@DDGRCF DDGRCF reopened this Dec 17, 2022
AllentDan
AllentDan reviewed Dec 20, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@AllentDan AllentDan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested OK in another machine. May update tests/regression/mmdet.yml

AllentDan
AllentDan approved these changes Dec 21, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@AllentDan AllentDan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lvhan028 lvhan028 merged commit 6e91614 into open-mmlab:dev-1.x Dec 21, 2022
lvhan028 pushed a commit that referenced this pull request Mar 1, 2023
@DDGRCF @lvhan028
CodeCamp #102: support SOLO deployment with OpenVINO (#1454)
c057541 
* feat: support solo mmdet3.x openvino 2.0 api

* feat: support solo mmdet3.x openvino

* fix: lint

* fix: add solo head test

* docs: add supported_modesl

* docs: add supported_models

* fix: fix unreasonable code

* fix: fix ci failed

* feat: add linspace func rewrite

* fix: fix unreasonable rewrite linspace__onnx

* fix: change func name from __onnx to __default

* feat: add solo test regression
ewernn pushed a commit to ewernn/mmdeploy that referenced this pull request Nov 17, 2025
@DDGRCF
CodeCamp open-mmlab#102: support SOLO deployment with OpenVINO (open-…
e368e9a 
…mmlab#1454)

* feat: support solo mmdet3.x openvino 2.0 api

* feat: support solo mmdet3.x openvino

* fix: lint

* fix: add solo head test

* docs: add supported_modesl

* docs: add supported_models

* fix: fix unreasonable code

* fix: fix ci failed

* feat: add linspace func rewrite

* fix: fix unreasonable rewrite linspace__onnx

* fix: change func name from __onnx to __default

* feat: add solo test regression
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hanrui1sensetime hanrui1sensetime hanrui1sensetime left review comments

@grimoire grimoire grimoire approved these changes

@AllentDan AllentDan AllentDan approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@DDGRCF @hanrui1sensetime @CLAassistant @grimoire @AllentDan @lvhan028