Skip to content

[Feature] Support NMS with cambricon MLU backend#1437

Closed
russ-spec wants to merge 18 commits intoopen-mmlab:mlufrom
russ-spec:mlu_nms
Closed

[Feature] Support NMS with cambricon MLU backend#1437
russ-spec wants to merge 18 commits intoopen-mmlab:mlufrom
russ-spec:mlu_nms

Conversation

@russ-spec
Copy link
Copy Markdown
Contributor

@russ-spec russ-spec commented Oct 27, 2021

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

The motivation of the PR is to support running NMS op with Cambricon MLU backend.

It includes three parts:

  • Refactor nms.cpp to support dispatching to MLU.
  • Add nms_mlu.cpp and nms_mlu_internal.mlu which is the implentation of NMS op.
  • Refactor test_nms.py to support testing NMS op with MLU backend.

Modification

  1. Dispatch
    -Add dispatching to NMSMLUKernelLauncher in nms.cpp.
  2. MLU source code
    -Add MLU source code of NMS in directory mmcv/ops/scrc/pytorch/mlu.
  3. Unit test
    -Support test NMS with various backends in test_nms.py.

BC-breaking (Optional)

Does the modification introduce changes that break the backward-compatibility of the downstream repositories?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

Use cases (Optional)

Please refer to mmcv/tests/test_ops/test_nms.py

Checklist

Before PR:

  • I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
  • Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
  • Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
  • New functionalities are covered by complete unit tests. If not, please add more unit test to ensure the correctness.
  • The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

  • If the modification has potential influence on downstream or other related projects, this PR should be tested with some of those projects, like MMDet or MMCls.
  • CLA has been signed and all committers have signed the CLA in this PR.

DCNSW and others added 10 commits October 21, 2021 20:00
@DCNSW @zhouzaida
[Feature] Add roiaware pool3d ops from mmdet3d (open-mmlab#1382)
4c8bfb4 
* add ops (roiaware pool3d) in mmdet3d

* refactor code

* fix typo

Co-authored-by: zhouzaida <zhouzaida@163.com>
@DCNSW @zhouzaida
[Feature] Add iou3d op from mmdet3d (open-mmlab#1356)
2157b70 
* add ops (iou3d) in mmdet3d

* add unit test

* refactor code

* refactor code

* refactor code

* refactor code

* refactor code

Co-authored-by: zhouzaida <zhouzaida@163.com>
@zhouzaida
[Fix] Update test data for test_iou3d (open-mmlab#1427)
6935a81 
* Update test data for test_iou3d

* delete blank lines

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
@DCNSW @zhouzaida
[Feature] Add group points ops from mmdet3d (open-mmlab#1415)
75cae78 
* add op (group points) and its related ops (ball query and knn) in mmdet3d

* refactor code

* fix typo

* refactor code

* fix typo

* refactor code

* make input contiguous

Co-authored-by: zhouzaida <zhouzaida@163.com>
@luopeichao @zhouzaida
add mmdet3d op (open-mmlab#1425)
2d73eaf 
Co-authored-by: zhouzaida <zhouzaida@163.com>
@zhouzaida
[Feature] Loading objects from different backends and dumping objects…
5b5b47f 
… to different backends (open-mmlab#1330)

* [Feature] Choose storage backend by the prefix of filepath

* refactor FileClient and add unittest

* support loading from different backends

* polish docstring

* fix unittet

* rename attribute str_like_obj to is_str_like_obj

* add infer_client method

* add check_exist method

* rename var client to file_client

* polish docstring

* add join_paths method

* remove join_paths and add _format_path

* enhance unittest

* refactor unittest

* singleton pattern

* fix test_clientio.py

* deprecate CephBackend

* enhance docstring

* refactor unittest for petrel

* refactor unittest for disk backend

* update io.md

* add concat_paths method

* improve docstring

* improve docstring

* add isdir and copyfile for file backend

* delete copyfile and add get_local_path

* remove isdir method of petrel

* fix typo

* add comment and polish docstring

* polish docstring

* rename _path_mapping to _map_path

* polish docstring and fix typo

* refactor get_local_path

* add list_dir_or_file for FileClient

* add list_dir_or_file for PetrelBackend

* fix windows ci

* Add return docstring

* polish docstring

* fix typo

* fix typo

* deprecate the conversion from Path to str

* add docs for loading checkpoints with FileClient

* refactor map_path

* add _ensure_methods to ensure methods have been implemented

* fix list_dir_or_file

* rename _ensure_method_implemented to has_method
@ZwwWayne
Add CI for pytorch 1.10 (open-mmlab#1431)
ef02219
@zhouzaida
[Feature] Upload checkpoints and logs to ceph (open-mmlab#1375)
32e09f4 
* [Feature] Choose storage backend by the prefix of filepath

* refactor FileClient and add unittest

* support loading from different backends

* polish docstring

* fix unittet

* rename attribute str_like_obj to is_str_like_obj

* [Docs] Upload checkpoint to petrel oss

* add infer_client method

* Support uploading checkpoint to petrel oss

* add check_exist method

* refactor CheckpointHook

* support uploading logs to ceph

* rename var client to file_client

* polish docstring

* enhance load_from_ceph

* refactor load_from_ceph

* refactor TextLoggerHook

* change the meaning of out_dir argument

* fix test_checkpoint_hook.py

* add join_paths method

* remove join_paths and add _format_path

* enhance unittest

* refactor unittest

* add a unittest for EvalHook when file backend is petrel

* singleton pattern

* fix test_clientio.py

* deprecate CephBackend

* add warning in load_from_ceph

* fix type of out_suffix

* enhance docstring

* refactor unittest for petrel

* refactor unittest for disk backend

* update io.md

* add concat_paths method

* fix CI

* mock check_exist

* improve docstring

* improve docstring

* improve docstring

* improve docstring

* add isdir and copyfile for file backend

* delete copyfile and add get_local_path

* remove isdir method of petrel

* fix typo

* rename check_exists to exists

* refactor code and polish docstring

* fix windows ci

* add comment and polish docstring

* polish docstring

* polish docstring

* rename _path_mapping to _map_path

* polish docstring and fix typo

* refactor get_local_path

* add list_dir_or_file for FileClient

* add list_dir_or_file for PetrelBackend

* fix windows ci

* Add return docstring

* polish docstring

* fix typo

* fix typo

* fix typo

* fix error when mocking PetrelBackend

* deprecate the conversion from Path to str

* add docs for loading checkpoints with FileClient

* rename keep_log to keep_local

* refactor map_path

* add _ensure_methods to ensure methods have been implemented

* fix list_dir_or_file

* rename _ensure_method_implemented to has_method

* refactor

* polish information

* format information
@zhouzaida
bump version to v1.3.16 (open-mmlab#1430)
f9be3bc
@russ-spec
[Feature] Support NMS with cambricon MLU backend
e2d5c27
@russ-spec
Copy link
Copy Markdown
Contributor Author

russ-spec commented Oct 27, 2021

This PR relys on some basic configurations in #1346

@zhouzaida zhouzaida added the MLU label Oct 27, 2021
grimoire and others added 8 commits November 2, 2021 20:50
@grimoire @zhouzaida
[Fix]: Update test data of test_tin_shift (open-mmlab#1426)
426e229 
* Update test data of test_tin_shift

* Delete tmp.engine

* add pytest raises asserterror test

* raise valueerror, update test log

* add more comment

* Apply suggestions from code review

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
@gaotongxiao
fix the wrong function reference bug in BaseTransformerLayer when bat…
c522b47 
…ch_first is True (open-mmlab#1418)
@zhouzaida
[Docs] Add mmcv itself in the docs list (open-mmlab#1441)
fd6eab6 
* Add mmcv itself in the docs list

* modify link of docs
@teamwong111 @ZwwWayne
[Improve] improve checkpoint loading log (open-mmlab#1446)
c9d464c
@zhouzaida @ZwwWayne
Update the table of pre-built packages in readme
d57026b
@zhouzaida @ZwwWayne
change the width from 80 to 100
1ab29de
@zhouzaida @ZwwWayne
add note
e22740b
@russ-spec
Merge branch 'open-mmlab:master' into mlu_nms
f78c753
@russ-spec russ-spec closed this Nov 5, 2021
@russ-spec russ-spec deleted the mlu_nms branch November 5, 2021 13:22
@russ-spec russ-spec mentioned this pull request Nov 8, 2021
Merged
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

MLU

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@russ-spec @zhouzaida @DCNSW @luopeichao @ZwwWayne @grimoire @gaotongxiao @teamwong111