Skip to content

Add palm detector from MediaPipe#51

Merged
fengyuentau merged 1 commit intoopencv:masterfrom
fengyuentau:handpose
May 25, 2022
Merged

Add palm detector from MediaPipe#51
fengyuentau merged 1 commit intoopencv:masterfrom
fengyuentau:handpose

Conversation

@fengyuentau
Copy link
Copy Markdown
Member

@fengyuentau fengyuentau commented Apr 28, 2022 •
edited
Loading

Palm detector:

Hand-skelton finger tracker:

  • put model
  • impl
  • demo
  • put license
  • quantize
  • add to benchmark
  • collect benchmark result

NOTE: OpenCV has some issues forwarding hand-skelton finger tracker model (aka handpose model), so this PR is targeting on palm detector instead.

@fengyuentau fengyuentau added the add model request to add a new model label Apr 28, 2022
@fengyuentau fengyuentau self-assigned this Apr 28, 2022
@fengyuentau fengyuentau linked an issue May 5, 2022 that may be closed by this pull request
Add Handpose from MediaPipe for palm detection and hand-skelton finger tracking #50
Closed
@fengyuentau fengyuentau changed the title Add hand pose detector Add MediaPipe Handpose for palm detection and hand-skelton finger tracking May 5, 2022
@fengyuentau
Copy link
Copy Markdown
Member Author

fengyuentau commented May 9, 2022 •
edited
Loading

Two issues are found:

OpenCV does not support asymmetric padding for QLinearConv for now

This feature will be evaluated this week (May 9 to May 13). I will make it support asymmetric padding if not too much effort is needed.

This feature is added in opencv/opencv#21991, which was merged several days ago.

Forwarding handpose model but failed

OpenCV(4.5.5-dev) Error: Assertion failed (start <= (int)shape.size() && end <= (int)shape.size() && start <= end) in total, file /home/fyt//opencv/modules/dnn/include/opencv2/dnn/shape_utils.hpp, line 171
Traceback (most recent call last):
  File "test.py", line 10, in <module>
    net.forward()
cv2.error: OpenCV(4.5.5-dev) /home/fyt/opencv/modules/dnn/include/opencv2/dnn/shape_utils.hpp:171: error: (-215:Assertion failed) start <= (int)shape.size() && end <= (int)shape.size() && start <= end in function 'total'

This issue is being investigating, so this PR is targeting on palm detector instead.

We need broadcast support for element-wise layers to solve this problem, which I am working on.

@fengyuentau fengyuentau changed the title Add MediaPipe Handpose for palm detection and hand-skelton finger tracking Add palm detector from MediaPipe May 9, 2022
@fengyuentau fengyuentau requested review from kaingwade and zihaomu May 9, 2022 09:06
@fengyuentau fengyuentau mentioned this pull request May 16, 2022
Merged
6 tasks
@fengyuentau
Copy link
Copy Markdown
Member Author

fengyuentau commented May 24, 2022

@kaingwade @zihaomu Please take a look.

@kaingwade
Copy link
Copy Markdown
Collaborator

kaingwade commented May 25, 2022

lgtm.

zihaomu
zihaomu approved these changes May 25, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@zihaomu zihaomu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 LGTM.

@fengyuentau fengyuentau merged commit 3d30298 into opencv:master May 25, 2022
@fengyuentau fengyuentau deleted the handpose branch May 25, 2022 12:05
@fengyuentau
Copy link
Copy Markdown
Member Author

fengyuentau commented Jul 19, 2022

opencv/opencv#21865 adds broadcast support for element-wise layers and is merged. We can be back on working on the handpose trakcer.

@WanliZhong WanliZhong mentioned this pull request Aug 16, 2022
Merged
8 tasks
fengyuentau added a commit that referenced this pull request Jun 8, 2023
@fengyuentau
add palm detector from mediapipe (#51)
1105111
@fengyuentau fengyuentau added this to the 4.9.0 (first release) milestone Dec 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zihaomu zihaomu zihaomu approved these changes

@kaingwade kaingwade kaingwade approved these changes

Assignees

@fengyuentau fengyuentau

Labels

add model request to add a new model

Projects

None yet

Milestone

4.9.0 (first release)

Development

Successfully merging this pull request may close these issues.

Add Handpose from MediaPipe for palm detection and hand-skelton finger tracking

3 participants

@fengyuentau @kaingwade @zihaomu