[ios][platform_view]force reset forwarding recognizer state when its stuck #55958

hellohuanlin · 2024-10-18T18:58:14Z

This PR force reset forwarding recognizer state when it's stuck at failed state, by recreating the recognizer (we are not able to reset the state back to possible - it has to be reset by UIKit).

This is a tricky one since pencil and finger triggers exactly the same callbacks. It turns out that when pencil is involved after finger interaction, the platform view's "forwarding" gesture recognizer is stuck at failed state. This seems to be an iOS bug, because according to the API doc, it should be reset back to "possible" state:

No action message is sent and the gesture recognizer is reset to UIGestureRecognizerStatePossible.

However, when iPad pencil is involved, the state is not reset. I tried to KVO the state property, and wasn't able to capture the change. This means the state change very likely happened internally within the recognizer via the backing ivar of the state property.

Fixes flutter/flutter#136244

If you had to change anything in the flutter/tests repo, include a link to the migration guide as per the breaking change policy.

Pre-launch Checklist

I read the Contributor Guide and followed the process outlined there for submitting PRs.
I read the Tree Hygiene wiki page, which explains my responsibilities.
I read and followed the Flutter Style Guide and the C++, Objective-C, Java style guides.
I listed at least one issue that this PR fixes in the description above.
I added new tests to check the change I am making or feature I am adding, or the PR is test-exempt. See testing the engine for instructions on writing and running engine tests.
I updated/added relevant documentation (doc comments with ///).
I signed the CLA.
All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

flutter-dashboard · 2024-10-18T18:58:17Z

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption, contact "@test-exemption-reviewer" in the #hackers channel in Discord (don't just cc them here, they won't see it!).

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. The test exemption team is a small volunteer group, so all reviewers should feel empowered to ask for tests, without delegating that responsibility entirely to the test exemption group.

hellohuanlin · 2024-10-18T18:59:09Z

Tested the demo app from github issue and confirmed it's working. I will do more testing before landing it.

…stuck

jmagman

I don't know if change would impact it, but can you confirm the totally-not-running-anywhere iPad gesture tests still pass?
https://github.com/flutter/engine/blob/4e6405e2a89fa8fda2c0e7017ddcd8b39d7b3052/testing/scenario_app/ios/Scenarios/ScenariosUITests/iPadGestureTests.m

jmagman · 2024-10-21T20:48:03Z

shell/platform/darwin/ios/framework/Source/FlutterPlatformViews_Internal.mm

  return self;
 }

+- (ForwardingGestureRecognizer*)recreateRecognizerWithTarget:(id)target {


Would it be possible for ForwardingGestureRecognizer to instead implement NSCopying?

Talked to @hellohuanlin, this isn't feasible because of the move which makes the old recognizer unusable.

jmagman · 2024-10-21T20:49:08Z

shell/platform/darwin/ios/framework/Source/FlutterPlatformViews_Internal.mm


+- (void)forceResetForwardingGestureRecognizerState {
+  // When iPad pencil is involved in a finger touch gesture, the gesture is not reset to "possible"
+  // state, which causes subsequent touches to be blocked. As a workaround, we force reset the state


and is stuck on "failed"

hellohuanlin · 2024-10-22T03:25:04Z

Verified both google map and web view plugin example apps still work as expected.

polina-c · 2024-10-22T14:47:48Z

@tdevaux confirmed this fixes the issue

hellohuanlin · 2024-10-22T21:24:24Z

I don't know if change would impact it, but can you confirm the totally-not-running-anywhere iPad gesture tests still pass?
https://github.com/flutter/engine/blob/4e6405e2a89fa8fda2c0e7017ddcd8b39d7b3052/testing/scenario_app/ios/Scenarios/ScenariosUITests/iPadGestureTests.m

I got all of them failed with main branch on simulator. Let me try physical devices.

jmagman · 2024-10-23T03:15:33Z

I got all of them failed with main branch on simulator. Let me try physical devices.

Don't worry about it, we can investigate later and get them passing and running somewhere.
Let's merge this! 🙂

hellohuanlin · 2024-10-23T17:54:35Z

I was able to run on device and got the same failure as simulator. Filed flutter/flutter#157454

auto-submit · 2024-10-23T18:27:59Z

auto label is removed for flutter/engine/55958, due to - The status or check suite Mac mac_unopt has failed. Please fix the issues identified (or deflake) before re-applying this label.

…hen its stuck (flutter/engine#55958)

flutter/engine@73c54fa...5d7caf7 2024-10-23 34871572+gmackall@users.noreply.github.com Delete v1 android engine embedding (flutter/engine#52022) 2024-10-23 41930132+hellohuanlin@users.noreply.github.com [ios][platform_view]force reset forwarding recognizer state when its stuck (flutter/engine#55958) 2024-10-23 jason-simmons@users.noreply.github.com Allow running the YAPF formatter using Python version 3.11 (flutter/engine#56062) 2024-10-23 jonahwilliams@google.com [Impeller] optimize clip rects by checking if integral coverage is sufficient. (flutter/engine#56041) 2024-10-23 a-siva@users.noreply.github.com Switch the web compilation step to use an AOT snapshot (flutter/engine#56040) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll Please CC codefu@google.com,zra@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

…7478) flutter/engine@73c54fa...5d7caf7 2024-10-23 34871572+gmackall@users.noreply.github.com Delete v1 android engine embedding (flutter/engine#52022) 2024-10-23 41930132+hellohuanlin@users.noreply.github.com [ios][platform_view]force reset forwarding recognizer state when its stuck (flutter/engine#55958) 2024-10-23 jason-simmons@users.noreply.github.com Allow running the YAPF formatter using Python version 3.11 (flutter/engine#56062) 2024-10-23 jonahwilliams@google.com [Impeller] optimize clip rects by checking if integral coverage is sufficient. (flutter/engine#56041) 2024-10-23 a-siva@users.noreply.github.com Switch the web compilation step to use an AOT snapshot (flutter/engine#56040) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll Please CC codefu@google.com,zra@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

…stuck (flutter/engine#55958) This PR force reset forwarding recognizer state when it's stuck at failed state, by recreating the recognizer (we are not able to reset the state back to possible - it has to be reset by UIKit). This is a tricky one since pencil and finger triggers exactly the same callbacks. It turns out that when pencil is involved after finger interaction, the platform view's "forwarding" gesture recognizer is stuck at failed state. This seems to be an iOS bug, because according to [the API doc](https://developer.apple.com/documentation/uikit/uigesturerecognizerstate/uigesturerecognizerstatefailed?language=objc), it should be reset back to "possible" state: > No action message is sent and the gesture recognizer is reset to [UIGestureRecognizerStatePossible](https://developer.apple.com/documentation/uikit/uigesturerecognizerstate/uigesturerecognizerstatepossible?language=objc). However, when iPad pencil is involved, the state is not reset. I tried to KVO the state property, and wasn't able to capture the change. This means the state change very likely happened internally within the recognizer via the backing ivar of the state property. Fixes flutter#136244 *If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].* [C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style

github-actions bot added the platform-ios label Oct 18, 2024

hellohuanlin force-pushed the pv_ipad_pencil_force_reset_recognizer branch from 5076c8e to 97c628b Compare October 18, 2024 19:26

[ios][platform_view]force reset forwarding recognizer state when its …

40e6eaa

…stuck

hellohuanlin force-pushed the pv_ipad_pencil_force_reset_recognizer branch from 97c628b to 40e6eaa Compare October 18, 2024 19:33

hellohuanlin requested a review from jmagman October 18, 2024 19:34

jmagman reviewed Oct 21, 2024

View reviewed changes

jmagman approved these changes Oct 22, 2024

View reviewed changes

nit

de91efa

hellohuanlin added the autosubmit Merge PR when tree becomes green via auto submit App label Oct 23, 2024

auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Oct 23, 2024

jmagman added the autosubmit Merge PR when tree becomes green via auto submit App label Oct 23, 2024

auto-submit bot merged commit 489aac4 into flutter:main Oct 23, 2024

engine-flutter-autoroll mentioned this pull request Oct 23, 2024

Roll Flutter Engine from 73c54fadf5ee to 5d7caf771022 (5 revisions) flutter/flutter#157478

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Oct 23, 2024

489aac4 [ios][platform_view]force reset forwarding recognizer state w…

2c3ab71

…hen its stuck (flutter/engine#55958)

hellohuanlin mentioned this pull request Dec 15, 2025

[ios] unit test fail on iOS 26: testFlutterPlatformViewTouchesEndedOrTouchesCancelledEventDoesNotFailTheGestureRecognizer flutter/flutter#179907

Open

[ios][platform_view]force reset forwarding recognizer state when its stuck #55958

[ios][platform_view]force reset forwarding recognizer state when its stuck #55958

Uh oh!

Conversation

hellohuanlin commented Oct 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pre-launch Checklist

Uh oh!

flutter-dashboard bot commented Oct 18, 2024

Uh oh!

hellohuanlin commented Oct 18, 2024

Uh oh!

jmagman left a comment

Choose a reason for hiding this comment

Uh oh!

jmagman Oct 21, 2024

Choose a reason for hiding this comment

Uh oh!

jmagman Oct 22, 2024

Choose a reason for hiding this comment

Uh oh!

jmagman Oct 21, 2024

Choose a reason for hiding this comment

Uh oh!

hellohuanlin commented Oct 22, 2024

Uh oh!

polina-c commented Oct 22, 2024

Uh oh!

hellohuanlin commented Oct 22, 2024

Uh oh!

jmagman commented Oct 23, 2024

Uh oh!

hellohuanlin commented Oct 23, 2024

Uh oh!

auto-submit bot commented Oct 23, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

hellohuanlin commented Oct 18, 2024 •

edited

Loading