Touch interaction: allow users to disable touch support completely by josephsl · Pull Request #10557 · nvaccess/nvda

josephsl · 2019-11-28T15:42:28Z

Link to issue number:

Summary of the issue:

Allow users to turn off touch support completely.

Description of how this pull request fixes the issue:

Added a checkbox (and a corresponding flag in NVDA configuration database) that disables touch support completely. If disabled, touchscreen will behave as though NVDA is not running. Also, added support for touch being disabled on per-profile basis i.e. touch can be disabled if an app comes with its own touch interaction model.

Testing performed:

Tested via try builds on a laptop with touch capabilities.

Known issues with pull request:

None so far.

Change log entry:

Changes (although this can go as a new feature):

You can now enable or disable NVDA's touchscreen support. This is done by checking a new checkbox in Touch Interaction panel of NVDA settings. (#9682)

Thanks.

josephsl · 2019-11-28T15:44:45Z

Hi,

This will be considered "draft" while NVDA 2019.3 is under development; once stable version is released, draft status will be removed. IN the meantime, comments (including design comments) are appreciated.

Thanks.

LeonarddeR

Could you please change the touch panel to guiHelper while at it?

LeonarddeR · 2019-11-28T15:46:00Z


-def touchSupported():
+
+def touchSupported(debugLog=False):


Suggested change

def touchSupported(debugLog=False):

def touchSupported(debugLog: bool = False):

LeonarddeR · 2019-11-28T15:48:28Z


-def initialize():
+
+def setTouchSupport(enable):


Suggested change

def setTouchSupport(enable):

def setTouchSupport(enable: bool):

josephsl · 2019-11-28T16:18:48Z

Hi, I don’t think we execute profile switch handlers when settings are saved – I borrowed this logic from Speech category/audio ducking. Thanks.

LeonarddeR · 2019-11-29T06:09:03Z

+==== Enable touch interaction support ====[TouchSupportEnable]
+This checkbox enables NVDA's touch interaction support.
+If enabled, you can use your fingers to navigate and interact with items on screen using a touchscreen device.
+If disabled, touchscreen support from NVDA will be disabled.


What does this mean?

Suggested change

If disabled, touchscreen support from NVDA will be disabled.

If disabled, touchscreen support from NVDA will be disabled, making the touch screen behave as normal.

josephsl · 2019-11-29T11:02:54Z

Hi, I had difficulty when writing that sentence, and the suggestion above confirms my thought that we need to clarify a bit. Thanks for pointing this out.

josephsl · 2020-01-21T04:28:20Z

Hi,

Now that 2019.3 is under translatable string freeze, and with 2020.1 changelog showing, I'd like to get this PR reviewed again.

Thanks.

Foundation: allow users to disable touch handler overlay by adding 'touch enabled' key to NVDA settings.

…rt should be used. Re nvaccess#9682

…. re nvaccess#9682. Enable touch support if configured to do so via settings. Also, change log text on touchscreen detection to just announce how many touch points are available before proceeding to enable/disable touch support.

Allow users to enable/disable touch support via Touch Interaction settings panel.

…be seen unless requested. Re nvaccess#9682.

…touch mode setter function. Re nvaccess#9682. Similar to audio ducking support: use a dedicated 'set touch support' function to enable/disable touch handler thread based on whether touch support should be anbeld or not. This function in turn will be called when configuration profile changes to support scenarios such as manually turning off touch support or using profiles for apps where NVDA's own touch support should be turned off.

… switching support. Re nvaccess#9682. Touch handler's initialize function will set appropriate touch support mode based on current config value. Also, both initialize and termiante methods support profile switching by calling register/unregister function, respectively.

…nction to toggle touch handler thread. Re nvaccess#9682.

…gs. Re nvaccess#9682

…9682. Reviewed by Leonard de Ruijter: add type annotations, along with docstring for touch supported function.

Comment from Leonard de Ruijter: convert touch interaction panel to use GUI helper code just like other panels.

…led. Re nvaccess#9682

josephsl · 2020-04-14T19:29:30Z

Hi,

Suspended until further notice in light of #11006.

Thanks.

feerrenrut · 2020-04-15T10:52:00Z

Suspended until further notice in light of #11006.

Please continue to work on this PR. #11006 requests no new PR's. This PR already exists and is thus in the backlog of PR's that we want to address before accepting new ones. We are trying to stop PR's from going stale, and then becoming hard to merge.

Forgive me for replying to you multiple times, I want to avoid the community being mislead.

josephsl · 2020-04-15T10:54:26Z

I’ll take a look at this PR as early as Thursday.

JulienCochuyt · 2020-05-02T12:11:51Z

@josephsl, thanks for taking care of this. The lack of this feature has been annoying me quite a lot lately.

My few grains of salt on this topic...

Would you please consider allowing to bind a gesture on toggling touch screen support on and off?
Shouldn't we consider to temporarily disable touch screen support when pressing NVDA+F2? If not, any suitable mean of letting a touch gesture through would be really handy.
Does UIA or IA2 expose the fact that a given control should have touch gestures not intercepted by AT? (such as found on touch musical instruments in IOS) If so, we should most likely plan on honoring it.
Should we consider shipping a default profile for Windows Touch Magnifier window to disable touch support when interacting with it?

josephsl · 2020-05-02T17:52:41Z

Hi,

Suspending touch support for passthrough command: not easy. In order to do so, NVDA must be able to detect that touch interaction has occurred after it is suspended. This isn't quite possible because once an assistive technology registers for touch interaction, it won't let go unless it voluntarily gives up, and manual work is involved to bring touch support back online. Also, if multiple assistive technologies wants to work with touch interaction, only one will get a chance to do it (when it starts, Narrator presents a dialog about this when NVDA is running on a touchscreen computer).

Command to toggle touch support: although it makes sense, it isn't consistent. One can assign a keyboard or a braille command to toggle touch interaction. One can turn off touch interaction using touch gestures, but you can't bring it back with touch gestures as NVDA doesn't know if a gesture has occurred (same problem as above).

UIA/IA2 and touch support: probably not unless I read the documentation wrong.

As for Windows Magnifier window: one way to enhance this in the future is letting app modules or overlay classes define if touch interaction should be turned off. This is already possible with this PR through app-specific profiles, which users must specify manually.

Thanks.

JulienCochuyt · 2020-05-02T19:05:07Z

@josephsl wrote:

Suspending touch support for passthrough command: not easy. In order to do so, NVDA must be able to detect that touch interaction has occurred after it is suspended. This isn't quite possible because once an assistive technology registers for touch interaction, it won't let go unless it voluntarily gives up, and manual work is involved to bring touch support back online.

Letting other ATs aside, do you mean there is no such mechanism as sending a touch gesture we received back to the control that would have received it if NVDA did not register?
If so, I think my behavior of choice would be to (at least optionally) suspend touch handling until the next keystroke, to allow for keyboard+touch usages.

Command to toggle touch support: although it makes sense, it isn't consistent. One can assign a keyboard or a braille command to toggle touch interaction. One can turn off touch interaction using touch gestures, but you can't bring it back with touch gestures as NVDA doesn't know if a gesture has occurred (same problem as above).

Idem, I feel that inconsistency here is better than inability, especially if we cannot achieve a "next touch gesture pass through" command.

As for Windows Magnifier window: one way to enhance this in the future is letting app modules or overlay classes define if touch interaction should be turned off. This is already possible with this PR through app-specific profiles, which users must specify manually.

Glad to read this. We for sure can take the time once this PR is merged to experiment with such a profile before deciding if it's worth being promoted as a default.

josephsl · 2020-05-02T19:11:30Z

Hi,

Once an AT unregisters itself from getting touch gestures, it isn't possible to listen to touch gestures. This is because an invisible iwndow is used by the AT to listen to touch interaction signals, and suspending touch means destroying that window.

As for toggle command: I imagine a scenario where people would prefer to use touchscreen alone rather than using a keyboard along with the device (Surface Pro without type cover, for example).

Thanks.

JulienCochuyt · 2020-05-02T19:16:02Z

Once an AT unregisters itself from getting touch gestures, it isn't possible to listen to touch gestures. This is because an invisible iwndow is used by the AT to listen to touch interaction signals, and suspending touch means destroying that window.

Sure, but again, do you mean we cannot send back what we don't want to keep (as keyboard handler does)?

As for toggle command: I imagine a scenario where people would prefer to use touchscreen alone rather than using a keyboard along with the device (Surface Pro without type cover, for example).

While this scenario makes sense, it's no reason IMHO to disallow altogether binding a keyboard gesture to do the toggling.

josephsl · 2020-05-02T19:27:39Z

Hi, no, we can’t do that with touchscreen gestures at this time, as the window itself isn’t going to be used. In short, Windows will forget that an AT is active. As for toggle command bindings, it might be possible to do that at least for keyboards – it is something we should think about for a future iteration of this PR. Thanks.

…onfigurableTouchOverlay

michaelDCurran · 2020-06-23T01:13:52Z

@JulienCochuyt You bring up all good points, but these should be all handled in one or more separate PRs. Not only are they a little complex and will be somewhat technically unrelated to how this is currently implemented. By itself this current PR still has a clear use case. There are certainly low vision users who use NVDA to read text along with a magnifier, but have enough vision to interact with their touch screen independently and do not want NVDA ever intercepting touch. So @josephsl assuming this passes review, I want to take this as soon as we can.

michaelDCurran · 2020-06-23T01:24:15Z

@josephsl just confirming that you have tested config profile switches where one profile has touch enabled and one does not? I.e. touch successfully gets re-enabled after being disabled by a previous profile?

josephsl · 2020-06-23T01:26:19Z

Hi, glad to contribute another feature from Enhanced Touch Gestures to NVDA Core – thanks.

josephsl · 2020-06-23T01:31:42Z

Hi, yes, tested via Enhanced Touch Gestures add-on, which has identical construction. Thanks.

josephsl · 2020-06-23T01:32:02Z

Hi, forgot to say that profile switching does work. Thanks.

michaelDCurran · 2020-06-23T01:44:31Z

@LeonarddeR Are there any pending changes you are waiting for on this pr?

LeonarddeR

If a sighted person can confirm that the gui looks good (though I don't have reason to think it wouldn't), I'm ok with this. Just one small thing.

LeonarddeR · 2020-06-23T19:01:31Z

 		return False
-	if (winVersion.winVersion.major*10+winVersion.winVersion.minor)<62:
-		log.debugWarning("Touch only supported on Windows 8 and higher")
+	if winVersion.winVersion.platform_version < (6, 2, 9200):


Is it really necessary to take the platform into account here?

Suggested change

if winVersion.winVersion.platform_version < (6, 2, 9200):

if (winVersion.winVersion.major, winVersion.winVersion.minor) < (6, 2):

josephsl · 2020-06-23T19:04:15Z

Hi, the platform attribute gives us all the info we need in one function call, and it is also used for app module version lookup. Thanks.

nvaccess#10557)

Follow-up of #10557 Default binding: NVDA+control+alt+t

josephsl requested a review from michaelDCurran November 28, 2019 15:42

josephsl self-assigned this Nov 28, 2019

LeonarddeR suggested changes Nov 28, 2019

View reviewed changes

LeonarddeR reviewed Nov 29, 2019

View reviewed changes

josephsl marked this pull request as ready for review January 21, 2020 04:30

josephsl added 15 commits February 12, 2020 22:18

ConfSpec: add touch enabled key. Re nvaccess#9682.

458a7f1

Foundation: allow users to disable touch handler overlay by adding 'touch enabled' key to NVDA settings.

Touch handler: copyright header

dfb0901

Touch handler: use platform version tuple to determine if touch suppo…

4892a39

…rt should be used. Re nvaccess#9682

Settings/touch interaction: add touch support toggle. Re nvaccess#9682.

eedf3a0

Allow users to enable/disable touch support via Touch Interaction settings panel.

Touch supporte3d: add debugLog flag so excessive log output will not …

79798b7

…be seen unless requested. Re nvaccess#9682.

Touch interaction settings panel: use touchHandler.setTouchSupport fu…

3ec6a28

…nction to toggle touch handler thread. Re nvaccess#9682.

Touch interaction panel: add accelerator.

b878e3c

User guide: add notes on touch interaction support checkbox in settin…

5ce560e

…gs. Re nvaccess#9682

Touch handler: review comments (annotations, docstring). Re nvaccess#…

d4bed76

…9682. Reviewed by Leonard de Ruijter: add type annotations, along with docstring for touch supported function.

Touch interaction panel: convert to use GUI helper. re nvaccess#9682.

2c54386

Comment from Leonard de Ruijter: convert touch interaction panel to use GUI helper code just like other panels.

User guide: clarify the behavior of touch support when touch is disab…

475ee44

…led. Re nvaccess#9682

Touch handler: update copyright years

ec3061b

josephsl force-pushed the i9682configurableTouchOverlay branch from 4df35f1 to ec3061b Compare February 13, 2020 06:19

feerrenrut added the feature label Apr 8, 2020

This comment has been minimized.

Sign in to view

Merge branch 'master' of https://github.com/nvaccess/nvda into i9682c…

b1376a2

…onfigurableTouchOverlay

michaelDCurran approved these changes Jun 23, 2020

View reviewed changes

LeonarddeR approved these changes Jun 23, 2020

View reviewed changes

michaelDCurran merged commit 38743ad into nvaccess:master Jun 23, 2020

nvaccessAuto added this to the 2020.2 milestone Jun 23, 2020

michaelDCurran added a commit that referenced this pull request Jun 23, 2020

Update what's new for pr #10557

b37ae7f

michaelDCurran modified the milestones: 2020.2, 2020.3 Jun 23, 2020

JulienCochuyt added a commit to accessolutions/nvda that referenced this pull request Jun 24, 2020

Touch interaction: Add a global command to toggle support (Follow-up of

d99c3ee

nvaccess#10557)

JulienCochuyt mentioned this pull request Jun 24, 2020

Touch interaction: Add a global command to toggle support (Follow-up of #10557) #11297

Merged

josephsl deleted the i9682configurableTouchOverlay branch June 24, 2020 06:39

feerrenrut pushed a commit that referenced this pull request Jun 30, 2020

Touch interaction: Add a global command to toggle support (PR #11297)

c7367c9

Follow-up of #10557 Default binding: NVDA+control+alt+t

	def touchSupported(debugLog=False):
	def touchSupported(debugLog: bool = False):

	def setTouchSupport(enable):
	def setTouchSupport(enable: bool):

	If disabled, touchscreen support from NVDA will be disabled.
	If disabled, touchscreen support from NVDA will be disabled, making the touch screen behave as normal.

	if winVersion.winVersion.platform_version < (6, 2, 9200):
	if (winVersion.winVersion.major, winVersion.winVersion.minor) < (6, 2):

Uh oh!

Conversation

josephsl commented Nov 28, 2019

Link to issue number:

Summary of the issue:

Description of how this pull request fixes the issue:

Testing performed:

Known issues with pull request:

Change log entry:

Uh oh!

josephsl commented Nov 28, 2019

Uh oh!

LeonarddeR left a comment

Choose a reason for hiding this comment

Uh oh!

LeonarddeR Nov 28, 2019

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

LeonarddeR Nov 28, 2019

Choose a reason for hiding this comment

Uh oh!

josephsl commented Nov 28, 2019 via email • edited by feerrenrut Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LeonarddeR Nov 29, 2019

Choose a reason for hiding this comment

Uh oh!

josephsl commented Nov 29, 2019 via email • edited by feerrenrut Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

josephsl commented Jan 21, 2020

Uh oh!

josephsl commented Apr 14, 2020

Uh oh!

feerrenrut commented Apr 15, 2020

Uh oh!

josephsl commented Apr 15, 2020 via email • edited by feerrenrut Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

JulienCochuyt commented May 2, 2020

Uh oh!

josephsl commented May 2, 2020

Uh oh!

JulienCochuyt commented May 2, 2020

Uh oh!

josephsl commented May 2, 2020

Uh oh!

JulienCochuyt commented May 2, 2020

Uh oh!

josephsl commented May 2, 2020 via email • edited by feerrenrut Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

michaelDCurran commented Jun 23, 2020

Uh oh!

michaelDCurran commented Jun 23, 2020

Uh oh!

josephsl commented Jun 23, 2020 via email • edited by feerrenrut Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

josephsl commented Jun 23, 2020 via email • edited by feerrenrut Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

josephsl commented Jun 23, 2020 via email • edited by feerrenrut Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

michaelDCurran commented Jun 23, 2020

Uh oh!

LeonarddeR left a comment

Choose a reason for hiding this comment

Uh oh!

LeonarddeR Jun 23, 2020

Choose a reason for hiding this comment

Uh oh!

josephsl commented Jun 23, 2020 via email • edited by feerrenrut Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

josephsl commented Nov 28, 2019 via email •

edited by feerrenrut

Loading

josephsl commented Nov 29, 2019 via email •

edited by feerrenrut

Loading

josephsl commented Apr 15, 2020 via email •

edited by feerrenrut

Loading

josephsl commented May 2, 2020 via email •

edited by feerrenrut

Loading

josephsl commented Jun 23, 2020 via email •

edited by feerrenrut

Loading

josephsl commented Jun 23, 2020 via email •

edited by feerrenrut

Loading

josephsl commented Jun 23, 2020 via email •

edited by feerrenrut

Loading

josephsl commented Jun 23, 2020 via email •

edited by feerrenrut

Loading