Also, if this PR is merged, I'm really thinking it should be gated behind a feature flag. This is a fundamental change to how terminals work in NVDA.

As a note for reviewers, I've written a few thoughts on doing this at #13781 (comment).

While a fundamental change, it shouldn't have that much user visible differences on the front side. I'm pretty reluctant to introduce a feature flag for every single fundamental change. Feature flags tend to stay for ages without bringing the feature any further.
I'd rather see the approach thoroughly tested before it lands in a release.

Also CCing @carlos-zamora.

• PASS: Translation comments check.
• PASS: Unit tests.
• FAIL: Lint check. See test results for more information.
• FAIL: System tests. See test results for more information.
• Build (for testing PR): https://ci.appveyor.com/api/buildjobs/krojigoeb2bcfqv2/artifacts/output/nvda_snapshot_pr14047-26266,3ed04af5.exe

See test results for failed build of commit 3ed04af591

I think that some check should be done on the windows terminal version as not all support UIA notifications

I think that some check should be done on the windows terminal version as not all support UIA notifications

No, because:

• With "undocking", the version number isn't always reliable.
• The versions that don't support notifications are no longer supported, and per @DHowett people really shouldn't run them.

I've noticed that when it announces new text, everything is spoken as one line, so that this code

for i in range(10):
    print(i)

It would read it as 012345... Instead of each one on one line like it should

I've noticed that when it announces new text, everything is spoken as one line

@LeonarddeR I've provided a patched notification handler that should work around this (you'll need to import api, then import config at the top of the file, please place above import ctypes). CC @Mazen428

I tested this briefly.

1. When typing certain punctuation with speak typed characters off (e.g. @#$%), I hear the character. Others don't do it (e.g. ., ,, /). This might be a bug in the terminal.
2. Toggling report dynamic content changes off doesn't stop the notifications from reading. This is sometimes useful.
3. The reading experience isn't that good when reading long text, partially because Windows Terminal has a limit of 1000 characters per notification.
   It's also not guaranteed that you'll get an entire line of text; connection delays might give you half a line no matter how large the notifications are.
   To get around this, notifications can be batched for a short time before speaking them. This won't be perfect, but it should be better than it is now.

2. Toggling report dynamic content changes off doesn't stop the notifications from reading. This is sometimes useful.

Fixed in the suggested change to the UIA notification handler above.

As for the others, I think these are probably terminal issues. I'll ping @carlos-zamora.

1. When typing certain punctuation with speak typed characters off (e.g. @#$%), I hear the character. Others don't do it (e.g. ., ,, /). This might be a bug in the terminal.

Strange! We shouldn't be doing anything differently, but I'll take a closer look at this later this week.

3. The reading experience isn't that good when reading long text, partially because Windows Terminal has a limit of 1000 characters per notification.
   It's also not guaranteed that you'll get an entire line of text; connection delays might give you half a line no matter how large the notifications are.
   To get around this, notifications can be batched for a short time before speaking them. This won't be perfect, but it should be better than it is now.

I could change this on the Terminal side and make it do less than the limit to fit an entire word/sentence in. But I'm concerned we would see some weird behavior when the newly output text is overwriting existing text or something like that. I think batching the notifications is the right approach here.

Fun fact: @lhecker and I were just discussing batching notifications and the buffer this past Monday haha

Hi,

Note that the base implementation of UIA notification event handler will ignore background notifications. If it works differently in Windows Terminal object, then this might be something to think about. Also, keep in mind that the nature of UIA notification event is not really that of a dynamic content announcer - it is there to announce accessible event notifications. This is why I intentionally did not tie this setting to dynamic content changes when I first introduced notification event support in 2018, and that is also embedded in Windows App Essentials - to turn off UIA notification via that add-on, one must turn off "report notifications" from NVDA's object presentation settings panel. It is also another reason why I did work on a dedicated setting for UIA notification announcement.

Thanks.

Wouldn't making this based on EnhancedTermTypedCharSupport, excluding liveText, fix the punctuation issues along with clearing typed characters buffer when pressing tab or enter?

• EnhancedTermTypedCharSupport subclasses LiveText.
• EnhancedTermTypedCharSupport depends on event_textChange for password suppression (in addition to LiveText using it to diff). Registering for this event in terminals introduces serious performance problems (see UIA freezes after a large number of events is received #11002).

How would we implement batching? Does Terminal know that it is sending a batch? If yes, I think it would help if Terminal communicates in the notification that this is a part of a batch that's not yet finished so NVDA can queue. If that isn't possible, I'm afraid we're getting into business with waiting and timeouts all over again, and I really want to avoid that if possible.

@LeonarddeR Can we find a way not to register for or receive textChange events in wt now that we're not using them? This should seriously improve performance.

Do you have any proof of that statement (i.e. that it improves performance)? In what circumstances?

Anyhow, you're raising an interesting question. I'm afraid the short answer is no, since we register to text changed notifications in the UIA handler, either globally or locally depending on selective event registration. However, as soon as we register, every text changed notification enters NVDA's python code and therefore can slow things down. I guess we can come up with a clone of AppModule.shouldProcessUIAPropertyChangedEvent, but that's again and again putting the real problem off before us.

Do you have any proof of that statement (i.e. that it improves performance)? In what circumstances?

Yes. In particular, when we're flooded with new text:

1. Run the wtNotifications branch of NVDA.
2. Open a wt instance. Start a Python interpreter, and run the following code: for i in range(10000): print("test")
   • Observe that NVDA reads "test" for a few seconds, then UIA dies.
3. Comment out the mapping UIA.UIA_Text_TextChangedEventId: "textChange" (around line 209 of UIAHandler/__init__.py) and restart NVDA.
4. Repeat step 3.
   • Observe that NVDA remains functional.

That's pretty disconcerting.
Would you be able to prototype something like AppModule.shouldProcessUIAEvent to see whether that fixes or at least improves it?

Does Terminal know that it is sending a batch? If yes, I think it would help if Terminal communicates in the notification that this is a part of a batch that's not yet finished so NVDA can queue. If that isn't possible, I'm afraid we're getting into business with waiting and timeouts all over again, and I really want to avoid that if possible.

Terminal won't be able to batch notifications because we get all the text from the shell we're connected to. So we have no knowledge if more text is on the way or not. You could loosely assume that if you received a large amount of text, a bunch more is on the way since you're presumably reading out the content of a large text file. But other than that (and that's a very loose heuristic), we can't really do much.

@lhecker and I chatted a bit more about this earlier today. I've written down some notes in the Terminal repo here: microsoft/terminal#10822 (comment)

Is there a way to retain backwards compatibility by keeping NVDAObjects.UIA.winConsoleUIA.WinTerminalUIA?

Is there a way to retain backwards compatibility by keeping NVDAObjects.UIA.winConsoleUIA.WinTerminalUIA?

I could make it an alias for the currently default implementation (diff-based for now, then notifications based later), but I don't think this would add very much value. This is an internal class that I highly doubt plugins would use.

This is an internal class that I highly doubt plugins would use.

If the new classes are also intended to be internal classes, please prefix them with underscores.
This was marked as part of the public API. According to current policy we aim to keep aliases where there is low ongoing maintenance cost.

FYI, I don't know if this is off topic for this request, but it looks like there is a spelling error in the change log: responsivity maybe should be responsiveness?