MS Word documents: report when headings are collapsed in both speech and braille. by michaelDCurran · Pull Request #17499 · nvaccess/nvda

michaelDCurran · 2024-12-11T01:15:52Z

Link to issue number:

None.

Summary of the issue:

In Microsoft Word, it is possible to collapse a heading, so that all the paragraphs directly following the heading are hidden. As of MS Word 16.0.18226, a new UIA text range custom attribute was added, to convey the expand collapse state of headings. NVDA should report when a heading is collapsed based on this value.

Description of user facing changes

In Microsoft Word, If a heading is collapsed, NVDA will now report this in speech and braille.

Description of development approach

When fetching format information for a text range in MS Word via UIA, if the range supports the expand collapse custom attribute, expose the 'collapsed' key on its format field, setting it to true for collapsed, and false for expanded.
speech.getFormatfieldSpeech: speak "collapsed" when moving by line or paragraph over text with 'collapsed' set to true in its format field, or if it changes to true when moving by word / character.
In Braille, Report a plus (+) sign at the start of text that is collapsed according to its format field.

Testing strategy:

Tested with MS Word 16.0.18331.

Open a new document in Microsoft word.
Type an initial line of text.
Type a second line of text and format it as a heading (press alt+shift+rightArrow)
Type another line of text directly below it formatted as a normal paragraph.
Arrow up to the heading, noting that the heading is spoken / braille dnormally (I.e. heading level 2 test). No expand / collapse info is reported.
Collapse the heading by choosing collapse in the context menu.
Arrow up and back down to read the heading. Note that NvDA now reports 'collapsed' in speech and a plus (+) sign in braille.

Known issues with pull request:

None known.

This implementation only reports when a heading is collapsed, and not explicitly when expanded. However, as expanded headings will far outweigh collapsed, I think it would be way too noisy to report expanded, when that is going to be the norm.

Code Review Checklist:

Documentation:
- Change log entry
- User Documentation
- Developer / Technical Documentation
- Context sensitive help for GUI changes
Testing:
- Unit tests
- System (end to end) tests
- Manual testing
UX of all users considered:
- Speech
- Braille
- Low Vision
- Different web browsers
- Localization in other languages / culture than English
API is compatible with existing add-ons.
Security precautions taken.

@coderabbitai summary

…and braille.

Co-authored-by: Sascha Cowley <16543535+SaschaCowley@users.noreply.github.com>

CyrilleB79 · 2024-12-11T08:22:32Z

For the feature to be complete, the following could be added to this PR:

There are still many people using legacy Word object model instead of UIA. When possible, it would be nice to have features developed for both interfaces at the same time. In the specific case, you can use Paragraph.CollapsedState property.
There are shortcuts to collapse or expand headings. My Word is in French, but according to ChatGPT, on English Word, it is alt+shift+plus or alt+shift+-. Could you implement feedback reporting of these actions? If it is already reported automatically in newer Word versions (I have not tested myself), ignore this request of course.

If you prefer have this development merged as is, at least, document these missing points in "Known issues" sections and open new issues to track these missing points. Thanks.

michaelDCurran · 2024-12-12T00:48:31Z

@CyrilleB79 I have addressed the docElement issue pre Windows 11, and also exposed collapsed for headings in Word with the object model.
Re shortcuts: you are welcome to open issues for these if you like, but it is out of scope for this particular PR.

CyrilleB79 · 2024-12-12T22:13:09Z

I have just tested this PR, launching the installer (nvda_snapshot_pr17499-34747,bb1e29a1.exe) and running the temporary version.

With UIA always, the feature is not working; maybe it is expected on Windows 10?
With Word Object Model (UIA Only when necessary), Word crashes.

I am with:

Microsoft® Word 2016 MSO (Version 2410 Build 16.0.18129.20158) 32 bits
Windows 10 2004 (AMD64) build 19041.388; note that this is not the last version (I cannot test on last Win 10); may it be the cause of the crash?

codeofdusk · 2024-12-15T16:33:23Z

@CyrilleB79 This is expected because Windows 10 doesn't have remote ops support in UIA core.

michaelDCurran · 2024-12-15T23:56:26Z

I can't reproduce the crash. I'll need a crash dump if you can get one. However, if this is hard to get / or we can not trace the reason, then I'll need to remove the object model support for reporting collapsed from this PR, as I cannot hold back the UIA part (which works fine on win11) - it is a request from Microsoft.

CyrilleB79 · 2024-12-16T22:36:54Z

I have not been able to get a crash dump for Word. I have tried both methods described in NVDA wiki:

paragraph Minidumps for Other Applications
paragraph Alternative Way to Get Minidumps for Other Applications, actually following MSDN article

In both cases, I cannot get any dumpfile. I have also tried to force a dumpfile, killing winword process from task manager without success, so I am probably doing something wrong when trying to get the dumpfile.

Word crashes, actually disappears, when arrowing in text heading formatted, or when trying to use expand / collapse commands. When I move the caret in normal (body) text, there is no problem.

It would be nice to be able to fix this. @Adriani90 would you be able to reproduce the crash described here with object model (legacy, i.e. "Only when necessary" in advanced settings)?

But if it is not possible to fix the issue with Word Object model, nor reproducible on your side, I understand that you may want to merge the UIA part, what is already an improvement.

CyrilleB79 · 2024-12-17T21:53:50Z

@michaelDCurran, I have found why it crashes on my end and tested that it does not crash anymore with the fix, running from source.

Strange that you have not experimented this crash; maybe you were not really testing Word object model due to 13704?

michaelDCurran · 2024-12-17T22:29:11Z

I was definitely testing with the object model. Perhaps you are using 32 bit MS Word and I am using 64 bit, or you have tighter execution / memory policies set in Windows? Either way, I'm glad you found the cause.

Co-authored-by: Cyrille Bougot <cyrille.bougot2@laposte.net>

CyrilleB79 · 2024-12-18T07:05:55Z

I was definitely testing with the object model. Perhaps you are using 32 bit MS Word and I am using 64 bit, or you have tighter execution / memory policies set in Windows? Either way, I'm glad you found the cause.

Yes, I am running Office 2016 32 bits.

Co-authored-by: Sascha Cowley <16543535+SaschaCowley@users.noreply.github.com> Co-authored-by: Cyrille Bougot <cyrille.bougot2@laposte.net>

SaschaCowley

Could you please update the copyright headers of the files modified in this PR?

SaschaCowley

Thanks, @michaelDCurran

…is now reported (#17545) Related to #17499, but independent of it. Summary of the issue: In MS Word, control+alt+plus and control+alt+- allow to expand and collapse the content below a heading. NVDA does not report the result of this action. Description of user facing changes: NVDA now reports the result of the Word commands to expand and collapse headings. This command does not exist in Draft mode, so nothing is reported when in this mode. The command also does not report anything in Outline mode (limitation of this PR). Description of development approach: Use Word object model (if available) as done for many Word gestures, no matter if UIA is enabled or not Testing strategy: Tested with and without UIA. Known issues with pull request:: This PR does not cover outline mode. The collapse / extend commands still don't report anything in Outline mode; see related issue #17645.

…is now reported (nvaccess#17545) Related to nvaccess#17499, but independent of it. Summary of the issue: In MS Word, control+alt+plus and control+alt+- allow to expand and collapse the content below a heading. NVDA does not report the result of this action. Description of user facing changes: NVDA now reports the result of the Word commands to expand and collapse headings. This command does not exist in Draft mode, so nothing is reported when in this mode. The command also does not report anything in Outline mode (limitation of this PR). Description of development approach: Use Word object model (if available) as done for many Word gestures, no matter if UIA is enabled or not Testing strategy: Tested with and without UIA. Known issues with pull request:: This PR does not cover outline mode. The collapse / extend commands still don't report anything in Outline mode; see related issue nvaccess#17645.

Adriani90 · 2025-05-11T19:41:04Z

This seems to fail in outline view in MS Word, at least in my case it delivers wrong information.

Colapse some headings while in print layout
Change to outline
Expand headings with alt+shift+plus or colapse them with alt+shift+minus
Navigate with up and down arrow or ctrl+up and down

Actual: Headings are always announced as "colapsed"
Expected: Headings should be announced according to their actual status.

seanbudd · 2025-05-11T23:07:40Z

@Adriani90 - please open a new issue so that this is tracked. comments on closed PRs are easily lost.

… version 16.0.18226 or higher. (#18280) Fixes #18279 Summary of the issue: PR #17499 added support for reporting if a heading is collapsed in MS word. However, this crashes some versions of MS Word before this feature was supported. For example, MS word version16.0.17932.20396 supports custom attribute values, but crashes if given this specific expandCollapseState constant. NVDA should limit fetching this data to when it was officially supported in MS Word. Description of user facing changes: Microsoft Word 2024 will no longer crash when NVDA tries to fetch the collapsed state of headings. Description of developer facing changes: Description of development approach: Limit fetching of expandcollapseState to MS Word 16.0.18226 and higher.

MS Word documents: report when headings are collapsed in both speech …

56495cd

…and braille.

michaelDCurran requested a review from a team as a code owner December 11, 2024 01:15

michaelDCurran requested a review from SaschaCowley December 11, 2024 01:15

Update what's new

df29f40

SaschaCowley requested changes Dec 11, 2024

View reviewed changes

Comment thread user_docs/en/changes.md Outdated

Comment thread source/NVDAObjects/UIA/wordDocument.py

Comment thread source/braille.py Outdated

Comment thread source/speech/speech.py Outdated

Apply suggestions from code review

5c0923c

Co-authored-by: Sascha Cowley <16543535+SaschaCowley@users.noreply.github.com>

michaelDCurran requested a review from SaschaCowley December 11, 2024 06:53

CyrilleB79 reviewed Dec 11, 2024

View reviewed changes

Comment thread source/NVDAObjects/UIA/wordDocument.py Outdated

michaelDCurran and others added 3 commits December 12, 2024 09:38

MS word via object model: report collapsed state on headings.

4a5b8f9

Pre-commit auto-fix

d0a3fe0

MS Word with UIA: avoid error when remote ops not available.

7cd66f1

SaschaCowley requested changes Dec 12, 2024

View reviewed changes

Comment thread nvdaHelper/remote/winword.cpp Outdated

Comment thread source/NVDAObjects/window/winword.py Outdated

Comment thread source/braille.py

Comment thread user_docs/en/changes.md Outdated

CyrilleB79 reviewed Dec 12, 2024

View reviewed changes

Comment thread user_docs/en/changes.md Outdated

SaschaCowley added the conceptApproved Similar 'triaged' for issues, PR accepted in theory, implementation needs review. label Dec 17, 2024

CyrilleB79 reviewed Dec 17, 2024

View reviewed changes

Comment thread nvdaHelper/remote/winword.cpp Outdated

CyrilleB79 mentioned this pull request Dec 17, 2024

MS Word: The result of Word commands to expand or collapse a heading is now reported #17545

Merged

5 tasks

Update nvdaHelper/remote/winword.cpp

eba2249

Co-authored-by: Cyrille Bougot <cyrille.bougot2@laposte.net>

Apply suggestions from code review

9b85d2e

Co-authored-by: Sascha Cowley <16543535+SaschaCowley@users.noreply.github.com> Co-authored-by: Cyrille Bougot <cyrille.bougot2@laposte.net>

michaelDCurran requested a review from SaschaCowley January 1, 2025 22:24

SaschaCowley requested changes Jan 9, 2025

View reviewed changes

Merge branch 'master' into collapsedHeadings

252543f

Update copyright headers

a449f7d

michaelDCurran requested a review from SaschaCowley January 20, 2025 01:23

SaschaCowley approved these changes Jan 20, 2025

View reviewed changes

SaschaCowley merged commit fa619c2 into master Jan 20, 2025

SaschaCowley deleted the collapsedHeadings branch January 20, 2025 04:28

github-actions Bot added this to the 2025.1 milestone Jan 20, 2025

CyrilleB79 mentioned this pull request Jan 23, 2025

MS Word: Collapsed state not reported for headings in outline view #17645

Open

nishimotz mentioned this pull request Apr 14, 2025

2025.1 本家の翻訳 nvdajp/nvdajp#498

Closed

Adriani90 mentioned this pull request May 12, 2025

Headings are reported as colapsed in MS Word although they are not #18089

Open

michaelDCurran mentioned this pull request Jun 18, 2025

MS Word with UIA: only fetch expandCollapseState for headings on Word version 16.0.18226 or higher. #18280

Merged

7 tasks

Uh oh!

Conversation

michaelDCurran commented Dec 11, 2024

Link to issue number:

Summary of the issue:

Description of user facing changes

Description of development approach

Testing strategy:

Known issues with pull request:

Code Review Checklist:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CyrilleB79 commented Dec 11, 2024

Uh oh!

Uh oh!

michaelDCurran commented Dec 12, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CyrilleB79 commented Dec 12, 2024

Uh oh!

codeofdusk commented Dec 15, 2024

Uh oh!

michaelDCurran commented Dec 15, 2024 via email

Uh oh!

CyrilleB79 commented Dec 16, 2024

Uh oh!

Uh oh!

CyrilleB79 commented Dec 17, 2024

Uh oh!

michaelDCurran commented Dec 17, 2024 via email

Uh oh!

CyrilleB79 commented Dec 18, 2024

Uh oh!

SaschaCowley left a comment

Choose a reason for hiding this comment

Uh oh!

SaschaCowley left a comment

Choose a reason for hiding this comment

Uh oh!

Adriani90 commented May 11, 2025

Uh oh!

seanbudd commented May 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants