Address review feedback for Chinese word segmentation#20178
Merged
seanbudd merged 11 commits intoMay 20, 2026
Merged
Conversation
… doesn't do it, causing uploaded symbols to mismatch (nvaccess#20129) Fixes nvaccess#20033 Replaces pr nvaccess#20084 Summary of the issue: The debug symbols archived from builds of NvDA do not match the actual files in the final NVDA launcher, if files are required to be signed. This is due to the fact that the API signing key was never set in the CI environment when building NVDA with scons source, although it was present when running scons launcher. NVDA would be initially built without signed files, and these files would be collected for the symbols, but then as the launcher was being created, the files would be retroactively signed. Also if trying to run CI instructing SCons to use multiple cores to speed up builds, some of the files may have been signed after the launcher was created, thus leaving them as the old unsigned copies. Signing with signpath takes a very long time, and can add up to 10 minutes for a build when using a single core. Thus signing properly in scons source would address the incorrect symbols and allow for building using multiple cores. Description of user facing changes: Description of developer facing changes: Description of development approach: Set the API signing token in the environment for the set scons args step in the Build NVDA job. this mirrors what is already done for the Create Launcher job. In the Create Launcher job, Split the powershell commands to install signpath out of the Create launcher step and into its own Install Signpath step. Add an Install Signpath step to the Build NVDA job, mirroring the step in the Create launcher job. In the Create launcher job, split out the building of the NVDA controller client from the Create Launcher step into its own Create NVDA controller client step. Remove the sconsCores variable from the scons launcher command in the Create Launcher step, meaning that scons launcher will always run with only one core, no matter what sconsCores is set to. It is very important that scons launcher never run with multiple cores, as if it ever has any dependencies, they must be build before the NSIS launcher builds. change the sconsCores variable to be --all-cores meaning that scons source, scons user_docs, scons client etc (but not scons launcher) will now run using all available cores). The biggest advantage here is signing files in scons source which dramatically decreases time as files can be signed in parallel to further source building.
Part of nvaccess#17010. Split out from nvaccess#19916 at reviewer request — this is the first of three PRs. The braille text-wrap refactor and syllable-aware wrap mode follow in separate PRs. Summary of the issue: NVDA lacks a locale-aware hyphenation API. One is needed to implement syllable-boundary braille text wrapping, so that long words can be broken at linguistically correct positions rather than always at the raw display edge. Description of user facing changes: None. This PR adds internal infrastructure only; no behaviour changes for users. Description of developer facing changes: New public function in textUtils.hyphenation: def getHyphenPositions(text: str, locale: str) -> tuple[int, ...] Returns the character offsets within text at which a hyphen may be inserted for the given locale. Returns an empty tuple for locales without a pyphen dictionary, logging a debug message once per locale per map lifetime, so callers can fall back cleanly without raising. A py2exe hook in source/setup.py bundles pyphen's hyph_*.dic files into dist/pyphenDictionaries/ and rewrites pyphen's dictionary lookup path at freeze time so the dictionaries are accessible in frozen builds. Description of development approach: LocaleDataMap (already used in NVDA for locale-aware character processing) handles locale fallback and caching. The _pyphenFactory function deliberately rejects region-subtag fallbacks — e.g. it will not silently serve en dictionaries for an en_US lookup — delegating that fallback logic to LocaleDataMap so region matching stays consistent with the rest of NVDA's locale handling.
Add GitHub Actions, C# and C/C++
Contributor
Author
|
cc @CrazySteve0605 @seanbudd I’ve completed most of Sean’s review suggestions in #19166. The remaining items are the user guide section and the question of whether For this setting, I just think we shouldn’t show it in the UI. A better approach might be a more user-transparent lazy load, but I’d like to hear @CrazySteve0605’s thoughts on that. Sean, I don’t think this is a blocking issue, so you can merge this and keep reviewing the remaining files. |
seanbudd
reviewed
May 20, 2026
Member
|
unit tests are failing it seems? |
Contributor
Author
|
Sorry, Fixed now. |
seanbudd
reviewed
May 20, 2026
|
|
||
| class ImmediateThread: | ||
| def __init__(self, target, args=None, kwargs=None, daemon=False): | ||
| def __init__(self, target, args=None, kwargs=None, daemon=False, name=None): |
Member
There was a problem hiding this comment.
can name get type hints (and the rest where easy)?
e25f593
into
nvaccess:try-chineseWordSegmentation-staging
33 of 37 checks passed
This was referenced May 20, 2026
cary-rowen
pushed a commit
to cary-rowen/nvda
that referenced
this pull request
May 27, 2026
commit e25f593 Author: wencong <manchen_0528@outlook.com> Date: Wed May 20 11:45:38 2026 +0800 Address review feedback for Chinese word segmentation (nvaccess#20178) This PR addresses selected review suggestions from Sean on PR nvaccess#19166 and syncs the branch with the latest master. Completed: - [x] Move the braille offset converter helper out of the nested function. - [x] Move the word segmentation braille converter import out of the local scope. - [x] Move the word segmentation initialization debug log into initialize(). - [x] Remove unsupported RuntimeError-specific handling around wordSeg.initialize(). - [x] Log word segmentation initializer resolution and execution failures. - [x] Add missing return type annotations for wordSegFlag properties. - [x] Remove redundant int annotations from IntFlag members. - [x] Move test helper objects out of nested scopes and add type hints. - [x] Move braille test configuration setup/cleanup into setUp and tearDown. - [x] Update the help ID casing for the word segmentation setting. - [x] Add a user guide section for the word segmentation setting. - [x] Move the Chinese word segmentation change log entries to 2026.3. - [x] Sync with the latest nvaccess/master. Remaining: - [ ] Confirm whether the new config option should be exposed in the UI. --------- Co-authored-by: Michael Curran <mick@nvaccess.org> Co-authored-by: Leonard de Ruijter <3049216+LeonarddeR@users.noreply.github.com> Co-authored-by: Sean Budd <sean@nvaccess.org> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> commit 887ad98 Merge: 9a6bc21 102a54e Author: Sean Budd <sean@nvaccess.org> Date: Mon May 18 16:23:14 2026 +1000 Merge branch 'master' into try-chineseWordSegmentation-staging commit 9a6bc21 Author: Wang Chong <306289287@qq.com> Date: Mon May 18 14:22:13 2026 +0800 Refactor Dicts Installation Logic in cppjieba's sconscript (nvaccess#20162) commit edf3429 Author: wencong <manchen_0528@outlook.com> Date: Tue May 12 08:26:06 2026 +0800 Remove reverted settings dialog debounce changes from Chinese word segmentation branch (nvaccess#20106) This removes settings dialog category-change debounce code that was accidentally reintroduced while resolving conflicts. That code was previously reverted by nvaccess#19846, so this branch should follow current master here and only keep the Document Navigation changes needed for Chinese word segmentation. This also removes an unrelated copyright-only change in browseMode.py. commit fab3060 Author: Wang Chong <306289287@qq.com> Date: Mon May 11 12:21:12 2026 +0800 Fixup merge master into Chinese Word Segmentation (nvaccess#20055) <!-- Please read and fill in the following template, for an explanation of the sections see: https://github.com/nvaccess/nvda/blob/master/projectDocs/dev/githubPullRequestTemplateExplanationAndExamples.md Please also note that the NVDA project has a Citizen and Contributor Code of Conduct which can be found at https://github.com/nvaccess/nvda/blob/master/CODE_OF_CONDUCT.md. NV Access expects that all contributors and other community members read and abide by the rules set out in this document while participating or contributing to this project. This includes creating or commenting on issues and pull requests. Please initially open PRs as a draft. When you would like a review, mark the PR as "ready for review". See https://github.com/nvaccess/nvda/blob/master/.github/CONTRIBUTING.md. --> <!-- Use Closes/Fixes/Resolves #xxx to link this PR to the issue it is responding to. --> <!-- This checklist is a reminder of things commonly forgotten in a new PR. Authors, please do a self-review of this pull-request. Check items to confirm you have thought about the relevance of the item. Where items are missing (eg unit / system tests), please explain in the PR. To check an item `- [ ]` becomes `- [x]`, note spacing. You can also check the checkboxes after the PR is created. A detailed explanation of this checklist is available here: https://github.com/nvaccess/nvda/blob/master/projectDocs/dev/githubPullRequestTemplateExplanationAndExamples.md#code-review-checklist --> - [ ] Documentation: - Change log entry - User Documentation - Developer / Technical Documentation - Context sensitive help for GUI changes - [ ] Testing: - Unit tests - System (end to end) tests - Manual testing - [ ] UX of all users considered: - Speech - Braille - Low Vision - Different web browsers - Localization in other languages / culture than English - [ ] API is compatible with existing add-ons. - [ ] Security precautions taken. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> commit 31c9585 Merge: 9edb45f d42d90c Author: Sean Budd <sean@nvaccess.org> Date: Mon May 4 09:25:02 2026 +1000 Merge master into chinese word segmentation (nvaccess#20041) <!-- Please read and fill in the following template, for an explanation of the sections see: https://github.com/nvaccess/nvda/blob/master/projectDocs/dev/githubPullRequestTemplateExplanationAndExamples.md Please also note that the NVDA project has a Citizen and Contributor Code of Conduct which can be found at https://github.com/nvaccess/nvda/blob/master/CODE_OF_CONDUCT.md. NV Access expects that all contributors and other community members read and abide by the rules set out in this document while participating or contributing to this project. This includes creating or commenting on issues and pull requests. Please initially open PRs as a draft. When you would like a review, mark the PR as "ready for review". See https://github.com/nvaccess/nvda/blob/master/.github/CONTRIBUTING.md. --> <!-- Use Closes/Fixes/Resolves #xxx to link this PR to the issue it is responding to. --> <!-- This checklist is a reminder of things commonly forgotten in a new PR. Authors, please do a self-review of this pull-request. Check items to confirm you have thought about the relevance of the item. Where items are missing (eg unit / system tests), please explain in the PR. To check an item `- [ ]` becomes `- [x]`, note spacing. You can also check the checkboxes after the PR is created. A detailed explanation of this checklist is available here: https://github.com/nvaccess/nvda/blob/master/projectDocs/dev/githubPullRequestTemplateExplanationAndExamples.md#code-review-checklist --> - [ ] Documentation: - Change log entry - User Documentation - Developer / Technical Documentation - Context sensitive help for GUI changes - [ ] Testing: - Unit tests - System (end to end) tests - Manual testing - [ ] UX of all users considered: - Speech - Braille - Low Vision - Different web browsers - Localization in other languages / culture than English - [ ] API is compatible with existing add-ons. - [ ] Security precautions taken. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions@github.com> Co-authored-by: Sascha Cowley <16543535+SaschaCowley@users.noreply.github.com> Co-authored-by: Peng-An Chen <andy72039@gmail.com> Co-authored-by: Sean Budd <sean@nvaccess.org> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Michael Curran <mick@nvaccess.org> Co-authored-by: GitHub Actions <actions@github.com> Co-authored-by: Bill Dengler <codeofdusk@gmail.com> Co-authored-by: Cyrille Bougot <cyrille.bougot2@laposte.net> Co-authored-by: Boumtchack <147637328+Boumtchack@users.noreply.github.com> Co-authored-by: Kefas Lungu <84945041+kefaslungu@users.noreply.github.com> Co-authored-by: Jani Kinnunen <janikinnunen340@gmail.com> Co-authored-by: Akash Kakkar <6381747+akash07k@users.noreply.github.com> Co-authored-by: makhlwf <78276231+makhlwf@users.noreply.github.com> Co-authored-by: Quin Gillespie <trypsynth@gmail.com> Co-authored-by: Bram Duvigneau <bram@bramd.nl> Co-authored-by: Luke Davis <8139760+XLTechie@users.noreply.github.com> Co-authored-by: audioses <67341000+audioses@users.noreply.github.com> Co-authored-by: gexgd0419 <55008943+gexgd0419@users.noreply.github.com> Co-authored-by: Cyrille Bougot <cyrille.bougot@laposte.net> Co-authored-by: Joseph Lee <joseph.lee22590@gmail.com> Co-authored-by: Christopher Proß <cpross@mailbox.org> Co-authored-by: Ryan McCleary <remccleary@gmail.com> Co-authored-by: wencong <manchen_0528@outlook.com> Co-authored-by: Emil-18 <135248352+Emil-18@users.noreply.github.com> Co-authored-by: Danil <50794055+Danstiv@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Noelia Ruiz Martínez <nrm1977@gmail.com> Co-authored-by: Sean Budd <seanbudd123@gmail.com> Co-authored-by: James Teh <jamie@jantrid.net> Co-authored-by: Thiago Seus <thiago.seus@yahoo.com.br> Co-authored-by: WangFeng Huang <1398969445@qq.com> Co-authored-by: ethanl-11 <124083447+ethanl-11@users.noreply.github.com> commit d42d90c Merge: 07d9b13 7f2d68f Author: cary-rowen <manchen_0528@outlook.com> Date: Fri May 1 22:21:05 2026 +0800 Merge latest master into chinese word segmentation branch commit 07d9b13 Merge: 9edb45f f7dc081 Author: cary-rowen <manchen_0528@outlook.com> Date: Mon Apr 20 12:03:50 2026 +0800 Merge latest master into try-chineseWordSegmentation-staging commit 9edb45f Merge: 096e985 4c9d616 Author: Sean Budd <sean@nvaccess.org> Date: Mon Apr 20 11:01:52 2026 +1000 Merge master into try-chineseWordSegmentation-staging (nvaccess#19747) commit 4c9d616 Author: cary-rowen <manchen_0528@outlook.com> Date: Thu Apr 9 22:19:56 2026 +0800 Fix textInfo word expansion test expectation commit 072b405 Author: cary-rowen <manchen_0528@outlook.com> Date: Thu Apr 9 22:00:30 2026 +0800 Fix word expansion without flowsTo commit 006277b Merge: 096e985 d8bf309 Author: Wang Chong <306289287@qq.com> Date: Fri Mar 6 00:35:58 2026 +0800 Merge branch 'master' commit 096e985 Author: Wang Chong <306289287@qq.com> Date: Mon Mar 2 12:02:27 2026 +0800 Fixup for Chinese Word Segmentation and Braille Output (nvaccess#19324) Summary of the issue: Some punctuations have extra separators (spaces) before or after them. Description of user facing changes: Braille output will be more accurate. commit 29d9f5a Merge: b50a0d5 9cafffb Author: Michael Curran <mick@nvaccess.org> Date: Thu Oct 30 12:12:25 2025 +1100 Merge pull request nvaccess#18865 from CrazySteve0605/brailleOutputForChinese blocked by nvaccess#18735, related nvaccess#1890 (comment), and [a part of OSPP 2025](https://summer-ospp.ac.cn/org/prodetail/25d3e0488?list=org&navpage=org) of NVDA. Braille output for Chinese become easier to read. The displayed braille and text are not aligned with each other in Braille Viewer, which shouldn't have an effect on reading from a braille displayer since only braille is transferred and the carets of raw text and output braille are aligned. <!-- This checklist is a reminder of things commonly forgotten in a new PR. Authors, please do a self-review of this pull-request. Check items to confirm you have thought about the relevance of the item. Where items are missing (eg unit / system tests), please explain in the PR. To check an item `- [ ]` becomes `- [x]`, note spacing. You can also check the checkboxes after the PR is created. A detailed explanation of this checklist is available here: https://github.com/nvaccess/nvda/blob/master/projectDocs/dev/githubPullRequestTemplateExplanationAndExamples.md#code-review-checklist --> - [ ] Documentation: - Change log entry - User Documentation - Developer / Technical Documentation - Context sensitive help for GUI changes - [ ] Testing: - Unit tests - System (end to end) tests - Manual testing - [ ] UX of all users considered: - Speech - Braille - Low Vision - Different web browsers - Localization in other languages / culture than English - [ ] API is compatible with existing add-ons. - [ ] Security precautions taken. commit 9cafffb Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue Oct 28 22:26:13 2025 +0000 Pre-commit auto-fix commit 0d27c91 Merge: d2714a3 b50a0d5 Author: Michael Curran <mick@nvaccess.org> Date: Wed Oct 29 08:23:08 2025 +1000 Merge branch 'try-chineseWordSegmentation-staging' into brailleOutputForChinese commit b50a0d5 Merge: c86b760 db90fff Author: Michael Curran <mick@nvaccess.org> Date: Wed Oct 29 09:20:19 2025 +1100 Merge pull request nvaccess#18735 from CrazySteve0605/wordNavigationForChineseText blocked by nvaccess#18548, closes nvaccess#4075, related nvaccess#16237 and [a part of OSPP 2025](https://summer-ospp.ac.cn/org/prodetail/25d3e0488?list=org&navpage=org) of NVDA. Chinese text can be navigated by word via system caret or review cursor. - [ ] update `textUtils` - [x] add `WordSegment` module - [x] create `WordSegmentationStrategy' as an abstract base class to select segmentation strategy based on text content, following Strategy Pattern - [x] implement `ChineseWordSegmentationStrategy` (for Chinese text) - [x] implement `UniscribeWordSegmentationStrategy` (for other languages as default strategy) - [x] update `textUtils/__init__.py` - [x] add `WordSegmenter` class for word segmentation, integrating segmentation strategies - [x] update `textInfos/offsets.py` - [x] replace `useUniscribe` with `useUniscribeForCharOffset` & `useWordSegmenterForWordOffset` for segmentation extensions - [x] integrate `WordSegmenter` for calculating word offsets - [ ] update document Word segmentation functionality was integrated in `OffsetsTextInfo`. In `OffsetsTextInfo`, word segmentation is based on Uniscribe by default and Unicode as a fall-back. Subclasses of OffsetsTextInfo 1. `NVDAObjectTextInfo` 2. `InputCompositionTextInfo`: 3. `JABTextInfo` 4. `SimpleResultTextInfo` 5. `VirtualBufferTextInfo`: use self-hosted function to calculate offset and invoke iits superclass' `_getWordOffsets` 1. `DisplayModelTextInfo`: straightly disable 2. `EditTextInfo`: straightly use Uniscribe 3. `ScintillaTextInfo`: entirely use self-defined 'word', for Scintilla-based editors such Notepad++ source/NVDAObjects/window/scintilla.py 4. `VsTextEditPaneTextInfo`: use a special COM automation API, for Microsoft Visual Studio and Microsoft SQL Server Management Studio source/appModules/devenv.py 5. `TextFrameTextInfo`: use self-defined 'word', based on PowerPoint's COM object, for PowerPoint's text frame source/appModules/powerpnt.py 6. `LwrTextInfo`: based on pre-computed words during related progress, for structured text using 'LineWordResult' (e.g. Windows OCR) source/contentRecog/__init__.py Some architectures totally or priorly use native difination of a 'word', which softwares depend on may not be able to use the new functionallity. 1. `IA2TextTextInfo`: override and fall back source/NVDAObjects/IAccessible/\_\_init\_\_.py <!-- This checklist is a reminder of things commonly forgotten in a new PR. Authors, please do a self-review of this pull-request. Check items to confirm you have thought about the relevance of the item. Where items are missing (eg unit / system tests), please explain in the PR. To check an item `- [ ]` becomes `- [x]`, note spacing. You can also check the checkboxes after the PR is created. A detailed explanation of this checklist is available here: https://github.com/nvaccess/nvda/blob/master/projectDocs/dev/githubPullRequestTemplateExplanationAndExamples.md#code-review-checklist --> - [ ] Documentation: - Change log entry - User Documentation - Developer / Technical Documentation - Context sensitive help for GUI changes - [ ] Testing: - Unit tests - System (end to end) tests - Manual testing - [ ] UX of all users considered: - Speech - Braille - Low Vision - Different web browsers - Localization in other languages / culture than English - [ ] API is compatible with existing add-ons. - [ ] Security precautions taken. commit db90fff Author: Wang Chong <306289287@qq.com> Date: Tue Oct 28 12:57:06 2025 +0800 remove duplicate importing lines commit d2714a3 Author: Wang Chong <306289287@qq.com> Date: Mon Oct 27 11:28:27 2025 +0800 fixup commit 042b778 Merge: b8ace76 c86b760 Author: Michael Curran <mick@nvaccess.org> Date: Mon Oct 27 11:47:18 2025 +1000 Merge branch 'try-chineseWordSegmentation-staging' into wordNavigationForChineseText commit c86b760 Merge: 4a4b1af 9e37e57 Author: Michael Curran <mick@nvaccess.org> Date: Mon Oct 27 11:45:35 2025 +1000 Merge branch 'master' into try-chineseWordSegmentation-staging commit b8ace76 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sat Oct 25 01:09:03 2025 +0000 Pre-commit auto-fix commit d32549f Author: Wang Chong <306289287@qq.com> Date: Sat Oct 25 09:03:22 2025 +0800 make word segmentation module reinitialized after settings are saved commit 2083095 Author: Wang Chong <306289287@qq.com> Date: Sat Oct 25 08:50:00 2025 +0800 fixup commit d55d077 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu Oct 9 21:19:50 2025 +0000 Pre-commit auto-fix commit 085ba2f Merge: 80b0472 4a4b1af Author: Michael Curran <mick@nvaccess.org> Date: Fri Oct 10 07:17:56 2025 +1000 Merge branch 'try-chineseWordSegmentation-staging' into wordNavigationForChineseText commit 4a4b1af Merge: 093a825 42dfcfe Author: Michael Curran <mick@nvaccess.org> Date: Fri Oct 10 07:13:22 2025 +1000 Merge branch 'master' into try-chineseWordSegmentation-staging commit 80b0472 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue Sep 30 05:12:16 2025 +0000 Pre-commit auto-fix commit f98b1b1 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue Sep 30 05:10:50 2025 +0000 Pre-commit auto-fix commit d1373b2 Author: Wang Chong <306289287@qq.com> Date: Tue Sep 30 13:06:54 2025 +0800 fixup commit 9ab3dba Merge: 251811e 0940a73 Author: Wang Chong <306289287@qq.com> Date: Tue Sep 30 13:06:22 2025 +0800 Merge branch 'wordNavigationForChineseText' into brailleOutputForChinese commit 0940a73 Author: Wang Chong <306289287@qq.com> Date: Tue Sep 30 12:59:46 2025 +0800 fixup commit c3a8562 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue Sep 30 03:25:34 2025 +0000 Pre-commit auto-fix commit 5e0e3fd Author: Wang Chong <306289287@qq.com> Date: Tue Sep 30 11:24:03 2025 +0800 simplify the logic for 'Auto' option in Word Segmentation Standard settings commit 552b42b Author: Wang Chong <306289287@qq.com> Date: Tue Sep 30 10:33:24 2025 +0800 fixup unittests commit 653e808 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon Sep 29 22:58:20 2025 +0000 Pre-commit auto-fix commit b40d709 Author: Wang Chong <306289287@qq.com> Date: Tue Sep 30 06:56:48 2025 +0800 revert `Initialize Word Segmenters for Unused Languages:` checkbox and fixup the initialization logic commit face4bd Merge: 6f586fd 093a825 Author: Michael Curran <mick@nvaccess.org> Date: Mon Sep 29 12:09:28 2025 +1000 Merge branch 'try-chineseWordSegmentation-staging' into wordNavigationForChineseText commit 093a825 Merge: 4bc116d b327e23 Author: Michael Curran <mick@nvaccess.org> Date: Mon Sep 29 12:07:17 2025 +1000 Merge branch 'integrateCppJieba' into try-chineseWordSegmentation-staging commit b327e23 Merge: 30e855f b236fe6 Author: Michael Curran <mick@nvaccess.org> Date: Mon Sep 29 08:38:17 2025 +1000 Merge branch 'try-chineseWordSegmentation-staging' into integrateCPPJieba commit 251811e Author: Wang Chong <306289287@qq.com> Date: Sun Sep 28 08:04:06 2025 +0800 update changelog commit 3b7bf5f Author: Wang Chong <306289287@qq.com> Date: Sun Sep 28 08:02:59 2025 +0800 correct and simplify the offset calculations commit 9304a39 Merge: 2d7c596 6f586fd Author: Wang Chong <306289287@qq.com> Date: Sun Sep 28 08:02:28 2025 +0800 Merge branch 'wordNavigationForChineseText' into brailleOutputForChinese commit 6f586fd Author: Wang Chong <306289287@qq.com> Date: Sun Sep 28 07:58:07 2025 +0800 update changelog commit b69d466 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 28 07:55:26 2025 +0800 fix up commit 9834b68 Author: Wang Chong <306289287@qq.com> Date: Sat Sep 27 09:16:37 2025 +0800 extract punctuation from `wordSegStrategy.py` to `wordSegUtils.py` commit 9479029 Author: Wang Chong <306289287@qq.com> Date: Sat Sep 27 09:15:51 2025 +0800 fixup commit f769457 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sat Sep 27 00:46:03 2025 +0000 Pre-commit auto-fix commit 2eec029 Author: Wang Chong <306289287@qq.com> Date: Sat Sep 27 08:40:57 2025 +0800 add unittest cases for `WordSegmenter` commit 43bfe03 Author: Wang Chong <306289287@qq.com> Date: Sat Sep 27 08:39:14 2025 +0800 make initialization of word segmenters conditional on language commit 111a24d Author: Wang Chong <306289287@qq.com> Date: Fri Sep 26 20:50:44 2025 +0800 update `wordSegSegmenter.py` to handle offsets at the end of the string commit 69617c4 Merge: 53b3870 fec70a9 Author: Wang Chong <306289287@qq.com> Date: Fri Sep 26 20:50:03 2025 +0800 Merge branch 'integrateCPPJieba' into wordNavigationForChineseText commit fec70a9 Merge: 30e855f b236fe6 Author: Wang Chong <306289287@qq.com> Date: Fri Sep 26 18:11:03 2025 +0800 Merge branch 'master' into integrateCPPJieba commit 53b3870 Author: Wang Chong <306289287@qq.com> Date: Thu Sep 25 23:09:55 2025 +0800 make `cppjieba` only available when NVDA's language is set to Chinese commit 250e700 Author: Wang Chong <306289287@qq.com> Date: Thu Sep 25 22:32:03 2025 +0800 update UI text for Uniscribe commit ccf07f9 Author: Wang Chong <306289287@qq.com> Date: Thu Sep 25 22:31:47 2025 +0800 correct method naming commit 38ec7ff Merge: dc23346 5562e70 Author: Wang Chong <306289287@qq.com> Date: Thu Sep 25 22:31:06 2025 +0800 Merge branch 'integrateCPPJieba' into wordNavigationForChineseText commit 5562e70 Merge: 30e855f 339af3e Author: Wang Chong <306289287@qq.com> Date: Thu Sep 25 22:29:07 2025 +0800 Merge branch 'master' into integrateCPPJieba commit 30e855f Merge: 2e730d6 dc22697 Author: Michael Curran <mick@nvaccess.org> Date: Mon Sep 22 18:20:36 2025 +1000 Merge branch 'try-chineseWordSegmentation-staging' into integrateCPPJieba commit dc23346 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 21 22:58:31 2025 +0800 Update source/core.py Co-authored-by: Cyrille Bougot <cyrille.bougot2@laposte.net> commit 9537999 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 21 22:52:10 2025 +0800 revert copyright header of `configSpec.py` commit 90660ba Author: Wang Chong <306289287@qq.com> Date: Sun Sep 21 22:24:18 2025 +0800 update `wordSegStrategy.py` commit 7ee08d0 Merge: a8955a3 2e730d6 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 21 22:22:54 2025 +0800 Merge branch 'integrateCPPJieba' into wordNavigationForChineseText commit a8955a3 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 21 22:19:19 2025 +0800 Revert "update module importing order and type annotations" This reverts commit 3bfbe59. commit 2e730d6 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 21 00:20:19 2025 +0800 Update .gitattributes Co-authored-by: Sean Budd <seanbudd123@gmail.com> commit 194a69e Author: Wang Chong <306289287@qq.com> Date: Sun Sep 21 00:10:40 2025 +0800 avoid using compilation time path commit c2cbb24 Author: Wang Chong <306289287@qq.com> Date: Sat Sep 20 23:53:56 2025 +0800 Revert "Update projectDocs/dev/createDevEnvironment.md" This reverts commit 06070c1. commit 0f507d5 Merge: b3e08ee 5a557d0 Author: Wang Chong <306289287@qq.com> Date: Sat Sep 20 23:53:19 2025 +0800 Merge branch 'master' into integrateCPPJieba commit bac3210 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sat Sep 13 01:33:24 2025 +0000 Pre-commit auto-fix commit 97eb6dd Author: Wang Chong <306289287@qq.com> Date: Sat Sep 13 09:19:40 2025 +0800 handle punctuation spacing commit 2b1d4b3 Merge: cf3e115 984b6eb Author: Wang Chong <306289287@qq.com> Date: Fri Sep 12 19:12:00 2025 +0800 Merge branch 'integrateCPPJieba' into wordNavigationForChineseText commit 984b6eb Merge: b3e08ee 3d74061 Author: Wang Chong <306289287@qq.com> Date: Fri Sep 12 19:09:24 2025 +0800 Merge branch 'master' into integrateCPPJieba commit cf3e115 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue Sep 9 16:49:47 2025 +0000 Pre-commit auto-fix commit 3a0badc Author: Wang Chong <306289287@qq.com> Date: Wed Sep 10 00:37:33 2025 +0800 update `wordSegStrategy.py` * add LRU caching commit f5087cc Merge: 3bfbe59 b3e08ee Author: Wang Chong <306289287@qq.com> Date: Wed Sep 10 00:24:14 2025 +0800 Merge branch 'integrateCPPJieba' into wordNavigationForChineseText commit b3e08ee Author: Wang Chong <306289287@qq.com> Date: Wed Sep 10 00:23:09 2025 +0800 update helper of `coojieba` commit bec5dc5 Merge: 09b1890 a210f97 Author: Wang Chong <306289287@qq.com> Date: Wed Sep 10 00:19:54 2025 +0800 Merge branch 'master' into integrateCPPJieba commit 09b1890 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 7 19:04:29 2025 +0800 fix building script commit 00796fe Author: Wang Chong <306289287@qq.com> Date: Sun Sep 7 18:39:59 2025 +0800 revert installing script commit 30120f8 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 7 18:38:36 2025 +0800 update building script commit 53158b6 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 7 18:17:50 2025 +0800 update changelog commit 3bfbe59 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 7 17:04:33 2025 +0800 update module importing order and type annotations commit b848e1b Author: Wang Chong <306289287@qq.com> Date: Sun Sep 7 16:28:01 2025 +0800 update `wordSegStrategy.py` commit abeb147 Merge: a1113d8 2955ca8 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 7 12:29:00 2025 +0800 Merge branch 'integrateCPPJieba' into wordNavigationForChineseText commit 2955ca8 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 7 12:28:14 2025 +0800 simplify helper of `cppjieba` * turn to build-in `Word` structure * remove some items we don't use commit a9281f6 Author: Wang Chong <306289287@qq.com> Date: Sat Sep 6 18:36:20 2025 +0800 update .gitattributes for .hpp header files commit 49cc1fe Author: Wang Chong <306289287@qq.com> Date: Sat Sep 6 18:31:05 2025 +0800 update cppjieba to the latest commit commit fe118ee Merge: 1869ed0 e6557a7 Author: Wang Chong <306289287@qq.com> Date: Sat Sep 6 18:27:32 2025 +0800 Merge branch 'master' into integrateCPPJieba commit 2d7c596 Author: Wang Chong <306289287@qq.com> Date: Fri Sep 5 00:09:16 2025 +0800 add word separator to optimize braille output for Chinese text * add a subclass of `OffsetConverter` to handle the offset mapping for raw text and separated one * add logic to invoke it when Chinese translation tables are used commit a1113d8 Author: Wang Chong <306289287@qq.com> Date: Thu Sep 4 14:07:24 2025 +0800 add `segmentedText` method commit c1fb4b8 Merge: 9e6a2e1 1869ed0 Author: Wang Chong <306289287@qq.com> Date: Thu Sep 4 12:50:22 2025 +0800 Merge branch 'integrateCPPJieba' into wordNavigationForChineseText commit 1869ed0 Author: Wang Chong <306289287@qq.com> Date: Thu Sep 4 12:41:29 2025 +0800 simplify the initialization of cppjieba drop off initialization of the keyword extractor which we don't need commit 11827fb Merge: 3e495d2 952db62 Author: Wang Chong <306289287@qq.com> Date: Thu Sep 4 09:51:52 2025 +0800 Merge branch 'master' into integrateCPPJieba commit 9e6a2e1 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu Aug 28 11:17:31 2025 +0000 Pre-commit auto-fix commit 3b2d835 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 28 19:11:05 2025 +0800 resolve deprecation commit a4edc9e Merge: 97b6db7 3e495d2 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 28 19:05:26 2025 +0800 Merge branch 'integrateCPPJieba' into wordNavigationForChineseText commit 3e495d2 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 28 18:13:11 2025 +0800 add wrappers for user dict management commit a643391 Merge: c853b64 c30a787 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 28 18:10:09 2025 +0800 Merge branch 'master' into integrateCPPJieba commit 97b6db7 Author: Wang Chong <306289287@qq.com> Date: Sun Aug 24 23:47:59 2025 +0800 update for pyright checks commit 4a680ea Author: Wang Chong <306289287@qq.com> Date: Sun Aug 24 23:45:48 2025 +0800 make "Auto" the default option for word navigation commit 356c11c Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sat Aug 23 15:35:45 2025 +0000 Pre-commit auto-fix commit 3ba56f0 Author: Wang Chong <306289287@qq.com> Date: Sat Aug 23 23:25:47 2025 +0800 add configuration for word navigation users can find it in NVDA settings -> Document Navigation -> Word Segmentation Method commit eeb96aa Author: Wang Chong <306289287@qq.com> Date: Sat Aug 23 20:33:54 2025 +0800 use multithreading for cppjieba's initialization commit 38b4bea Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu Aug 21 07:00:36 2025 +0000 Pre-commit auto-fix commit 3f54d62 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 21 14:59:22 2025 +0800 add initialization logic to wordSeg module - add an decorator to easily add initializers - extract `cppjieba`'s initializer to fit the decorator commit 8244a76 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 21 11:04:44 2025 +0800 make wordSegment module to make file structure clearer - create `wordSeg` package - migrate wordSegment module into wordSeg package and rename to wordSeg commit d69e8b7 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 21 10:44:37 2025 +0800 add trailing commas in multi-line constructs commit 3c65868 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 21 10:36:16 2025 +0800 update log commit ddd48e8 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 21 10:33:28 2025 +0800 add type annotations commit 676fc42 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 21 09:51:24 2025 +0800 add copyright header commit 0d40f0a Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed Aug 20 13:31:57 2025 +0000 Pre-commit auto-fix commit 407d4b2 Merge: 4adac07 adc22fb Author: Wang Chong <306289287@qq.com> Date: Wed Aug 20 21:27:13 2025 +0800 Merge branch 'integrateCPPJieba' into wordNavigationForChineseText commit 4adac07 Author: Wang Chong <306289287@qq.com> Date: Wed Aug 20 21:22:52 2025 +0800 update the word segmentation structure - redesign 2 properties of 'OffsetTextInfo' as enums to make them more configurable, inspired by @LeonarddeR - override them in some subclasses to simulate specific behaviors commit adc22fb Author: Wang Chong <306289287@qq.com> Date: Wed Aug 20 19:22:28 2025 +0800 add wrapper for word manager commit 19cad8a Merge: eba63ab 92f345e Author: Wang Chong <306289287@qq.com> Date: Wed Aug 20 17:35:03 2025 +0800 Merge branch 'master' into integrateCPPJieba commit f72d348 Author: Wang Chong <306289287@qq.com> Date: Mon Aug 18 15:06:19 2025 +0800 update type annotations commit 557f404 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon Aug 18 04:09:20 2025 +0000 Pre-commit auto-fix commit da64cd8 Author: Wang Chong <306289287@qq.com> Date: Mon Aug 18 12:02:49 2025 +0800 update `displayModel.py` make `DisplayModelTextInfo`'s flag aligned with its superclass commit 81f2040 Author: Wang Chong <306289287@qq.com> Date: Mon Aug 18 11:55:36 2025 +0800 update `textInfos/offsets.py` - replace `useUniscribe` with `useUniscribeForCharOffset` & `useWordSegmenterForWordOffset` for segmentation extensions - integrate `WordSegmenter` for calculating word offsets commit 9f62f04 Author: Wang Chong <306289287@qq.com> Date: Mon Aug 18 11:54:41 2025 +0800 update `textUtils/__init__.py` add `WordSegmenter` class for word segmentation, integrating segmentation strategies commit b0ac081 Author: Wang Chong <306289287@qq.com> Date: Mon Aug 18 11:38:48 2025 +0800 add `WordSegment` module - create `WordSegmentationStrategy' as an abstract base class to select segmentation strategy based on text content, following Strategy Pattern - implement `ChineseWordSegmentationStrategy` (for Chinese text) - implement `UniscribeWordSegmentationStrategy` (for other languages as default strategy) commit eba63ab Merge: 53dd3bb b9f19e6 Author: Wang Chong <306289287@qq.com> Date: Fri Aug 15 14:16:16 2025 +0800 Merge branch 'master' into integrateCPPJieba merge for Python updated commit 53dd3bb Merge: c853b64 b0241da Author: Wang Chong <306289287@qq.com> Date: Sat Aug 9 20:43:34 2025 +0800 Merge branch 'master' into integrateCPPJieba commit c853b64 Author: Wang Chong <306289287@qq.com> Date: Sat Aug 9 20:33:12 2025 +0800 Update include/readme.md Co-authored-by: Sean Budd <seanbudd123@gmail.com> commit c60c2da Author: Wang Chong <306289287@qq.com> Date: Sat Aug 9 20:30:58 2025 +0800 update copyright headers based on @seanbudd's suggestions commit 38a12dc Author: Wang Chong <306289287@qq.com> Date: Wed Aug 6 21:29:22 2025 +0800 Update building and setup script for cppjieba's dicts installation commit da662be Author: Wang Chong <306289287@qq.com> Date: Wed Aug 6 21:27:44 2025 +0800 Update .gitignore for cppjieba commit 0d92c08 Author: Wang Chong <306289287@qq.com> Date: Wed Aug 6 12:05:54 2025 +0800 Update GitHub action workflow to fetch cppjieba's submodule - Change 'submodules' in 'jobs - buildNVDA - Build NVDA - Checkout NVDA' from 'true' to 'recursive' to ensure cppjieba's submodule is fetched. - This will cause the submodule of sonic to be fetched as well, which seems currently unused. commit d4c3a92 Merge: f4cab8a 432364c Author: Wang Chong <306289287@qq.com> Date: Wed Aug 6 11:49:16 2025 +0800 Merge branch 'master' into integrateCPPJieba commit f4cab8a Merge: 7de7464 4ba948a Author: Wang Chong <306289287@qq.com> Date: Tue Aug 5 17:38:55 2025 +0800 Merge branch 'master' into integrateCPPJieba commit 7de7464 Author: Wang Chong <306289287@qq.com> Date: Tue Aug 5 12:52:39 2025 +0800 add JiebaSingleton wrapper and C API for NVDA segmentation - Introduce `JiebaSingleton` class in `cppjieba.hpp`/`cppjieba.cpp` with def file under nvdaHelper/cppjieba/' - Inherits from `cppjieba::Jieba` and exposes a thread-safe `getOffsets()` method - Implements Meyers’ singleton via `getInstance()` with a private constructor - Deletes copy constructor, copy assignment, move constructor, and move assignment to enforce single instance - Add C-style API in the same module: - `int initJieba()` to force singleton initialization - `int segmentOffsets(const char* text, int** charOffsets, int* outLen)` to perform segmentation and return character offsets - `void freeOffsets(int* ptr)` to release allocated offset buffer commit 1fbf05f Author: Wang Chong <306289287@qq.com> Date: Mon Aug 4 09:59:03 2025 +0800 add building script for cppjieba commit 3d4d9f1 Author: Wang Chong <306289287@qq.com> Date: Fri Aug 1 13:59:53 2025 +0800 Remove changes in sconscript for localLIb commit 2273a60 Author: Wang Chong <306289287@qq.com> Date: Fri Aug 1 13:06:08 2025 +0800 Update include/readme.md Co-authored-by: Sean Budd <seanbudd123@gmail.com> commit 06070c1 Author: Wang Chong <306289287@qq.com> Date: Fri Aug 1 13:04:22 2025 +0800 Update projectDocs/dev/createDevEnvironment.md Co-authored-by: Sean Budd <seanbudd123@gmail.com> commit ae58e9b Author: Wang Chong <306289287@qq.com> Date: Thu Jul 24 16:59:26 2025 +0800 Add comments for building script of cppjieba and its dependency commit fb4efef Author: Wang Chong <306289287@qq.com> Date: Thu Jul 24 16:51:49 2025 +0800 Update what's new commit 5cb5189 Author: Wang Chong <306289287@qq.com> Date: Wed Jul 23 17:30:56 2025 +0800 Introduce cppjieba as a submodule for Chinese word segmentation - Add `cppjieba` as a Git submodule under `third_party/cppjieba/` to provide robust Chinese word segmentation capabilities. - Update `.gitmodules` to point to the official `cppjieba` repository and configure it to track the `master` branch. - Update 'sconscript' to include the paths of 'cppjieba' and its dependency 'limonp' - Modify `copying.txt` to include the `cppjieba` license (MIT) alongside the project’s existing license, ensuring proper attribution and compliance. - Update documents
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR addresses selected review suggestions from Sean on PR #19166 and syncs the branch with the latest master.
Completed:
Remaining: