Skip to content

Fixup for Chinese Word Segmentation and Braille Output#19324

Merged
seanbudd merged 7 commits into
nvaccess:try-chineseWordSegmentation-stagingfrom
CrazySteve0605:fixupOSPP2025
Mar 2, 2026
Merged

Fixup for Chinese Word Segmentation and Braille Output#19324
seanbudd merged 7 commits into
nvaccess:try-chineseWordSegmentation-stagingfrom
CrazySteve0605:fixupOSPP2025

Conversation

@CrazySteve0605

@CrazySteve0605 CrazySteve0605 commented Dec 3, 2025

Copy link
Copy Markdown
Contributor

Fixup for Chinese word segmentation and braille output.

Link to issue number:

Summary of the issue:

Some punctuations have extra separators (spaces) before or after them.

Description of user facing changes:

Braille output will be more accurate.

Description of developer facing changes:

None

Description of development approach:

Testing strategy:

Run the updated code and check if the punctuation marks are handled the way they should be.

Known issues with pull request:

Code Review Checklist:

  • Documentation:
    • Change log entry
    • User Documentation
    • Developer / Technical Documentation
    • Context sensitive help for GUI changes
  • Testing:
    • Unit tests
    • System (end to end) tests
    • Manual testing
  • UX of all users considered:
    • Speech
    • Braille
    • Low Vision
    • Different web browsers
    • Localization in other languages / culture than English
  • API is compatible with existing add-ons.
  • Security precautions taken.

@seanbudd seanbudd added the conceptApproved Similar 'triaged' for issues, PR accepted in theory, implementation needs review. label Dec 4, 2025
@seanbudd

seanbudd commented Feb 3, 2026

Copy link
Copy Markdown
Member

@CrazySteve0605 - do you intend to continue to work on this?

@CrazySteve0605

Copy link
Copy Markdown
Contributor Author

Hi @seanbudd ,
Yes but I'm not sure when I can solve the issue. Currently, NVDA doesn't run successfully in launcher way but works well in source way, and I didn't find key information from log.

@seanbudd

seanbudd commented Feb 3, 2026

Copy link
Copy Markdown
Member

Can you explain what you mean by that? Are you downloading the launcher from this PR? Can you share the log after you run the PR? The system tests here show that the launcher runs fine

@cary-rowen

Copy link
Copy Markdown
Contributor

I tested it manually, and from my brief test, the word segmentation works fine. I believe you can continue with the next steps.

@CrazySteve0605 CrazySteve0605 marked this pull request as ready for review February 10, 2026 07:41
@CrazySteve0605 CrazySteve0605 requested a review from a team as a code owner February 10, 2026 07:41
@CrazySteve0605 CrazySteve0605 requested review from SaschaCowley and removed request for a team February 10, 2026 07:41
@CrazySteve0605

Copy link
Copy Markdown
Contributor Author

Now I'd like to make it ready for review.

@cary-rowen

Copy link
Copy Markdown
Contributor

Hi there, it looks like you might need to fill out the PR template and sync your branch with the latest changes from master . Also, please keep an eye on the unit test results, as some checks seem to be failing. Thanks!

@CrazySteve0605 CrazySteve0605 marked this pull request as draft February 11, 2026 02:54
@seanbudd

Copy link
Copy Markdown
Member

@CrazySteve0605 - do you intend to continue to work on this?

@CrazySteve0605

Copy link
Copy Markdown
Contributor Author

Hi @cary-rowen and @seanbudd,

I'd like to mark this PR as ready for review. Thanks for your continued attention to this PR.

@cary-rowen, regarding your feedback:

sync your branch with the latest changes from master.

This will be addressed in a separate PR (#19166).

keep an eye on the unit test results, as some checks seem to be failing.

The errors appear to be intermittent or possibly related to external dependencies. I ran the CI/CD in my fork (https://github.com/CrazySteve0605/nvda/actions/runs/21888959415) without any code modifications, and it passed successfully.

@CrazySteve0605 CrazySteve0605 marked this pull request as ready for review March 1, 2026 01:04
Comment thread source/textUtils/wordSeg/wordSegStrategy.py Outdated
Comment thread source/textUtils/wordSeg/wordSegStrategy.py
@seanbudd seanbudd merged commit 096e985 into nvaccess:try-chineseWordSegmentation-staging Mar 2, 2026
1 of 2 checks passed
@github-actions github-actions Bot added this to the 2026.2 milestone Mar 2, 2026
@CrazySteve0605 CrazySteve0605 deleted the fixupOSPP2025 branch March 5, 2026 16:40
cary-rowen pushed a commit to cary-rowen/nvda that referenced this pull request May 27, 2026
commit e25f593
Author: wencong <manchen_0528@outlook.com>
Date:   Wed May 20 11:45:38 2026 +0800

    Address review feedback for Chinese word segmentation (nvaccess#20178)

    This PR addresses selected review suggestions from Sean on PR nvaccess#19166 and
    syncs the branch with the latest master.

    Completed:
    - [x] Move the braille offset converter helper out of the nested
    function.
    - [x] Move the word segmentation braille converter import out of the
    local scope.
    - [x] Move the word segmentation initialization debug log into
    initialize().
    - [x] Remove unsupported RuntimeError-specific handling around
    wordSeg.initialize().
    - [x] Log word segmentation initializer resolution and execution
    failures.
    - [x] Add missing return type annotations for wordSegFlag properties.
    - [x] Remove redundant int annotations from IntFlag members.
    - [x] Move test helper objects out of nested scopes and add type hints.
    - [x] Move braille test configuration setup/cleanup into setUp and
    tearDown.
    - [x] Update the help ID casing for the word segmentation setting.
    - [x] Add a user guide section for the word segmentation setting.
    - [x] Move the Chinese word segmentation change log entries to 2026.3.
    - [x] Sync with the latest nvaccess/master.

    Remaining:
    - [ ] Confirm whether the new config option should be exposed in the UI.

    ---------

    Co-authored-by: Michael Curran <mick@nvaccess.org>
    Co-authored-by: Leonard de Ruijter <3049216+LeonarddeR@users.noreply.github.com>
    Co-authored-by: Sean Budd <sean@nvaccess.org>
    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

commit 887ad98
Merge: 9a6bc21 102a54e
Author: Sean Budd <sean@nvaccess.org>
Date:   Mon May 18 16:23:14 2026 +1000

    Merge branch 'master' into try-chineseWordSegmentation-staging

commit 9a6bc21
Author: Wang Chong <306289287@qq.com>
Date:   Mon May 18 14:22:13 2026 +0800

    Refactor Dicts Installation Logic in cppjieba's sconscript (nvaccess#20162)

commit edf3429
Author: wencong <manchen_0528@outlook.com>
Date:   Tue May 12 08:26:06 2026 +0800

    Remove reverted settings dialog debounce changes from Chinese word segmentation branch (nvaccess#20106)

    This removes settings dialog category-change debounce code that was
    accidentally reintroduced while resolving conflicts. That code was
    previously reverted by nvaccess#19846, so this branch should follow current
    master here and only keep the Document Navigation changes needed for
    Chinese word segmentation.

    This also removes an unrelated copyright-only change in browseMode.py.

commit fab3060
Author: Wang Chong <306289287@qq.com>
Date:   Mon May 11 12:21:12 2026 +0800

    Fixup merge master into Chinese Word Segmentation (nvaccess#20055)

    <!-- Please read and fill in the following template, for an explanation
    of the sections see:

    https://github.com/nvaccess/nvda/blob/master/projectDocs/dev/githubPullRequestTemplateExplanationAndExamples.md
    Please also note that the NVDA project has a Citizen and Contributor
    Code of Conduct which can be found at
    https://github.com/nvaccess/nvda/blob/master/CODE_OF_CONDUCT.md. NV
    Access expects that all contributors and other community members read
    and abide by the rules set out in this document while participating or
    contributing to this project. This includes creating or commenting on
    issues and pull requests.

    Please initially open PRs as a draft.
    When you would like a review, mark the PR as "ready for review".
    See
    https://github.com/nvaccess/nvda/blob/master/.github/CONTRIBUTING.md.
    -->
    <!-- Use Closes/Fixes/Resolves #xxx to link this PR to the issue it is
    responding to. -->

    <!--
    This checklist is a reminder of things commonly forgotten in a new PR.
    Authors, please do a self-review of this pull-request.
    Check items to confirm you have thought about the relevance of the item.
    Where items are missing (eg unit / system tests), please explain in the
    PR.
    To check an item `- [ ]` becomes `- [x]`, note spacing.
    You can also check the checkboxes after the PR is created.
    A detailed explanation of this checklist is available here:

    https://github.com/nvaccess/nvda/blob/master/projectDocs/dev/githubPullRequestTemplateExplanationAndExamples.md#code-review-checklist
    -->

    - [ ] Documentation:
      - Change log entry
      - User Documentation
      - Developer / Technical Documentation
      - Context sensitive help for GUI changes
    - [ ] Testing:
      - Unit tests
      - System (end to end) tests
      - Manual testing
    - [ ] UX of all users considered:
      - Speech
      - Braille
      - Low Vision
      - Different web browsers
      - Localization in other languages / culture than English
    - [ ] API is compatible with existing add-ons.
    - [ ] Security precautions taken.

    ---------

    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

commit 31c9585
Merge: 9edb45f d42d90c
Author: Sean Budd <sean@nvaccess.org>
Date:   Mon May 4 09:25:02 2026 +1000

    Merge master into chinese word segmentation (nvaccess#20041)

    <!-- Please read and fill in the following template, for an explanation
    of the sections see:

    https://github.com/nvaccess/nvda/blob/master/projectDocs/dev/githubPullRequestTemplateExplanationAndExamples.md
    Please also note that the NVDA project has a Citizen and Contributor
    Code of Conduct which can be found at
    https://github.com/nvaccess/nvda/blob/master/CODE_OF_CONDUCT.md. NV
    Access expects that all contributors and other community members read
    and abide by the rules set out in this document while participating or
    contributing to this project. This includes creating or commenting on
    issues and pull requests.

    Please initially open PRs as a draft.
    When you would like a review, mark the PR as "ready for review".
    See
    https://github.com/nvaccess/nvda/blob/master/.github/CONTRIBUTING.md.
    -->
    <!-- Use Closes/Fixes/Resolves #xxx to link this PR to the issue it is
    responding to. -->

    <!--
    This checklist is a reminder of things commonly forgotten in a new PR.
    Authors, please do a self-review of this pull-request.
    Check items to confirm you have thought about the relevance of the item.
    Where items are missing (eg unit / system tests), please explain in the
    PR.
    To check an item `- [ ]` becomes `- [x]`, note spacing.
    You can also check the checkboxes after the PR is created.
    A detailed explanation of this checklist is available here:

    https://github.com/nvaccess/nvda/blob/master/projectDocs/dev/githubPullRequestTemplateExplanationAndExamples.md#code-review-checklist
    -->

    - [ ] Documentation:
      - Change log entry
      - User Documentation
      - Developer / Technical Documentation
      - Context sensitive help for GUI changes
    - [ ] Testing:
      - Unit tests
      - System (end to end) tests
      - Manual testing
    - [ ] UX of all users considered:
      - Speech
      - Braille
      - Low Vision
      - Different web browsers
      - Localization in other languages / culture than English
    - [ ] API is compatible with existing add-ons.
    - [ ] Security precautions taken.

    ---------

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
    Co-authored-by: GitHub Actions <github-actions@github.com>
    Co-authored-by: Sascha Cowley <16543535+SaschaCowley@users.noreply.github.com>
    Co-authored-by: Peng-An Chen <andy72039@gmail.com>
    Co-authored-by: Sean Budd <sean@nvaccess.org>
    Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
    Co-authored-by: Michael Curran <mick@nvaccess.org>
    Co-authored-by: GitHub Actions <actions@github.com>
    Co-authored-by: Bill Dengler <codeofdusk@gmail.com>
    Co-authored-by: Cyrille Bougot <cyrille.bougot2@laposte.net>
    Co-authored-by: Boumtchack <147637328+Boumtchack@users.noreply.github.com>
    Co-authored-by: Kefas Lungu <84945041+kefaslungu@users.noreply.github.com>
    Co-authored-by: Jani Kinnunen <janikinnunen340@gmail.com>
    Co-authored-by: Akash Kakkar <6381747+akash07k@users.noreply.github.com>
    Co-authored-by: makhlwf <78276231+makhlwf@users.noreply.github.com>
    Co-authored-by: Quin Gillespie <trypsynth@gmail.com>
    Co-authored-by: Bram Duvigneau <bram@bramd.nl>
    Co-authored-by: Luke Davis <8139760+XLTechie@users.noreply.github.com>
    Co-authored-by: audioses <67341000+audioses@users.noreply.github.com>
    Co-authored-by: gexgd0419 <55008943+gexgd0419@users.noreply.github.com>
    Co-authored-by: Cyrille Bougot <cyrille.bougot@laposte.net>
    Co-authored-by: Joseph Lee <joseph.lee22590@gmail.com>
    Co-authored-by: Christopher Proß <cpross@mailbox.org>
    Co-authored-by: Ryan McCleary <remccleary@gmail.com>
    Co-authored-by: wencong <manchen_0528@outlook.com>
    Co-authored-by: Emil-18 <135248352+Emil-18@users.noreply.github.com>
    Co-authored-by: Danil <50794055+Danstiv@users.noreply.github.com>
    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
    Co-authored-by: Noelia Ruiz Martínez <nrm1977@gmail.com>
    Co-authored-by: Sean Budd <seanbudd123@gmail.com>
    Co-authored-by: James Teh <jamie@jantrid.net>
    Co-authored-by: Thiago Seus <thiago.seus@yahoo.com.br>
    Co-authored-by: WangFeng Huang <1398969445@qq.com>
    Co-authored-by: ethanl-11 <124083447+ethanl-11@users.noreply.github.com>

commit d42d90c
Merge: 07d9b13 7f2d68f
Author: cary-rowen <manchen_0528@outlook.com>
Date:   Fri May 1 22:21:05 2026 +0800

    Merge latest master into chinese word segmentation branch

commit 07d9b13
Merge: 9edb45f f7dc081
Author: cary-rowen <manchen_0528@outlook.com>
Date:   Mon Apr 20 12:03:50 2026 +0800

    Merge latest master into try-chineseWordSegmentation-staging

commit 9edb45f
Merge: 096e985 4c9d616
Author: Sean Budd <sean@nvaccess.org>
Date:   Mon Apr 20 11:01:52 2026 +1000

    Merge master into try-chineseWordSegmentation-staging (nvaccess#19747)

commit 4c9d616
Author: cary-rowen <manchen_0528@outlook.com>
Date:   Thu Apr 9 22:19:56 2026 +0800

    Fix textInfo word expansion test expectation

commit 072b405
Author: cary-rowen <manchen_0528@outlook.com>
Date:   Thu Apr 9 22:00:30 2026 +0800

    Fix word expansion without flowsTo

commit 006277b
Merge: 096e985 d8bf309
Author: Wang Chong <306289287@qq.com>
Date:   Fri Mar 6 00:35:58 2026 +0800

    Merge branch 'master'

commit 096e985
Author: Wang Chong <306289287@qq.com>
Date:   Mon Mar 2 12:02:27 2026 +0800

    Fixup for Chinese Word Segmentation and Braille Output (nvaccess#19324)

    Summary of the issue:

    Some punctuations have extra separators (spaces) before or after them.
    Description of user facing changes:

    Braille output will be more accurate.

commit 29d9f5a
Merge: b50a0d5 9cafffb
Author: Michael Curran <mick@nvaccess.org>
Date:   Thu Oct 30 12:12:25 2025 +1100

    Merge pull request nvaccess#18865 from CrazySteve0605/brailleOutputForChinese
    blocked by nvaccess#18735, related nvaccess#1890 (comment), and [a part of OSPP 2025](https://summer-ospp.ac.cn/org/prodetail/25d3e0488?list=org&navpage=org) of NVDA.
    Braille output for Chinese become easier to read.
    The displayed braille and text are not aligned with each other in Braille Viewer, which shouldn't have an effect on reading from a braille displayer since only braille is transferred and the carets of raw text and output braille are aligned.

    <!--
    This checklist is a reminder of things commonly forgotten in a new PR.
    Authors, please do a self-review of this pull-request.
    Check items to confirm you have thought about the relevance of the item.
    Where items are missing (eg unit / system tests), please explain in the PR.
    To check an item `- [ ]` becomes `- [x]`, note spacing.
    You can also check the checkboxes after the PR is created.
    A detailed explanation of this checklist is available here:
    https://github.com/nvaccess/nvda/blob/master/projectDocs/dev/githubPullRequestTemplateExplanationAndExamples.md#code-review-checklist
    -->

    - [ ] Documentation:
      - Change log entry
      - User Documentation
      - Developer / Technical Documentation
      - Context sensitive help for GUI changes
    - [ ] Testing:
      - Unit tests
      - System (end to end) tests
      - Manual testing
    - [ ] UX of all users considered:
      - Speech
      - Braille
      - Low Vision
      - Different web browsers
      - Localization in other languages / culture than English
    - [ ] API is compatible with existing add-ons.
    - [ ] Security precautions taken.

commit 9cafffb
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date:   Tue Oct 28 22:26:13 2025 +0000

    Pre-commit auto-fix

commit 0d27c91
Merge: d2714a3 b50a0d5
Author: Michael Curran <mick@nvaccess.org>
Date:   Wed Oct 29 08:23:08 2025 +1000

    Merge branch 'try-chineseWordSegmentation-staging' into brailleOutputForChinese

commit b50a0d5
Merge: c86b760 db90fff
Author: Michael Curran <mick@nvaccess.org>
Date:   Wed Oct 29 09:20:19 2025 +1100

    Merge pull request nvaccess#18735 from CrazySteve0605/wordNavigationForChineseText
    blocked by nvaccess#18548, closes nvaccess#4075, related nvaccess#16237 and [a part of OSPP 2025](https://summer-ospp.ac.cn/org/prodetail/25d3e0488?list=org&navpage=org) of NVDA.
    Chinese text can be navigated by word via system caret or review cursor.
    - [ ] update `textUtils`
    	- [x] add `WordSegment` module
    		- [x] create `WordSegmentationStrategy' as an abstract base class to select segmentation strategy based on text content, following Strategy Pattern
    		- [x] implement `ChineseWordSegmentationStrategy` (for Chinese text)
    		- [x] implement `UniscribeWordSegmentationStrategy` (for other languages as default strategy)
    	- [x] update `textUtils/__init__.py`
    		- [x] add `WordSegmenter` class for word segmentation, integrating segmentation strategies
    - [x] update `textInfos/offsets.py`
    	- [x] replace `useUniscribe` with `useUniscribeForCharOffset` & `useWordSegmenterForWordOffset` for segmentation extensions
    	- [x] integrate `WordSegmenter` for calculating word offsets
    - [ ] update document
    Word segmentation functionality was integrated in `OffsetsTextInfo`.
    In `OffsetsTextInfo`, word segmentation is based on Uniscribe by default and Unicode as a fall-back.

    Subclasses of OffsetsTextInfo
    1. `NVDAObjectTextInfo`
    2. `InputCompositionTextInfo`:
    3. `JABTextInfo`
    4. `SimpleResultTextInfo`
    5. `VirtualBufferTextInfo`: use self-hosted function to calculate offset and invoke iits superclass' `_getWordOffsets`
    1. `DisplayModelTextInfo`: straightly disable
    2. `EditTextInfo`: straightly use Uniscribe
    3. `ScintillaTextInfo`: entirely use self-defined 'word', for Scintilla-based editors such Notepad++  source/NVDAObjects/window/scintilla.py
    4. `VsTextEditPaneTextInfo`: use a special COM automation API, for Microsoft Visual Studio and Microsoft SQL Server Management Studio source/appModules/devenv.py
    5. `TextFrameTextInfo`: use self-defined 'word', based on PowerPoint's COM object, for PowerPoint's text frame  source/appModules/powerpnt.py
    6. `LwrTextInfo`: based on pre-computed words during related progress, for structured text using 'LineWordResult' (e.g. Windows OCR) source/contentRecog/__init__.py
    Some architectures totally or priorly use native difination of a 'word', which softwares depend on may not be able to use the new functionallity.
    1.  `IA2TextTextInfo`: override and fall back  source/NVDAObjects/IAccessible/\_\_init\_\_.py

    <!--
    This checklist is a reminder of things commonly forgotten in a new PR.
    Authors, please do a self-review of this pull-request.
    Check items to confirm you have thought about the relevance of the item.
    Where items are missing (eg unit / system tests), please explain in the PR.
    To check an item `- [ ]` becomes `- [x]`, note spacing.
    You can also check the checkboxes after the PR is created.
    A detailed explanation of this checklist is available here:
    https://github.com/nvaccess/nvda/blob/master/projectDocs/dev/githubPullRequestTemplateExplanationAndExamples.md#code-review-checklist
    -->

    - [ ] Documentation:
      - Change log entry
      - User Documentation
      - Developer / Technical Documentation
      - Context sensitive help for GUI changes
    - [ ] Testing:
      - Unit tests
      - System (end to end) tests
      - Manual testing
    - [ ] UX of all users considered:
      - Speech
      - Braille
      - Low Vision
      - Different web browsers
      - Localization in other languages / culture than English
    - [ ] API is compatible with existing add-ons.
    - [ ] Security precautions taken.

commit db90fff
Author: Wang Chong <306289287@qq.com>
Date:   Tue Oct 28 12:57:06 2025 +0800

    remove duplicate importing lines

commit d2714a3
Author: Wang Chong <306289287@qq.com>
Date:   Mon Oct 27 11:28:27 2025 +0800

    fixup

commit 042b778
Merge: b8ace76 c86b760
Author: Michael Curran <mick@nvaccess.org>
Date:   Mon Oct 27 11:47:18 2025 +1000

    Merge branch 'try-chineseWordSegmentation-staging' into wordNavigationForChineseText

commit c86b760
Merge: 4a4b1af 9e37e57
Author: Michael Curran <mick@nvaccess.org>
Date:   Mon Oct 27 11:45:35 2025 +1000

    Merge branch 'master' into try-chineseWordSegmentation-staging

commit b8ace76
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date:   Sat Oct 25 01:09:03 2025 +0000

    Pre-commit auto-fix

commit d32549f
Author: Wang Chong <306289287@qq.com>
Date:   Sat Oct 25 09:03:22 2025 +0800

    make word segmentation module reinitialized after settings are saved

commit 2083095
Author: Wang Chong <306289287@qq.com>
Date:   Sat Oct 25 08:50:00 2025 +0800

    fixup

commit d55d077
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date:   Thu Oct 9 21:19:50 2025 +0000

    Pre-commit auto-fix

commit 085ba2f
Merge: 80b0472 4a4b1af
Author: Michael Curran <mick@nvaccess.org>
Date:   Fri Oct 10 07:17:56 2025 +1000

    Merge branch 'try-chineseWordSegmentation-staging' into wordNavigationForChineseText

commit 4a4b1af
Merge: 093a825 42dfcfe
Author: Michael Curran <mick@nvaccess.org>
Date:   Fri Oct 10 07:13:22 2025 +1000

    Merge branch 'master' into try-chineseWordSegmentation-staging

commit 80b0472
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date:   Tue Sep 30 05:12:16 2025 +0000

    Pre-commit auto-fix

commit f98b1b1
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date:   Tue Sep 30 05:10:50 2025 +0000

    Pre-commit auto-fix

commit d1373b2
Author: Wang Chong <306289287@qq.com>
Date:   Tue Sep 30 13:06:54 2025 +0800

    fixup

commit 9ab3dba
Merge: 251811e 0940a73
Author: Wang Chong <306289287@qq.com>
Date:   Tue Sep 30 13:06:22 2025 +0800

    Merge branch 'wordNavigationForChineseText' into brailleOutputForChinese

commit 0940a73
Author: Wang Chong <306289287@qq.com>
Date:   Tue Sep 30 12:59:46 2025 +0800

    fixup

commit c3a8562
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date:   Tue Sep 30 03:25:34 2025 +0000

    Pre-commit auto-fix

commit 5e0e3fd
Author: Wang Chong <306289287@qq.com>
Date:   Tue Sep 30 11:24:03 2025 +0800

    simplify the logic for 'Auto' option in Word Segmentation Standard settings

commit 552b42b
Author: Wang Chong <306289287@qq.com>
Date:   Tue Sep 30 10:33:24 2025 +0800

    fixup unittests

commit 653e808
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date:   Mon Sep 29 22:58:20 2025 +0000

    Pre-commit auto-fix

commit b40d709
Author: Wang Chong <306289287@qq.com>
Date:   Tue Sep 30 06:56:48 2025 +0800

    revert `Initialize Word Segmenters for Unused Languages:` checkbox and fixup the initialization logic

commit face4bd
Merge: 6f586fd 093a825
Author: Michael Curran <mick@nvaccess.org>
Date:   Mon Sep 29 12:09:28 2025 +1000

    Merge branch 'try-chineseWordSegmentation-staging' into wordNavigationForChineseText

commit 093a825
Merge: 4bc116d b327e23
Author: Michael Curran <mick@nvaccess.org>
Date:   Mon Sep 29 12:07:17 2025 +1000

    Merge branch 'integrateCppJieba' into try-chineseWordSegmentation-staging

commit b327e23
Merge: 30e855f b236fe6
Author: Michael Curran <mick@nvaccess.org>
Date:   Mon Sep 29 08:38:17 2025 +1000

    Merge branch 'try-chineseWordSegmentation-staging' into integrateCPPJieba

commit 251811e
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 28 08:04:06 2025 +0800

    update changelog

commit 3b7bf5f
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 28 08:02:59 2025 +0800

    correct and simplify the offset calculations

commit 9304a39
Merge: 2d7c596 6f586fd
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 28 08:02:28 2025 +0800

    Merge branch 'wordNavigationForChineseText' into brailleOutputForChinese

commit 6f586fd
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 28 07:58:07 2025 +0800

    update changelog

commit b69d466
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 28 07:55:26 2025 +0800

    fix up

commit 9834b68
Author: Wang Chong <306289287@qq.com>
Date:   Sat Sep 27 09:16:37 2025 +0800

    extract punctuation from `wordSegStrategy.py` to `wordSegUtils.py`

commit 9479029
Author: Wang Chong <306289287@qq.com>
Date:   Sat Sep 27 09:15:51 2025 +0800

    fixup

commit f769457
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date:   Sat Sep 27 00:46:03 2025 +0000

    Pre-commit auto-fix

commit 2eec029
Author: Wang Chong <306289287@qq.com>
Date:   Sat Sep 27 08:40:57 2025 +0800

    add unittest cases for `WordSegmenter`

commit 43bfe03
Author: Wang Chong <306289287@qq.com>
Date:   Sat Sep 27 08:39:14 2025 +0800

    make initialization of word segmenters conditional on language

commit 111a24d
Author: Wang Chong <306289287@qq.com>
Date:   Fri Sep 26 20:50:44 2025 +0800

    update `wordSegSegmenter.py` to handle offsets at the end of the string

commit 69617c4
Merge: 53b3870 fec70a9
Author: Wang Chong <306289287@qq.com>
Date:   Fri Sep 26 20:50:03 2025 +0800

    Merge branch 'integrateCPPJieba' into wordNavigationForChineseText

commit fec70a9
Merge: 30e855f b236fe6
Author: Wang Chong <306289287@qq.com>
Date:   Fri Sep 26 18:11:03 2025 +0800

    Merge branch 'master' into integrateCPPJieba

commit 53b3870
Author: Wang Chong <306289287@qq.com>
Date:   Thu Sep 25 23:09:55 2025 +0800

    make `cppjieba` only available when NVDA's language is set to Chinese

commit 250e700
Author: Wang Chong <306289287@qq.com>
Date:   Thu Sep 25 22:32:03 2025 +0800

    update UI text for Uniscribe

commit ccf07f9
Author: Wang Chong <306289287@qq.com>
Date:   Thu Sep 25 22:31:47 2025 +0800

    correct method naming

commit 38ec7ff
Merge: dc23346 5562e70
Author: Wang Chong <306289287@qq.com>
Date:   Thu Sep 25 22:31:06 2025 +0800

    Merge branch 'integrateCPPJieba' into wordNavigationForChineseText

commit 5562e70
Merge: 30e855f 339af3e
Author: Wang Chong <306289287@qq.com>
Date:   Thu Sep 25 22:29:07 2025 +0800

    Merge branch 'master' into integrateCPPJieba

commit 30e855f
Merge: 2e730d6 dc22697
Author: Michael Curran <mick@nvaccess.org>
Date:   Mon Sep 22 18:20:36 2025 +1000

    Merge branch 'try-chineseWordSegmentation-staging' into integrateCPPJieba

commit dc23346
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 21 22:58:31 2025 +0800

    Update source/core.py

    Co-authored-by: Cyrille Bougot <cyrille.bougot2@laposte.net>

commit 9537999
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 21 22:52:10 2025 +0800

    revert copyright header of `configSpec.py`

commit 90660ba
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 21 22:24:18 2025 +0800

    update `wordSegStrategy.py`

commit 7ee08d0
Merge: a8955a3 2e730d6
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 21 22:22:54 2025 +0800

    Merge branch 'integrateCPPJieba' into wordNavigationForChineseText

commit a8955a3
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 21 22:19:19 2025 +0800

    Revert "update module importing order and type annotations"

    This reverts commit 3bfbe59.

commit 2e730d6
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 21 00:20:19 2025 +0800

    Update .gitattributes

    Co-authored-by: Sean Budd <seanbudd123@gmail.com>

commit 194a69e
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 21 00:10:40 2025 +0800

    avoid using compilation time path

commit c2cbb24
Author: Wang Chong <306289287@qq.com>
Date:   Sat Sep 20 23:53:56 2025 +0800

    Revert "Update projectDocs/dev/createDevEnvironment.md"

    This reverts commit 06070c1.

commit 0f507d5
Merge: b3e08ee 5a557d0
Author: Wang Chong <306289287@qq.com>
Date:   Sat Sep 20 23:53:19 2025 +0800

    Merge branch 'master' into integrateCPPJieba

commit bac3210
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date:   Sat Sep 13 01:33:24 2025 +0000

    Pre-commit auto-fix

commit 97eb6dd
Author: Wang Chong <306289287@qq.com>
Date:   Sat Sep 13 09:19:40 2025 +0800

    handle punctuation spacing

commit 2b1d4b3
Merge: cf3e115 984b6eb
Author: Wang Chong <306289287@qq.com>
Date:   Fri Sep 12 19:12:00 2025 +0800

    Merge branch 'integrateCPPJieba' into wordNavigationForChineseText

commit 984b6eb
Merge: b3e08ee 3d74061
Author: Wang Chong <306289287@qq.com>
Date:   Fri Sep 12 19:09:24 2025 +0800

    Merge branch 'master' into integrateCPPJieba

commit cf3e115
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date:   Tue Sep 9 16:49:47 2025 +0000

    Pre-commit auto-fix

commit 3a0badc
Author: Wang Chong <306289287@qq.com>
Date:   Wed Sep 10 00:37:33 2025 +0800

    update `wordSegStrategy.py`

    * add LRU caching

commit f5087cc
Merge: 3bfbe59 b3e08ee
Author: Wang Chong <306289287@qq.com>
Date:   Wed Sep 10 00:24:14 2025 +0800

    Merge branch 'integrateCPPJieba' into wordNavigationForChineseText

commit b3e08ee
Author: Wang Chong <306289287@qq.com>
Date:   Wed Sep 10 00:23:09 2025 +0800

    update helper of `coojieba`

commit bec5dc5
Merge: 09b1890 a210f97
Author: Wang Chong <306289287@qq.com>
Date:   Wed Sep 10 00:19:54 2025 +0800

    Merge branch 'master' into integrateCPPJieba

commit 09b1890
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 7 19:04:29 2025 +0800

    fix building script

commit 00796fe
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 7 18:39:59 2025 +0800

    revert installing script

commit 30120f8
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 7 18:38:36 2025 +0800

    update building script

commit 53158b6
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 7 18:17:50 2025 +0800

    update changelog

commit 3bfbe59
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 7 17:04:33 2025 +0800

    update module importing order and type annotations

commit b848e1b
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 7 16:28:01 2025 +0800

    update `wordSegStrategy.py`

commit abeb147
Merge: a1113d8 2955ca8
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 7 12:29:00 2025 +0800

    Merge branch 'integrateCPPJieba' into wordNavigationForChineseText

commit 2955ca8
Author: Wang Chong <306289287@qq.com>
Date:   Sun Sep 7 12:28:14 2025 +0800

    simplify helper of `cppjieba`

    * turn to build-in `Word` structure
    * remove some items we don't use

commit a9281f6
Author: Wang Chong <306289287@qq.com>
Date:   Sat Sep 6 18:36:20 2025 +0800

    update .gitattributes for .hpp header files

commit 49cc1fe
Author: Wang Chong <306289287@qq.com>
Date:   Sat Sep 6 18:31:05 2025 +0800

    update cppjieba to the latest commit

commit fe118ee
Merge: 1869ed0 e6557a7
Author: Wang Chong <306289287@qq.com>
Date:   Sat Sep 6 18:27:32 2025 +0800

    Merge branch 'master' into integrateCPPJieba

commit 2d7c596
Author: Wang Chong <306289287@qq.com>
Date:   Fri Sep 5 00:09:16 2025 +0800

    add word separator to optimize braille output for Chinese text

    * add a subclass of `OffsetConverter` to handle the offset mapping for raw text and separated one
    * add logic to invoke it when Chinese translation tables are used

commit a1113d8
Author: Wang Chong <306289287@qq.com>
Date:   Thu Sep 4 14:07:24 2025 +0800

    add `segmentedText` method

commit c1fb4b8
Merge: 9e6a2e1 1869ed0
Author: Wang Chong <306289287@qq.com>
Date:   Thu Sep 4 12:50:22 2025 +0800

    Merge branch 'integrateCPPJieba' into wordNavigationForChineseText

commit 1869ed0
Author: Wang Chong <306289287@qq.com>
Date:   Thu Sep 4 12:41:29 2025 +0800

    simplify the initialization of cppjieba

    drop off initialization of the keyword extractor which we don't need

commit 11827fb
Merge: 3e495d2 952db62
Author: Wang Chong <306289287@qq.com>
Date:   Thu Sep 4 09:51:52 2025 +0800

    Merge branch 'master' into integrateCPPJieba

commit 9e6a2e1
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date:   Thu Aug 28 11:17:31 2025 +0000

    Pre-commit auto-fix

commit 3b2d835
Author: Wang Chong <306289287@qq.com>
Date:   Thu Aug 28 19:11:05 2025 +0800

     resolve deprecation

commit a4edc9e
Merge: 97b6db7 3e495d2
Author: Wang Chong <306289287@qq.com>
Date:   Thu Aug 28 19:05:26 2025 +0800

    Merge branch 'integrateCPPJieba' into wordNavigationForChineseText

commit 3e495d2
Author: Wang Chong <306289287@qq.com>
Date:   Thu Aug 28 18:13:11 2025 +0800

    add wrappers for user dict management

commit a643391
Merge: c853b64 c30a787
Author: Wang Chong <306289287@qq.com>
Date:   Thu Aug 28 18:10:09 2025 +0800

    Merge branch 'master' into integrateCPPJieba

commit 97b6db7
Author: Wang Chong <306289287@qq.com>
Date:   Sun Aug 24 23:47:59 2025 +0800

    update for pyright checks

commit 4a680ea
Author: Wang Chong <306289287@qq.com>
Date:   Sun Aug 24 23:45:48 2025 +0800

    make "Auto" the default option for word navigation

commit 356c11c
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date:   Sat Aug 23 15:35:45 2025 +0000

    Pre-commit auto-fix

commit 3ba56f0
Author: Wang Chong <306289287@qq.com>
Date:   Sat Aug 23 23:25:47 2025 +0800

    add configuration for word navigation

    users can find it in NVDA settings -> Document Navigation -> Word Segmentation Method

commit eeb96aa
Author: Wang Chong <306289287@qq.com>
Date:   Sat Aug 23 20:33:54 2025 +0800

    use multithreading for cppjieba's initialization

commit 38b4bea
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date:   Thu Aug 21 07:00:36 2025 +0000

    Pre-commit auto-fix

commit 3f54d62
Author: Wang Chong <306289287@qq.com>
Date:   Thu Aug 21 14:59:22 2025 +0800

    add initialization logic to wordSeg module

    - add an decorator to easily add initializers
    - extract `cppjieba`'s initializer to fit the decorator

commit 8244a76
Author: Wang Chong <306289287@qq.com>
Date:   Thu Aug 21 11:04:44 2025 +0800

    make wordSegment module to make file structure clearer

    - create `wordSeg` package
    - migrate wordSegment module into wordSeg package and rename to wordSeg

commit d69e8b7
Author: Wang Chong <306289287@qq.com>
Date:   Thu Aug 21 10:44:37 2025 +0800

    add trailing commas in multi-line constructs

commit 3c65868
Author: Wang Chong <306289287@qq.com>
Date:   Thu Aug 21 10:36:16 2025 +0800

    update log

commit ddd48e8
Author: Wang Chong <306289287@qq.com>
Date:   Thu Aug 21 10:33:28 2025 +0800

    add type annotations

commit 676fc42
Author: Wang Chong <306289287@qq.com>
Date:   Thu Aug 21 09:51:24 2025 +0800

    add copyright header

commit 0d40f0a
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date:   Wed Aug 20 13:31:57 2025 +0000

    Pre-commit auto-fix

commit 407d4b2
Merge: 4adac07 adc22fb
Author: Wang Chong <306289287@qq.com>
Date:   Wed Aug 20 21:27:13 2025 +0800

    Merge branch 'integrateCPPJieba' into wordNavigationForChineseText

commit 4adac07
Author: Wang Chong <306289287@qq.com>
Date:   Wed Aug 20 21:22:52 2025 +0800

    update the word segmentation structure

    - redesign 2 properties of 'OffsetTextInfo' as enums to make them more configurable, inspired by @LeonarddeR
    - override them in some subclasses to simulate specific behaviors

commit adc22fb
Author: Wang Chong <306289287@qq.com>
Date:   Wed Aug 20 19:22:28 2025 +0800

    add wrapper for word manager

commit 19cad8a
Merge: eba63ab 92f345e
Author: Wang Chong <306289287@qq.com>
Date:   Wed Aug 20 17:35:03 2025 +0800

    Merge branch 'master' into integrateCPPJieba

commit f72d348
Author: Wang Chong <306289287@qq.com>
Date:   Mon Aug 18 15:06:19 2025 +0800

    update type annotations

commit 557f404
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date:   Mon Aug 18 04:09:20 2025 +0000

    Pre-commit auto-fix

commit da64cd8
Author: Wang Chong <306289287@qq.com>
Date:   Mon Aug 18 12:02:49 2025 +0800

    update `displayModel.py`

    make `DisplayModelTextInfo`'s flag aligned with its superclass

commit 81f2040
Author: Wang Chong <306289287@qq.com>
Date:   Mon Aug 18 11:55:36 2025 +0800

    update `textInfos/offsets.py`

    - replace `useUniscribe` with `useUniscribeForCharOffset` & `useWordSegmenterForWordOffset` for segmentation extensions
    - integrate `WordSegmenter` for calculating word offsets

commit 9f62f04
Author: Wang Chong <306289287@qq.com>
Date:   Mon Aug 18 11:54:41 2025 +0800

    update `textUtils/__init__.py`

    add `WordSegmenter` class for word segmentation, integrating segmentation strategies

commit b0ac081
Author: Wang Chong <306289287@qq.com>
Date:   Mon Aug 18 11:38:48 2025 +0800

    add `WordSegment` module

    - create `WordSegmentationStrategy' as an abstract base class to select segmentation strategy based on text content, following Strategy Pattern
    - implement `ChineseWordSegmentationStrategy` (for Chinese text)
    - implement `UniscribeWordSegmentationStrategy` (for other languages as default strategy)

commit eba63ab
Merge: 53dd3bb b9f19e6
Author: Wang Chong <306289287@qq.com>
Date:   Fri Aug 15 14:16:16 2025 +0800

    Merge branch 'master' into integrateCPPJieba

    merge for Python updated

commit 53dd3bb
Merge: c853b64 b0241da
Author: Wang Chong <306289287@qq.com>
Date:   Sat Aug 9 20:43:34 2025 +0800

    Merge branch 'master' into integrateCPPJieba

commit c853b64
Author: Wang Chong <306289287@qq.com>
Date:   Sat Aug 9 20:33:12 2025 +0800

    Update include/readme.md

    Co-authored-by: Sean Budd <seanbudd123@gmail.com>

commit c60c2da
Author: Wang Chong <306289287@qq.com>
Date:   Sat Aug 9 20:30:58 2025 +0800

    update copyright headers based on @seanbudd's suggestions

commit 38a12dc
Author: Wang Chong <306289287@qq.com>
Date:   Wed Aug 6 21:29:22 2025 +0800

    Update building and setup script for cppjieba's dicts installation

commit da662be
Author: Wang Chong <306289287@qq.com>
Date:   Wed Aug 6 21:27:44 2025 +0800

    Update .gitignore for cppjieba

commit 0d92c08
Author: Wang Chong <306289287@qq.com>
Date:   Wed Aug 6 12:05:54 2025 +0800

    Update GitHub action workflow to fetch cppjieba's submodule

    - Change 'submodules' in 'jobs - buildNVDA - Build NVDA - Checkout NVDA' from 'true' to 'recursive' to ensure cppjieba's submodule is fetched.
    - This will cause the submodule of sonic to be fetched as well, which seems currently unused.

commit d4c3a92
Merge: f4cab8a 432364c
Author: Wang Chong <306289287@qq.com>
Date:   Wed Aug 6 11:49:16 2025 +0800

    Merge branch 'master' into integrateCPPJieba

commit f4cab8a
Merge: 7de7464 4ba948a
Author: Wang Chong <306289287@qq.com>
Date:   Tue Aug 5 17:38:55 2025 +0800

    Merge branch 'master' into integrateCPPJieba

commit 7de7464
Author: Wang Chong <306289287@qq.com>
Date:   Tue Aug 5 12:52:39 2025 +0800

    add JiebaSingleton wrapper and C API for NVDA segmentation

    - Introduce `JiebaSingleton` class in `cppjieba.hpp`/`cppjieba.cpp` with def file under nvdaHelper/cppjieba/'
      - Inherits from `cppjieba::Jieba` and exposes a thread-safe `getOffsets()` method
      - Implements Meyers’ singleton via `getInstance()` with a private constructor
      - Deletes copy constructor, copy assignment, move constructor, and move assignment to enforce single instance
    - Add C-style API in the same module:
      - `int initJieba()` to force singleton initialization
      - `int segmentOffsets(const char* text, int** charOffsets, int* outLen)` to perform segmentation and return character offsets
      - `void freeOffsets(int* ptr)` to release allocated offset buffer

commit 1fbf05f
Author: Wang Chong <306289287@qq.com>
Date:   Mon Aug 4 09:59:03 2025 +0800

    add building script for cppjieba

commit 3d4d9f1
Author: Wang Chong <306289287@qq.com>
Date:   Fri Aug 1 13:59:53 2025 +0800

    Remove changes in sconscript for localLIb

commit 2273a60
Author: Wang Chong <306289287@qq.com>
Date:   Fri Aug 1 13:06:08 2025 +0800

    Update include/readme.md

    Co-authored-by: Sean Budd <seanbudd123@gmail.com>

commit 06070c1
Author: Wang Chong <306289287@qq.com>
Date:   Fri Aug 1 13:04:22 2025 +0800

    Update projectDocs/dev/createDevEnvironment.md

    Co-authored-by: Sean Budd <seanbudd123@gmail.com>

commit ae58e9b
Author: Wang Chong <306289287@qq.com>
Date:   Thu Jul 24 16:59:26 2025 +0800

    Add comments for building script of cppjieba and its dependency

commit fb4efef
Author: Wang Chong <306289287@qq.com>
Date:   Thu Jul 24 16:51:49 2025 +0800

    Update what's new

commit 5cb5189
Author: Wang Chong <306289287@qq.com>
Date:   Wed Jul 23 17:30:56 2025 +0800

    Introduce cppjieba as a submodule for Chinese word segmentation

    - Add `cppjieba` as a Git submodule under `third_party/cppjieba/` to provide
      robust Chinese word segmentation capabilities.
    - Update `.gitmodules` to point to the official `cppjieba` repository and
      configure it to track the `master` branch.
    - Update 'sconscript' to include the paths of 'cppjieba' and its dependency 'limonp'
    - Modify `copying.txt` to include the `cppjieba` license (MIT) alongside the
      project’s existing license, ensuring proper attribution and compliance.
    - Update documents
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conceptApproved Similar 'triaged' for issues, PR accepted in theory, implementation needs review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants