[WIP] Merge Chinese Word Segmentation work by seanbudd · Pull Request #19166 · nvaccess/nvda

seanbudd · 2025-11-05T01:53:21Z

Merges:

- Add `cppjieba` as a Git submodule under `third_party/cppjieba/` to provide robust Chinese word segmentation capabilities. - Update `.gitmodules` to point to the official `cppjieba` repository and configure it to track the `master` branch. - Update 'sconscript' to include the paths of 'cppjieba' and its dependency 'limonp' - Modify `copying.txt` to include the `cppjieba` license (MIT) alongside the project’s existing license, ensuring proper attribution and compliance. - Update documents

Co-authored-by: Sean Budd <seanbudd123@gmail.com>

- Introduce `JiebaSingleton` class in `cppjieba.hpp`/`cppjieba.cpp` with def file under nvdaHelper/cppjieba/' - Inherits from `cppjieba::Jieba` and exposes a thread-safe `getOffsets()` method - Implements Meyers’ singleton via `getInstance()` with a private constructor - Deletes copy constructor, copy assignment, move constructor, and move assignment to enforce single instance - Add C-style API in the same module: - `int initJieba()` to force singleton initialization - `int segmentOffsets(const char* text, int** charOffsets, int* outLen)` to perform segmentation and return character offsets - `void freeOffsets(int* ptr)` to release allocated offset buffer

- Change 'submodules' in 'jobs - buildNVDA - Build NVDA - Checkout NVDA' from 'true' to 'recursive' to ensure cppjieba's submodule is fetched. - This will cause the submodule of sonic to be fetched as well, which seems currently unused.

Co-authored-by: Sean Budd <seanbudd123@gmail.com>

merge for Python updated

- create `WordSegmentationStrategy' as an abstract base class to select segmentation strategy based on text content, following Strategy Pattern - implement `ChineseWordSegmentationStrategy` (for Chinese text) - implement `UniscribeWordSegmentationStrategy` (for other languages as default strategy)

add `WordSegmenter` class for word segmentation, integrating segmentation strategies

- replace `useUniscribe` with `useUniscribeForCharOffset` & `useWordSegmenterForWordOffset` for segmentation extensions - integrate `WordSegmenter` for calculating word offsets

make `DisplayModelTextInfo`'s flag aligned with its superclass

@LeonarddeR

- redesign 2 properties of 'OffsetTextInfo' as enums to make them more configurable, inspired by @LeonarddeR - override them in some subclasses to simulate specific behaviors

### Link to issue number:  ### Summary of the issue: ### Description of user facing changes: ### Description of developer facing changes: ### Description of development approach: ### Testing strategy: ### Known issues with pull request: ### Code Review Checklist:  - [ ] Documentation: - Change log entry - User Documentation - Developer / Technical Documentation - Context sensitive help for GUI changes - [ ] Testing: - Unit tests - System (end to end) tests - Manual testing - [ ] UX of all users considered: - Speech - Braille - Low Vision - Different web browsers - Localization in other languages / culture than English - [ ] API is compatible with existing add-ons. - [ ] Security precautions taken. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions@github.com> Co-authored-by: Sascha Cowley <16543535+SaschaCowley@users.noreply.github.com> Co-authored-by: Peng-An Chen <andy72039@gmail.com> Co-authored-by: Sean Budd <sean@nvaccess.org> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Michael Curran <mick@nvaccess.org> Co-authored-by: GitHub Actions <actions@github.com> Co-authored-by: Bill Dengler <codeofdusk@gmail.com> Co-authored-by: Cyrille Bougot <cyrille.bougot2@laposte.net> Co-authored-by: Boumtchack <147637328+Boumtchack@users.noreply.github.com> Co-authored-by: Kefas Lungu <84945041+kefaslungu@users.noreply.github.com> Co-authored-by: Jani Kinnunen <janikinnunen340@gmail.com> Co-authored-by: Akash Kakkar <6381747+akash07k@users.noreply.github.com> Co-authored-by: makhlwf <78276231+makhlwf@users.noreply.github.com> Co-authored-by: Quin Gillespie <trypsynth@gmail.com> Co-authored-by: Bram Duvigneau <bram@bramd.nl> Co-authored-by: Luke Davis <8139760+XLTechie@users.noreply.github.com> Co-authored-by: audioses <67341000+audioses@users.noreply.github.com> Co-authored-by: gexgd0419 <55008943+gexgd0419@users.noreply.github.com> Co-authored-by: Cyrille Bougot <cyrille.bougot@laposte.net> Co-authored-by: Joseph Lee <joseph.lee22590@gmail.com> Co-authored-by: Christopher Proß <cpross@mailbox.org> Co-authored-by: Ryan McCleary <remccleary@gmail.com> Co-authored-by: wencong <manchen_0528@outlook.com> Co-authored-by: Emil-18 <135248352+Emil-18@users.noreply.github.com> Co-authored-by: Danil <50794055+Danstiv@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Noelia Ruiz Martínez <nrm1977@gmail.com> Co-authored-by: Sean Budd <seanbudd123@gmail.com> Co-authored-by: James Teh <jamie@jantrid.net> Co-authored-by: Thiago Seus <thiago.seus@yahoo.com.br> Co-authored-by: WangFeng Huang <1398969445@qq.com> Co-authored-by: ethanl-11 <124083447+ethanl-11@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 30 out of 31 changed files in this pull request and generated 8 comments.

Comments suppressed due to low confidence (1)

source/braille.py:644

converter is overwritten when both Chinese word-segmentation spacing and Unicode normalization are active. In that case, brailleToRawPos / rawToBraillePos are only converted using the last converter, so offsets will be wrong (and cursor routing/selection mapping can break). Consider composing converters (apply both mappings), or apply normalization first and then segmentation while keeping a combined mapping back to the original raw text.

		converter: OffsetConverter | None = None
		textToTranslate = self.rawText
		textToTranslateTypeforms = self.rawTextTypeforms
		cursorPos = self.cursorPos
		if (
			config.conf["braille"]["translationTable"].startswith("zh")
			or config.conf["braille"]["translationTable"] == "auto"
			and brailleTables.getDefaultTableForCurLang(brailleTables.TableType.OUTPUT).startswith("zh")
		):
			from textUtils.wordSeg.wordSegUtils import WordSegWithSeparatorOffsetConverter  # noqa: F401

			converter = WordSegWithSeparatorOffsetConverter(textToTranslate)
			textToTranslate = converter.encoded
			if cursorPos is not None:
				cursorPos = converter.strToEncodedOffsets(cursorPos)
		if config.conf["braille"]["unicodeNormalization"] and not isUnicodeNormalized(textToTranslate):
			converter = UnicodeNormalizationOffsetConverter(textToTranslate)
			textToTranslate = converter.encoded
			if textToTranslateTypeforms is not None:
				# Typeforms must be adapted to represent normalized characters.
				textToTranslateTypeforms = [
					textToTranslateTypeforms[strOffset] for strOffset in converter.computedEncodedToStrOffsets
				]
			if cursorPos is not None:
				# Convert the cursor position to a normalized offset.
				cursorPos = converter.strToEncodedOffsets(cursorPos)
		self.brailleCells, brailleToRawPos, rawToBraillePos, self.brailleCursorPos = louisHelper.translate(
			[handler.table.fileName, "braille-patterns.cti"],
			textToTranslate,
			typeform=textToTranslateTypeforms,
			mode=mode,
			cursorPos=cursorPos,
		)

		if converter:
			# The received brailleToRawPos contains braille to normalized positions.
			# Process them to represent real raw positions by converting them from normalized ones.
			brailleToRawPos = [converter.encodedToStrOffsets(i) for i in brailleToRawPos]
			# The received rawToBraillePos contains normalized to braille positions.
			# Create a new list based on real raw positions.
			rawToBraillePos = [rawToBraillePos[i] for i in converter.computedStrToEncodedOffsets]
		self.brailleToRawPos = brailleToRawPos

+JiebaSingleton& JiebaSingleton::getInstance(const char* dictDir) {
+    // convert incoming C-string+length to std::string (handles dictDir == nullptr)
+    std::string dir = dictDir;
+
+    // ensure singleton is constructed exactly once
+    std::call_once(JiebaSingleton::initFlag, [&]() {
+        // allocate on heap, so we avoid copy/move and control lifetime
+        JiebaSingleton::instance = new JiebaSingleton(dir.c_str());
+        // optional: register deleter at exit
+        std::atexit([]() {
+            delete JiebaSingleton::instance;
+            JiebaSingleton::instance = nullptr;
+        });
+    });
+
+    // after call_once, instance must be non-null
+    return *JiebaSingleton::instance;
+}


+bool insertUserWord(const char* word, int freq, const char* tag = cppjieba::UNKNOWN_TAG) {
+	return JiebaSingleton::getInstance().InsertUserWord(string(word), freq, string(tag));
+}
+
+bool deleteUserWord(const char* word, const char* tag = cppjieba::UNKNOWN_TAG) {
+	return JiebaSingleton::getInstance().DeleteUserWord(string(word), string(tag));
+}
+
+bool find(const char* word) {
+	return JiebaSingleton::getInstance().Find(string(word));
+}


+		Compute a list of offsets so that:
+			encodedIndex = strIndex + relevantStrToEncodedOffsets[strIndex]
+
+		We build an explicit mapping from original string indices to encoded indices
+		by marking separator positions in the encoded string and then assigning
+		each non-separator encoded slot to the next original-character index.
+		The returned list contains the delta (encodedIndex - strIndex) for each
+		original index.


+			Thread(target=callable_to_call, args=args, kwargs=kwargs, daemon=True).start()
+		except Exception as e:
+			log.debug("Initializer %s.%s failed: %s", module_name, qualname, e)
+		return


+			case config.featureFlagEnums.WordNavigationUnitFlag.CHINESE:
+				return WordSegFlag.CHINESE
+			case _:
+				log.error(f"Unknown word segmentation standard, {self.__wordSegConf.calculated()!r}")


+	def segmentedText(self, sep: str = " ", newSepIndex: list[int] | None = None) -> str:
+		"""Segments the text using the word end indices."""
+
+		if len(self.wordEnds) <= 1:
+			return self.text
+
+		result = ""
+		for sepIndex in range(len(self.wordEnds) - 1):
+			preIndex = 0 if sepIndex == 0 else self.wordEnds[sepIndex - 1]
+			curIndex = self.wordEnds[sepIndex]
+			postIndex = self.wordEnds[sepIndex + 1]
+
+			# append the token before the potential separator position
+			result += self.text[preIndex:curIndex]
+
+			# quick checks: avoid adding duplicate separator if already present
+			if result.endswith(sep) or self.text[curIndex:postIndex].startswith(sep):
+				# separator already present at either side -> skip adding
+				continue
+
+			# Unicode categories for punctuation
+			PUNCTUATION_CATEGORIES: str = "pP"
+			# Determine whether any punctuation forbids a separator
+			noSep = (
+				unicodedata.category(self.text[curIndex - 1])[0] in PUNCTUATION_CATEGORIES
+				or unicodedata.category(self.text[curIndex])[0] in PUNCTUATION_CATEGORIES
+			)
+
+			if not noSep:
+				# If neither side forbids the separator, add it
+				result += sep
+				if newSepIndex is not None:
+					newSepIndex.append(len(result) - len(sep))
+		else:
+			# append the final trailing token after the loop
+			result += self.text[curIndex:postIndex]
+
+		return result
+
+	def getSegmentForOffset(self, offset: int) -> tuple[int, int] | None:
+		return self.getWordOffsetRange(offset)
+
+	def __init__(self, text, encoding=None):
+		super().__init__(text, encoding)
+		self.wordEnds = self._callCPPJieba()


+	def encodedToStrOffsets(
+		self,
+		encodedStart: int,
+		encodedEnd: int | None = None,
+		raiseOnError: bool = False,
+	) -> int | tuple[int]:
+		super().encodedToStrOffsets(encodedStart, encodedEnd, raiseOnError)
+		if encodedStart == 0:
+			resultStart = 0
+		else:
+			resultStart = self.computedEncodedToStrOffsets[encodedStart]
+		if encodedEnd is None:
+			return resultStart
+		elif encodedStart == encodedEnd:
+			return (resultStart, resultStart)
+		else:
+			resultEnd = self.computedEncodedToStrOffsets[encodedEnd]
+			return (resultStart, resultEnd)


seanbudd · 2026-05-04T02:24:02Z

@CrazySteve0605 - could you open a PR to address the copilot feedback?

### Link to issue number:  ### Summary of the issue: ### Description of user facing changes: ### Description of developer facing changes: ### Description of development approach: ### Testing strategy: ### Known issues with pull request: ### Code Review Checklist:  - [ ] Documentation: - Change log entry - User Documentation - Developer / Technical Documentation - Context sensitive help for GUI changes - [ ] Testing: - Unit tests - System (end to end) tests - Manual testing - [ ] UX of all users considered: - Speech - Braille - Low Vision - Different web browsers - Localization in other languages / culture than English - [ ] API is compatible with existing add-ons. - [ ] Security precautions taken. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

seanbudd · 2026-05-11T04:22:27Z

@cary-rowen / @CrazySteve0605 - do you mind doing a final re-review, and then I will review this PR?

cary-rowen · 2026-05-11T04:28:30Z

Of course, I'd be happy to do it.

cary-rowen · 2026-05-11T11:59:14Z

+
+cppjiebaLib = env.SharedLibrary(target="cppjieba", source=sourceFiles)
+
+if not os.path.exists(outDir.Dir("dicts").get_abspath()) or not os.listdir(outDir.Dir("dicts").get_abspath()): # ensure dicts installation happens only once and avoid a scons' warning


@CrazySteve0605 I think this install rule should not depend on whether the target directory currently exists or is empty. Other NVDA SCons scripts declare install targets unconditionally and let SCons decide whether they are up to date. With this guard, an incremental build where source/cppjieba/dicts already exists but a dict file is missing or stale will not declare the install target, so the build will not repair it.

…gmentation branch (#20106) This removes settings dialog category-change debounce code that was accidentally reintroduced while resolving conflicts. That code was previously reverted by #19846, so this branch should follow current master here and only keep the Document Navigation changes needed for Chinese word segmentation. This also removes an unrelated copyright-only change in browseMode.py.

seanbudd · 2026-05-18T06:23:57Z

@CrazySteve0605 @cary-rowen - is this ready for us to review yet?

cary-rowen · 2026-05-18T06:34:52Z

Hi @seanbudd

I believe we are ready now.

Thanks Sean

seanbudd

Hi @cary-rowen @CrazySteve0605 , this is a preliminary review of 20 comments.
I have not reviewed any of the larger files yet

This PR addresses selected review suggestions from Sean on PR #19166 and syncs the branch with the latest master. Completed: - [x] Move the braille offset converter helper out of the nested function. - [x] Move the word segmentation braille converter import out of the local scope. - [x] Move the word segmentation initialization debug log into initialize(). - [x] Remove unsupported RuntimeError-specific handling around wordSeg.initialize(). - [x] Log word segmentation initializer resolution and execution failures. - [x] Add missing return type annotations for wordSegFlag properties. - [x] Remove redundant int annotations from IntFlag members. - [x] Move test helper objects out of nested scopes and add type hints. - [x] Move braille test configuration setup/cleanup into setUp and tearDown. - [x] Update the help ID casing for the word segmentation setting. - [x] Add a user guide section for the word segmentation setting. - [x] Move the Chinese word segmentation change log entries to 2026.3. - [x] Sync with the latest nvaccess/master. Remaining: - [ ] Confirm whether the new config option should be exposed in the UI. --------- Co-authored-by: Michael Curran <mick@nvaccess.org> Co-authored-by: Leonard de Ruijter <3049216+LeonarddeR@users.noreply.github.com> Co-authored-by: Sean Budd <sean@nvaccess.org> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

seanbudd · 2026-05-20T03:49:44Z

I think I am going to open a new PR with a squashed history, as the diff with master is incorrect right now, and the history of this PR is quite unwieldy.

cary-rowen · 2026-05-20T04:08:51Z

Yes, absolutely. Thanks @seanbudd

seanbudd · 2026-05-20T05:58:23Z

Closing in favour of #20183

@LeonarddeR

commit e25f593 Author: wencong <manchen_0528@outlook.com> Date: Wed May 20 11:45:38 2026 +0800 Address review feedback for Chinese word segmentation (nvaccess#20178) This PR addresses selected review suggestions from Sean on PR nvaccess#19166 and syncs the branch with the latest master. Completed: - [x] Move the braille offset converter helper out of the nested function. - [x] Move the word segmentation braille converter import out of the local scope. - [x] Move the word segmentation initialization debug log into initialize(). - [x] Remove unsupported RuntimeError-specific handling around wordSeg.initialize(). - [x] Log word segmentation initializer resolution and execution failures. - [x] Add missing return type annotations for wordSegFlag properties. - [x] Remove redundant int annotations from IntFlag members. - [x] Move test helper objects out of nested scopes and add type hints. - [x] Move braille test configuration setup/cleanup into setUp and tearDown. - [x] Update the help ID casing for the word segmentation setting. - [x] Add a user guide section for the word segmentation setting. - [x] Move the Chinese word segmentation change log entries to 2026.3. - [x] Sync with the latest nvaccess/master. Remaining: - [ ] Confirm whether the new config option should be exposed in the UI. --------- Co-authored-by: Michael Curran <mick@nvaccess.org> Co-authored-by: Leonard de Ruijter <3049216+LeonarddeR@users.noreply.github.com> Co-authored-by: Sean Budd <sean@nvaccess.org> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> commit 887ad98 Merge: 9a6bc21 102a54e Author: Sean Budd <sean@nvaccess.org> Date: Mon May 18 16:23:14 2026 +1000 Merge branch 'master' into try-chineseWordSegmentation-staging commit 9a6bc21 Author: Wang Chong <306289287@qq.com> Date: Mon May 18 14:22:13 2026 +0800 Refactor Dicts Installation Logic in cppjieba's sconscript (nvaccess#20162) commit edf3429 Author: wencong <manchen_0528@outlook.com> Date: Tue May 12 08:26:06 2026 +0800 Remove reverted settings dialog debounce changes from Chinese word segmentation branch (nvaccess#20106) This removes settings dialog category-change debounce code that was accidentally reintroduced while resolving conflicts. That code was previously reverted by nvaccess#19846, so this branch should follow current master here and only keep the Document Navigation changes needed for Chinese word segmentation. This also removes an unrelated copyright-only change in browseMode.py. commit fab3060 Author: Wang Chong <306289287@qq.com> Date: Mon May 11 12:21:12 2026 +0800 Fixup merge master into Chinese Word Segmentation (nvaccess#20055)    - [ ] Documentation: - Change log entry - User Documentation - Developer / Technical Documentation - Context sensitive help for GUI changes - [ ] Testing: - Unit tests - System (end to end) tests - Manual testing - [ ] UX of all users considered: - Speech - Braille - Low Vision - Different web browsers - Localization in other languages / culture than English - [ ] API is compatible with existing add-ons. - [ ] Security precautions taken. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> commit 31c9585 Merge: 9edb45f d42d90c Author: Sean Budd <sean@nvaccess.org> Date: Mon May 4 09:25:02 2026 +1000 Merge master into chinese word segmentation (nvaccess#20041)    - [ ] Documentation: - Change log entry - User Documentation - Developer / Technical Documentation - Context sensitive help for GUI changes - [ ] Testing: - Unit tests - System (end to end) tests - Manual testing - [ ] UX of all users considered: - Speech - Braille - Low Vision - Different web browsers - Localization in other languages / culture than English - [ ] API is compatible with existing add-ons. - [ ] Security precautions taken. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions@github.com> Co-authored-by: Sascha Cowley <16543535+SaschaCowley@users.noreply.github.com> Co-authored-by: Peng-An Chen <andy72039@gmail.com> Co-authored-by: Sean Budd <sean@nvaccess.org> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Michael Curran <mick@nvaccess.org> Co-authored-by: GitHub Actions <actions@github.com> Co-authored-by: Bill Dengler <codeofdusk@gmail.com> Co-authored-by: Cyrille Bougot <cyrille.bougot2@laposte.net> Co-authored-by: Boumtchack <147637328+Boumtchack@users.noreply.github.com> Co-authored-by: Kefas Lungu <84945041+kefaslungu@users.noreply.github.com> Co-authored-by: Jani Kinnunen <janikinnunen340@gmail.com> Co-authored-by: Akash Kakkar <6381747+akash07k@users.noreply.github.com> Co-authored-by: makhlwf <78276231+makhlwf@users.noreply.github.com> Co-authored-by: Quin Gillespie <trypsynth@gmail.com> Co-authored-by: Bram Duvigneau <bram@bramd.nl> Co-authored-by: Luke Davis <8139760+XLTechie@users.noreply.github.com> Co-authored-by: audioses <67341000+audioses@users.noreply.github.com> Co-authored-by: gexgd0419 <55008943+gexgd0419@users.noreply.github.com> Co-authored-by: Cyrille Bougot <cyrille.bougot@laposte.net> Co-authored-by: Joseph Lee <joseph.lee22590@gmail.com> Co-authored-by: Christopher Proß <cpross@mailbox.org> Co-authored-by: Ryan McCleary <remccleary@gmail.com> Co-authored-by: wencong <manchen_0528@outlook.com> Co-authored-by: Emil-18 <135248352+Emil-18@users.noreply.github.com> Co-authored-by: Danil <50794055+Danstiv@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Noelia Ruiz Martínez <nrm1977@gmail.com> Co-authored-by: Sean Budd <seanbudd123@gmail.com> Co-authored-by: James Teh <jamie@jantrid.net> Co-authored-by: Thiago Seus <thiago.seus@yahoo.com.br> Co-authored-by: WangFeng Huang <1398969445@qq.com> Co-authored-by: ethanl-11 <124083447+ethanl-11@users.noreply.github.com> commit d42d90c Merge: 07d9b13 7f2d68f Author: cary-rowen <manchen_0528@outlook.com> Date: Fri May 1 22:21:05 2026 +0800 Merge latest master into chinese word segmentation branch commit 07d9b13 Merge: 9edb45f f7dc081 Author: cary-rowen <manchen_0528@outlook.com> Date: Mon Apr 20 12:03:50 2026 +0800 Merge latest master into try-chineseWordSegmentation-staging commit 9edb45f Merge: 096e985 4c9d616 Author: Sean Budd <sean@nvaccess.org> Date: Mon Apr 20 11:01:52 2026 +1000 Merge master into try-chineseWordSegmentation-staging (nvaccess#19747) commit 4c9d616 Author: cary-rowen <manchen_0528@outlook.com> Date: Thu Apr 9 22:19:56 2026 +0800 Fix textInfo word expansion test expectation commit 072b405 Author: cary-rowen <manchen_0528@outlook.com> Date: Thu Apr 9 22:00:30 2026 +0800 Fix word expansion without flowsTo commit 006277b Merge: 096e985 d8bf309 Author: Wang Chong <306289287@qq.com> Date: Fri Mar 6 00:35:58 2026 +0800 Merge branch 'master' commit 096e985 Author: Wang Chong <306289287@qq.com> Date: Mon Mar 2 12:02:27 2026 +0800 Fixup for Chinese Word Segmentation and Braille Output (nvaccess#19324) Summary of the issue: Some punctuations have extra separators (spaces) before or after them. Description of user facing changes: Braille output will be more accurate. commit 29d9f5a Merge: b50a0d5 9cafffb Author: Michael Curran <mick@nvaccess.org> Date: Thu Oct 30 12:12:25 2025 +1100 Merge pull request nvaccess#18865 from CrazySteve0605/brailleOutputForChinese blocked by nvaccess#18735, related nvaccess#1890 (comment), and [a part of OSPP 2025](https://summer-ospp.ac.cn/org/prodetail/25d3e0488?list=org&navpage=org) of NVDA. Braille output for Chinese become easier to read. The displayed braille and text are not aligned with each other in Braille Viewer, which shouldn't have an effect on reading from a braille displayer since only braille is transferred and the carets of raw text and output braille are aligned.  - [ ] Documentation: - Change log entry - User Documentation - Developer / Technical Documentation - Context sensitive help for GUI changes - [ ] Testing: - Unit tests - System (end to end) tests - Manual testing - [ ] UX of all users considered: - Speech - Braille - Low Vision - Different web browsers - Localization in other languages / culture than English - [ ] API is compatible with existing add-ons. - [ ] Security precautions taken. commit 9cafffb Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue Oct 28 22:26:13 2025 +0000 Pre-commit auto-fix commit 0d27c91 Merge: d2714a3 b50a0d5 Author: Michael Curran <mick@nvaccess.org> Date: Wed Oct 29 08:23:08 2025 +1000 Merge branch 'try-chineseWordSegmentation-staging' into brailleOutputForChinese commit b50a0d5 Merge: c86b760 db90fff Author: Michael Curran <mick@nvaccess.org> Date: Wed Oct 29 09:20:19 2025 +1100 Merge pull request nvaccess#18735 from CrazySteve0605/wordNavigationForChineseText blocked by nvaccess#18548, closes nvaccess#4075, related nvaccess#16237 and [a part of OSPP 2025](https://summer-ospp.ac.cn/org/prodetail/25d3e0488?list=org&navpage=org) of NVDA. Chinese text can be navigated by word via system caret or review cursor. - [ ] update `textUtils` - [x] add `WordSegment` module - [x] create `WordSegmentationStrategy' as an abstract base class to select segmentation strategy based on text content, following Strategy Pattern - [x] implement `ChineseWordSegmentationStrategy` (for Chinese text) - [x] implement `UniscribeWordSegmentationStrategy` (for other languages as default strategy) - [x] update `textUtils/__init__.py` - [x] add `WordSegmenter` class for word segmentation, integrating segmentation strategies - [x] update `textInfos/offsets.py` - [x] replace `useUniscribe` with `useUniscribeForCharOffset` & `useWordSegmenterForWordOffset` for segmentation extensions - [x] integrate `WordSegmenter` for calculating word offsets - [ ] update document Word segmentation functionality was integrated in `OffsetsTextInfo`. In `OffsetsTextInfo`, word segmentation is based on Uniscribe by default and Unicode as a fall-back. Subclasses of OffsetsTextInfo 1. `NVDAObjectTextInfo` 2. `InputCompositionTextInfo`: 3. `JABTextInfo` 4. `SimpleResultTextInfo` 5. `VirtualBufferTextInfo`: use self-hosted function to calculate offset and invoke iits superclass' `_getWordOffsets` 1. `DisplayModelTextInfo`: straightly disable 2. `EditTextInfo`: straightly use Uniscribe 3. `ScintillaTextInfo`: entirely use self-defined 'word', for Scintilla-based editors such Notepad++ source/NVDAObjects/window/scintilla.py 4. `VsTextEditPaneTextInfo`: use a special COM automation API, for Microsoft Visual Studio and Microsoft SQL Server Management Studio source/appModules/devenv.py 5. `TextFrameTextInfo`: use self-defined 'word', based on PowerPoint's COM object, for PowerPoint's text frame source/appModules/powerpnt.py 6. `LwrTextInfo`: based on pre-computed words during related progress, for structured text using 'LineWordResult' (e.g. Windows OCR) source/contentRecog/__init__.py Some architectures totally or priorly use native difination of a 'word', which softwares depend on may not be able to use the new functionallity. 1. `IA2TextTextInfo`: override and fall back source/NVDAObjects/IAccessible/\_\_init\_\_.py  - [ ] Documentation: - Change log entry - User Documentation - Developer / Technical Documentation - Context sensitive help for GUI changes - [ ] Testing: - Unit tests - System (end to end) tests - Manual testing - [ ] UX of all users considered: - Speech - Braille - Low Vision - Different web browsers - Localization in other languages / culture than English - [ ] API is compatible with existing add-ons. - [ ] Security precautions taken. commit db90fff Author: Wang Chong <306289287@qq.com> Date: Tue Oct 28 12:57:06 2025 +0800 remove duplicate importing lines commit d2714a3 Author: Wang Chong <306289287@qq.com> Date: Mon Oct 27 11:28:27 2025 +0800 fixup commit 042b778 Merge: b8ace76 c86b760 Author: Michael Curran <mick@nvaccess.org> Date: Mon Oct 27 11:47:18 2025 +1000 Merge branch 'try-chineseWordSegmentation-staging' into wordNavigationForChineseText commit c86b760 Merge: 4a4b1af 9e37e57 Author: Michael Curran <mick@nvaccess.org> Date: Mon Oct 27 11:45:35 2025 +1000 Merge branch 'master' into try-chineseWordSegmentation-staging commit b8ace76 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sat Oct 25 01:09:03 2025 +0000 Pre-commit auto-fix commit d32549f Author: Wang Chong <306289287@qq.com> Date: Sat Oct 25 09:03:22 2025 +0800 make word segmentation module reinitialized after settings are saved commit 2083095 Author: Wang Chong <306289287@qq.com> Date: Sat Oct 25 08:50:00 2025 +0800 fixup commit d55d077 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu Oct 9 21:19:50 2025 +0000 Pre-commit auto-fix commit 085ba2f Merge: 80b0472 4a4b1af Author: Michael Curran <mick@nvaccess.org> Date: Fri Oct 10 07:17:56 2025 +1000 Merge branch 'try-chineseWordSegmentation-staging' into wordNavigationForChineseText commit 4a4b1af Merge: 093a825 42dfcfe Author: Michael Curran <mick@nvaccess.org> Date: Fri Oct 10 07:13:22 2025 +1000 Merge branch 'master' into try-chineseWordSegmentation-staging commit 80b0472 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue Sep 30 05:12:16 2025 +0000 Pre-commit auto-fix commit f98b1b1 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue Sep 30 05:10:50 2025 +0000 Pre-commit auto-fix commit d1373b2 Author: Wang Chong <306289287@qq.com> Date: Tue Sep 30 13:06:54 2025 +0800 fixup commit 9ab3dba Merge: 251811e 0940a73 Author: Wang Chong <306289287@qq.com> Date: Tue Sep 30 13:06:22 2025 +0800 Merge branch 'wordNavigationForChineseText' into brailleOutputForChinese commit 0940a73 Author: Wang Chong <306289287@qq.com> Date: Tue Sep 30 12:59:46 2025 +0800 fixup commit c3a8562 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue Sep 30 03:25:34 2025 +0000 Pre-commit auto-fix commit 5e0e3fd Author: Wang Chong <306289287@qq.com> Date: Tue Sep 30 11:24:03 2025 +0800 simplify the logic for 'Auto' option in Word Segmentation Standard settings commit 552b42b Author: Wang Chong <306289287@qq.com> Date: Tue Sep 30 10:33:24 2025 +0800 fixup unittests commit 653e808 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon Sep 29 22:58:20 2025 +0000 Pre-commit auto-fix commit b40d709 Author: Wang Chong <306289287@qq.com> Date: Tue Sep 30 06:56:48 2025 +0800 revert `Initialize Word Segmenters for Unused Languages:` checkbox and fixup the initialization logic commit face4bd Merge: 6f586fd 093a825 Author: Michael Curran <mick@nvaccess.org> Date: Mon Sep 29 12:09:28 2025 +1000 Merge branch 'try-chineseWordSegmentation-staging' into wordNavigationForChineseText commit 093a825 Merge: 4bc116d b327e23 Author: Michael Curran <mick@nvaccess.org> Date: Mon Sep 29 12:07:17 2025 +1000 Merge branch 'integrateCppJieba' into try-chineseWordSegmentation-staging commit b327e23 Merge: 30e855f b236fe6 Author: Michael Curran <mick@nvaccess.org> Date: Mon Sep 29 08:38:17 2025 +1000 Merge branch 'try-chineseWordSegmentation-staging' into integrateCPPJieba commit 251811e Author: Wang Chong <306289287@qq.com> Date: Sun Sep 28 08:04:06 2025 +0800 update changelog commit 3b7bf5f Author: Wang Chong <306289287@qq.com> Date: Sun Sep 28 08:02:59 2025 +0800 correct and simplify the offset calculations commit 9304a39 Merge: 2d7c596 6f586fd Author: Wang Chong <306289287@qq.com> Date: Sun Sep 28 08:02:28 2025 +0800 Merge branch 'wordNavigationForChineseText' into brailleOutputForChinese commit 6f586fd Author: Wang Chong <306289287@qq.com> Date: Sun Sep 28 07:58:07 2025 +0800 update changelog commit b69d466 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 28 07:55:26 2025 +0800 fix up commit 9834b68 Author: Wang Chong <306289287@qq.com> Date: Sat Sep 27 09:16:37 2025 +0800 extract punctuation from `wordSegStrategy.py` to `wordSegUtils.py` commit 9479029 Author: Wang Chong <306289287@qq.com> Date: Sat Sep 27 09:15:51 2025 +0800 fixup commit f769457 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sat Sep 27 00:46:03 2025 +0000 Pre-commit auto-fix commit 2eec029 Author: Wang Chong <306289287@qq.com> Date: Sat Sep 27 08:40:57 2025 +0800 add unittest cases for `WordSegmenter` commit 43bfe03 Author: Wang Chong <306289287@qq.com> Date: Sat Sep 27 08:39:14 2025 +0800 make initialization of word segmenters conditional on language commit 111a24d Author: Wang Chong <306289287@qq.com> Date: Fri Sep 26 20:50:44 2025 +0800 update `wordSegSegmenter.py` to handle offsets at the end of the string commit 69617c4 Merge: 53b3870 fec70a9 Author: Wang Chong <306289287@qq.com> Date: Fri Sep 26 20:50:03 2025 +0800 Merge branch 'integrateCPPJieba' into wordNavigationForChineseText commit fec70a9 Merge: 30e855f b236fe6 Author: Wang Chong <306289287@qq.com> Date: Fri Sep 26 18:11:03 2025 +0800 Merge branch 'master' into integrateCPPJieba commit 53b3870 Author: Wang Chong <306289287@qq.com> Date: Thu Sep 25 23:09:55 2025 +0800 make `cppjieba` only available when NVDA's language is set to Chinese commit 250e700 Author: Wang Chong <306289287@qq.com> Date: Thu Sep 25 22:32:03 2025 +0800 update UI text for Uniscribe commit ccf07f9 Author: Wang Chong <306289287@qq.com> Date: Thu Sep 25 22:31:47 2025 +0800 correct method naming commit 38ec7ff Merge: dc23346 5562e70 Author: Wang Chong <306289287@qq.com> Date: Thu Sep 25 22:31:06 2025 +0800 Merge branch 'integrateCPPJieba' into wordNavigationForChineseText commit 5562e70 Merge: 30e855f 339af3e Author: Wang Chong <306289287@qq.com> Date: Thu Sep 25 22:29:07 2025 +0800 Merge branch 'master' into integrateCPPJieba commit 30e855f Merge: 2e730d6 dc22697 Author: Michael Curran <mick@nvaccess.org> Date: Mon Sep 22 18:20:36 2025 +1000 Merge branch 'try-chineseWordSegmentation-staging' into integrateCPPJieba commit dc23346 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 21 22:58:31 2025 +0800 Update source/core.py Co-authored-by: Cyrille Bougot <cyrille.bougot2@laposte.net> commit 9537999 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 21 22:52:10 2025 +0800 revert copyright header of `configSpec.py` commit 90660ba Author: Wang Chong <306289287@qq.com> Date: Sun Sep 21 22:24:18 2025 +0800 update `wordSegStrategy.py` commit 7ee08d0 Merge: a8955a3 2e730d6 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 21 22:22:54 2025 +0800 Merge branch 'integrateCPPJieba' into wordNavigationForChineseText commit a8955a3 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 21 22:19:19 2025 +0800 Revert "update module importing order and type annotations" This reverts commit 3bfbe59. commit 2e730d6 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 21 00:20:19 2025 +0800 Update .gitattributes Co-authored-by: Sean Budd <seanbudd123@gmail.com> commit 194a69e Author: Wang Chong <306289287@qq.com> Date: Sun Sep 21 00:10:40 2025 +0800 avoid using compilation time path commit c2cbb24 Author: Wang Chong <306289287@qq.com> Date: Sat Sep 20 23:53:56 2025 +0800 Revert "Update projectDocs/dev/createDevEnvironment.md" This reverts commit 06070c1. commit 0f507d5 Merge: b3e08ee 5a557d0 Author: Wang Chong <306289287@qq.com> Date: Sat Sep 20 23:53:19 2025 +0800 Merge branch 'master' into integrateCPPJieba commit bac3210 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sat Sep 13 01:33:24 2025 +0000 Pre-commit auto-fix commit 97eb6dd Author: Wang Chong <306289287@qq.com> Date: Sat Sep 13 09:19:40 2025 +0800 handle punctuation spacing commit 2b1d4b3 Merge: cf3e115 984b6eb Author: Wang Chong <306289287@qq.com> Date: Fri Sep 12 19:12:00 2025 +0800 Merge branch 'integrateCPPJieba' into wordNavigationForChineseText commit 984b6eb Merge: b3e08ee 3d74061 Author: Wang Chong <306289287@qq.com> Date: Fri Sep 12 19:09:24 2025 +0800 Merge branch 'master' into integrateCPPJieba commit cf3e115 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue Sep 9 16:49:47 2025 +0000 Pre-commit auto-fix commit 3a0badc Author: Wang Chong <306289287@qq.com> Date: Wed Sep 10 00:37:33 2025 +0800 update `wordSegStrategy.py` * add LRU caching commit f5087cc Merge: 3bfbe59 b3e08ee Author: Wang Chong <306289287@qq.com> Date: Wed Sep 10 00:24:14 2025 +0800 Merge branch 'integrateCPPJieba' into wordNavigationForChineseText commit b3e08ee Author: Wang Chong <306289287@qq.com> Date: Wed Sep 10 00:23:09 2025 +0800 update helper of `coojieba` commit bec5dc5 Merge: 09b1890 a210f97 Author: Wang Chong <306289287@qq.com> Date: Wed Sep 10 00:19:54 2025 +0800 Merge branch 'master' into integrateCPPJieba commit 09b1890 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 7 19:04:29 2025 +0800 fix building script commit 00796fe Author: Wang Chong <306289287@qq.com> Date: Sun Sep 7 18:39:59 2025 +0800 revert installing script commit 30120f8 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 7 18:38:36 2025 +0800 update building script commit 53158b6 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 7 18:17:50 2025 +0800 update changelog commit 3bfbe59 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 7 17:04:33 2025 +0800 update module importing order and type annotations commit b848e1b Author: Wang Chong <306289287@qq.com> Date: Sun Sep 7 16:28:01 2025 +0800 update `wordSegStrategy.py` commit abeb147 Merge: a1113d8 2955ca8 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 7 12:29:00 2025 +0800 Merge branch 'integrateCPPJieba' into wordNavigationForChineseText commit 2955ca8 Author: Wang Chong <306289287@qq.com> Date: Sun Sep 7 12:28:14 2025 +0800 simplify helper of `cppjieba` * turn to build-in `Word` structure * remove some items we don't use commit a9281f6 Author: Wang Chong <306289287@qq.com> Date: Sat Sep 6 18:36:20 2025 +0800 update .gitattributes for .hpp header files commit 49cc1fe Author: Wang Chong <306289287@qq.com> Date: Sat Sep 6 18:31:05 2025 +0800 update cppjieba to the latest commit commit fe118ee Merge: 1869ed0 e6557a7 Author: Wang Chong <306289287@qq.com> Date: Sat Sep 6 18:27:32 2025 +0800 Merge branch 'master' into integrateCPPJieba commit 2d7c596 Author: Wang Chong <306289287@qq.com> Date: Fri Sep 5 00:09:16 2025 +0800 add word separator to optimize braille output for Chinese text * add a subclass of `OffsetConverter` to handle the offset mapping for raw text and separated one * add logic to invoke it when Chinese translation tables are used commit a1113d8 Author: Wang Chong <306289287@qq.com> Date: Thu Sep 4 14:07:24 2025 +0800 add `segmentedText` method commit c1fb4b8 Merge: 9e6a2e1 1869ed0 Author: Wang Chong <306289287@qq.com> Date: Thu Sep 4 12:50:22 2025 +0800 Merge branch 'integrateCPPJieba' into wordNavigationForChineseText commit 1869ed0 Author: Wang Chong <306289287@qq.com> Date: Thu Sep 4 12:41:29 2025 +0800 simplify the initialization of cppjieba drop off initialization of the keyword extractor which we don't need commit 11827fb Merge: 3e495d2 952db62 Author: Wang Chong <306289287@qq.com> Date: Thu Sep 4 09:51:52 2025 +0800 Merge branch 'master' into integrateCPPJieba commit 9e6a2e1 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu Aug 28 11:17:31 2025 +0000 Pre-commit auto-fix commit 3b2d835 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 28 19:11:05 2025 +0800 resolve deprecation commit a4edc9e Merge: 97b6db7 3e495d2 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 28 19:05:26 2025 +0800 Merge branch 'integrateCPPJieba' into wordNavigationForChineseText commit 3e495d2 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 28 18:13:11 2025 +0800 add wrappers for user dict management commit a643391 Merge: c853b64 c30a787 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 28 18:10:09 2025 +0800 Merge branch 'master' into integrateCPPJieba commit 97b6db7 Author: Wang Chong <306289287@qq.com> Date: Sun Aug 24 23:47:59 2025 +0800 update for pyright checks commit 4a680ea Author: Wang Chong <306289287@qq.com> Date: Sun Aug 24 23:45:48 2025 +0800 make "Auto" the default option for word navigation commit 356c11c Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sat Aug 23 15:35:45 2025 +0000 Pre-commit auto-fix commit 3ba56f0 Author: Wang Chong <306289287@qq.com> Date: Sat Aug 23 23:25:47 2025 +0800 add configuration for word navigation users can find it in NVDA settings -> Document Navigation -> Word Segmentation Method commit eeb96aa Author: Wang Chong <306289287@qq.com> Date: Sat Aug 23 20:33:54 2025 +0800 use multithreading for cppjieba's initialization commit 38b4bea Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu Aug 21 07:00:36 2025 +0000 Pre-commit auto-fix commit 3f54d62 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 21 14:59:22 2025 +0800 add initialization logic to wordSeg module - add an decorator to easily add initializers - extract `cppjieba`'s initializer to fit the decorator commit 8244a76 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 21 11:04:44 2025 +0800 make wordSegment module to make file structure clearer - create `wordSeg` package - migrate wordSegment module into wordSeg package and rename to wordSeg commit d69e8b7 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 21 10:44:37 2025 +0800 add trailing commas in multi-line constructs commit 3c65868 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 21 10:36:16 2025 +0800 update log commit ddd48e8 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 21 10:33:28 2025 +0800 add type annotations commit 676fc42 Author: Wang Chong <306289287@qq.com> Date: Thu Aug 21 09:51:24 2025 +0800 add copyright header commit 0d40f0a Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed Aug 20 13:31:57 2025 +0000 Pre-commit auto-fix commit 407d4b2 Merge: 4adac07 adc22fb Author: Wang Chong <306289287@qq.com> Date: Wed Aug 20 21:27:13 2025 +0800 Merge branch 'integrateCPPJieba' into wordNavigationForChineseText commit 4adac07 Author: Wang Chong <306289287@qq.com> Date: Wed Aug 20 21:22:52 2025 +0800 update the word segmentation structure - redesign 2 properties of 'OffsetTextInfo' as enums to make them more configurable, inspired by @LeonarddeR - override them in some subclasses to simulate specific behaviors commit adc22fb Author: Wang Chong <306289287@qq.com> Date: Wed Aug 20 19:22:28 2025 +0800 add wrapper for word manager commit 19cad8a Merge: eba63ab 92f345e Author: Wang Chong <306289287@qq.com> Date: Wed Aug 20 17:35:03 2025 +0800 Merge branch 'master' into integrateCPPJieba commit f72d348 Author: Wang Chong <306289287@qq.com> Date: Mon Aug 18 15:06:19 2025 +0800 update type annotations commit 557f404 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon Aug 18 04:09:20 2025 +0000 Pre-commit auto-fix commit da64cd8 Author: Wang Chong <306289287@qq.com> Date: Mon Aug 18 12:02:49 2025 +0800 update `displayModel.py` make `DisplayModelTextInfo`'s flag aligned with its superclass commit 81f2040 Author: Wang Chong <306289287@qq.com> Date: Mon Aug 18 11:55:36 2025 +0800 update `textInfos/offsets.py` - replace `useUniscribe` with `useUniscribeForCharOffset` & `useWordSegmenterForWordOffset` for segmentation extensions - integrate `WordSegmenter` for calculating word offsets commit 9f62f04 Author: Wang Chong <306289287@qq.com> Date: Mon Aug 18 11:54:41 2025 +0800 update `textUtils/__init__.py` add `WordSegmenter` class for word segmentation, integrating segmentation strategies commit b0ac081 Author: Wang Chong <306289287@qq.com> Date: Mon Aug 18 11:38:48 2025 +0800 add `WordSegment` module - create `WordSegmentationStrategy' as an abstract base class to select segmentation strategy based on text content, following Strategy Pattern - implement `ChineseWordSegmentationStrategy` (for Chinese text) - implement `UniscribeWordSegmentationStrategy` (for other languages as default strategy) commit eba63ab Merge: 53dd3bb b9f19e6 Author: Wang Chong <306289287@qq.com> Date: Fri Aug 15 14:16:16 2025 +0800 Merge branch 'master' into integrateCPPJieba merge for Python updated commit 53dd3bb Merge: c853b64 b0241da Author: Wang Chong <306289287@qq.com> Date: Sat Aug 9 20:43:34 2025 +0800 Merge branch 'master' into integrateCPPJieba commit c853b64 Author: Wang Chong <306289287@qq.com> Date: Sat Aug 9 20:33:12 2025 +0800 Update include/readme.md Co-authored-by: Sean Budd <seanbudd123@gmail.com> commit c60c2da Author: Wang Chong <306289287@qq.com> Date: Sat Aug 9 20:30:58 2025 +0800 update copyright headers based on @seanbudd's suggestions commit 38a12dc Author: Wang Chong <306289287@qq.com> Date: Wed Aug 6 21:29:22 2025 +0800 Update building and setup script for cppjieba's dicts installation commit da662be Author: Wang Chong <306289287@qq.com> Date: Wed Aug 6 21:27:44 2025 +0800 Update .gitignore for cppjieba commit 0d92c08 Author: Wang Chong <306289287@qq.com> Date: Wed Aug 6 12:05:54 2025 +0800 Update GitHub action workflow to fetch cppjieba's submodule - Change 'submodules' in 'jobs - buildNVDA - Build NVDA - Checkout NVDA' from 'true' to 'recursive' to ensure cppjieba's submodule is fetched. - This will cause the submodule of sonic to be fetched as well, which seems currently unused. commit d4c3a92 Merge: f4cab8a 432364c Author: Wang Chong <306289287@qq.com> Date: Wed Aug 6 11:49:16 2025 +0800 Merge branch 'master' into integrateCPPJieba commit f4cab8a Merge: 7de7464 4ba948a Author: Wang Chong <306289287@qq.com> Date: Tue Aug 5 17:38:55 2025 +0800 Merge branch 'master' into integrateCPPJieba commit 7de7464 Author: Wang Chong <306289287@qq.com> Date: Tue Aug 5 12:52:39 2025 +0800 add JiebaSingleton wrapper and C API for NVDA segmentation - Introduce `JiebaSingleton` class in `cppjieba.hpp`/`cppjieba.cpp` with def file under nvdaHelper/cppjieba/' - Inherits from `cppjieba::Jieba` and exposes a thread-safe `getOffsets()` method - Implements Meyers’ singleton via `getInstance()` with a private constructor - Deletes copy constructor, copy assignment, move constructor, and move assignment to enforce single instance - Add C-style API in the same module: - `int initJieba()` to force singleton initialization - `int segmentOffsets(const char* text, int** charOffsets, int* outLen)` to perform segmentation and return character offsets - `void freeOffsets(int* ptr)` to release allocated offset buffer commit 1fbf05f Author: Wang Chong <306289287@qq.com> Date: Mon Aug 4 09:59:03 2025 +0800 add building script for cppjieba commit 3d4d9f1 Author: Wang Chong <306289287@qq.com> Date: Fri Aug 1 13:59:53 2025 +0800 Remove changes in sconscript for localLIb commit 2273a60 Author: Wang Chong <306289287@qq.com> Date: Fri Aug 1 13:06:08 2025 +0800 Update include/readme.md Co-authored-by: Sean Budd <seanbudd123@gmail.com> commit 06070c1 Author: Wang Chong <306289287@qq.com> Date: Fri Aug 1 13:04:22 2025 +0800 Update projectDocs/dev/createDevEnvironment.md Co-authored-by: Sean Budd <seanbudd123@gmail.com> commit ae58e9b Author: Wang Chong <306289287@qq.com> Date: Thu Jul 24 16:59:26 2025 +0800 Add comments for building script of cppjieba and its dependency commit fb4efef Author: Wang Chong <306289287@qq.com> Date: Thu Jul 24 16:51:49 2025 +0800 Update what's new commit 5cb5189 Author: Wang Chong <306289287@qq.com> Date: Wed Jul 23 17:30:56 2025 +0800 Introduce cppjieba as a submodule for Chinese word segmentation - Add `cppjieba` as a Git submodule under `third_party/cppjieba/` to provide robust Chinese word segmentation capabilities. - Update `.gitmodules` to point to the official `cppjieba` repository and configure it to track the `master` branch. - Update 'sconscript' to include the paths of 'cppjieba' and its dependency 'limonp' - Modify `copying.txt` to include the `cppjieba` license (MIT) alongside the project’s existing license, ensuring proper attribution and compliance. - Update documents

CrazySteve0605 and others added 30 commits July 23, 2025 17:30

Update what's new

fb4efef

Add comments for building script of cppjieba and its dependency

ae58e9b

Update projectDocs/dev/createDevEnvironment.md

06070c1

Co-authored-by: Sean Budd <seanbudd123@gmail.com>

Update include/readme.md

2273a60

Co-authored-by: Sean Budd <seanbudd123@gmail.com>

Remove changes in sconscript for localLIb

3d4d9f1

add building script for cppjieba

1fbf05f

Merge branch 'master' into integrateCPPJieba

f4cab8a

Merge branch 'master' into integrateCPPJieba

d4c3a92

Update .gitignore for cppjieba

da662be

Update building and setup script for cppjieba's dicts installation

38a12dc

update copyright headers based on @seanbudd's suggestions

c60c2da

Update include/readme.md

c853b64

Co-authored-by: Sean Budd <seanbudd123@gmail.com>

Merge branch 'master' into integrateCPPJieba

53dd3bb

Merge branch 'master' into integrateCPPJieba

eba63ab

merge for Python updated

update textUtils/__init__.py

9f62f04

add `WordSegmenter` class for word segmentation, integrating segmentation strategies

update textInfos/offsets.py

81f2040

- replace `useUniscribe` with `useUniscribeForCharOffset` & `useWordSegmenterForWordOffset` for segmentation extensions - integrate `WordSegmenter` for calculating word offsets

update displayModel.py

da64cd8

make `DisplayModelTextInfo`'s flag aligned with its superclass

Pre-commit auto-fix

557f404

update type annotations

f72d348

Merge branch 'master' into integrateCPPJieba

19cad8a

add wrapper for word manager

adc22fb

update the word segmentation structure

4adac07

- redesign 2 properties of 'OffsetTextInfo' as enums to make them more configurable, inspired by @LeonarddeR - override them in some subclasses to simulate specific behaviors

Merge branch 'integrateCPPJieba' into wordNavigationForChineseText

407d4b2

Pre-commit auto-fix

0d40f0a

add copyright header

676fc42

add type annotations

ddd48e8

cary-rowen and others added 2 commits May 1, 2026 22:21

Merge latest master into chinese word segmentation branch

d42d90c

seanbudd requested a review from Copilot May 3, 2026 23:26

Copilot started reviewing on behalf of seanbudd May 3, 2026 23:27 View session

github-code-quality Bot found potential problems May 3, 2026

View reviewed changes

Comment thread source/textUtils/__init__.py Fixed

Comment thread source/textUtils/wordSeg/wordSegStrategy.py Fixed

Copilot AI reviewed May 3, 2026

View reviewed changes

seanbudd removed the request for review from SaschaCowley May 4, 2026 02:23

cary-rowen mentioned this pull request May 5, 2026

Fixup merge master into Chinese Word Segmentation #20055

Merged

5 tasks

cary-rowen reviewed May 11, 2026

View reviewed changes

cary-rowen and others added 3 commits May 12, 2026 10:26

Refactor Dicts Installation Logic in cppjieba's sconscript (#20162)

9a6bc21

Merge branch 'master' into try-chineseWordSegmentation-staging

887ad98

seanbudd marked this pull request as ready for review May 18, 2026 06:43

seanbudd commented May 19, 2026

View reviewed changes

cary-rowen mentioned this pull request May 19, 2026

Address review feedback for Chinese word segmentation #20178

Merged

14 tasks

seanbudd requested a review from a team as a code owner May 20, 2026 03:45

seanbudd requested a review from Qchristensen May 20, 2026 03:45

seanbudd mentioned this pull request May 20, 2026

Add Chinese Word Segmentation #20183

Open

seanbudd closed this May 20, 2026


		cppjiebaLib = env.SharedLibrary(target="cppjieba", source=sourceFiles)

		if not os.path.exists(outDir.Dir("dicts").get_abspath()) or not os.listdir(outDir.Dir("dicts").get_abspath()): # ensure dicts installation happens only once and avoid a scons' warning

Uh oh!

Conversation

seanbudd commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

seanbudd commented May 4, 2026

Uh oh!

seanbudd commented May 11, 2026

Uh oh!

cary-rowen commented May 11, 2026

Uh oh!

cary-rowen May 11, 2026

Choose a reason for hiding this comment

Uh oh!

seanbudd commented May 18, 2026

Uh oh!

cary-rowen commented May 18, 2026

Uh oh!

seanbudd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

seanbudd commented May 20, 2026

Uh oh!

cary-rowen commented May 20, 2026

Uh oh!

seanbudd commented May 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

seanbudd commented Nov 5, 2025 •

edited

Loading