Skip to content

chore(cli): replace threads with tinypool#2420

Merged
andrii-bodnar merged 2 commits intolingui:nextfrom
yslpn:chore/replace-threads-with-tinypool
Jan 29, 2026
Merged

chore(cli): replace threads with tinypool#2420
andrii-bodnar merged 2 commits intolingui:nextfrom
yslpn:chore/replace-threads-with-tinypool

Conversation

@yslpn
Copy link
Contributor

@yslpn yslpn commented Jan 28, 2026

Description

Replace threads library with tinypool for worker pool management in CLI package.

Motivation:

  • tinypool is a more actively maintained and lightweight alternative
  • Eliminates the need for a custom yarn patch for threads
  • New Promise-based usage with Promise.all instead of pool.queue/pool.completed

Key changes:

  • Created typedPool.ts - generic typed wrapper around Tinypool
  • Created workerPools.ts - factory functions for all worker pools (extract, extract-experimental, compile)
  • Updated all worker wrappers to export default function instead of using expose() from threads
  • Simplified worker invocation: pool.run([...args]) instead of pool.queue(worker => worker(...))
  • Removed yarn patch for threads library

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Examples update

Fixes # (issue)

Checklist

  • I have read the CONTRIBUTING and CODE_OF_CONDUCT docs
  • I have added tests that prove my fix is effective or that my feature works
  • I have added the necessary documentation (if appropriate)

@vercel
Copy link

vercel bot commented Jan 28, 2026

@yslpn is attempting to deploy a commit to the Crowdin Team on Vercel.

A member of the Team first needs to authorize it.

@codecov
Copy link

codecov bot commented Jan 28, 2026

Codecov Report

❌ Patch coverage is 68.62745% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.00%. Comparing base (dd43fb0) to head (50c4ee3).
⚠️ Report is 291 commits behind head on next.

Files with missing lines Patch % Lines
packages/cli/src/workers/compileWorker.ts 0.00% 9 Missing and 2 partials ⚠️
packages/cli/src/api/catalog/extractFromFiles.ts 85.71% 1 Missing ⚠️
packages/cli/src/api/typedPool.ts 91.66% 1 Missing ⚠️
...-experimental/workers/extractWorkerWrapper.prod.ts 0.00% 1 Missing ⚠️
...kages/cli/src/workers/compileWorkerWrapper.prod.ts 0.00% 1 Missing ⚠️
...kages/cli/src/workers/extractWorkerWrapper.prod.ts 0.00% 1 Missing ⚠️

❌ Your patch status has failed because the patch coverage (68.62%) is below the target coverage (70.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@             Coverage Diff             @@
##             next    #2420       +/-   ##
===========================================
+ Coverage   76.66%   88.00%   +11.33%     
===========================================
  Files          81      113       +32     
  Lines        2083     3301     +1218     
  Branches      532      965      +433     
===========================================
+ Hits         1597     2905     +1308     
+ Misses        375      356       -19     
+ Partials      111       40       -71     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@timofei-iatsenko timofei-iatsenko self-requested a review January 28, 2026 11:39
@timofei-iatsenko
Copy link
Collaborator

timofei-iatsenko commented Jan 28, 2026

I pushed some slight changes in types, just wanted to have call to pool.run(...args) look more like a method execution with argument instead passing args as array/tuple.

Also i'm wondering about:

minThreads: poolSize,
maxThreads: poolSize,

Probably we should set only a maxThreads, so if there would be only 2 tasks, and poolSize = 5, it will not spawn 3 extra workers.

It's better to consult actually what their docs saying about best practicies and maybe check vitest code how they works with those parameters.

@yslpn
Copy link
Contributor Author

yslpn commented Jan 28, 2026

I pushed some slight changes in types, just wanted to have call to pool.run(...args) look more like a method execution with argument instead passing args as array/tuple.

Thanks

Also i'm wondering about:

minThreads: poolSize,
maxThreads: poolSize,

Probably we should set only a maxThreads, so if there would be only 2 tasks, and poolSize = 5, it will not spawn 3 extra workers.

It's better to consult actually what their docs saying about best practicies and maybe check vitest code how they works with those parameters.

I found that vitest was removed from vitest and they wrote their own implementation https://vitest.dev/guide/migration.html#pool-rework

I looked at how facebook/docusaurus uses it; they also use the same values ​​for maxThreads and minThreads. They calculate the ideal number for themselves based on pages:
https://github.com/facebook/docusaurus/blob/df259ddb269153e652f3731ab5cd52d12244b45f/packages/docusaurus/src/ssg/ssgExecutor.ts#L126

50 pages → 1 thread
150 pages → 2 threads
and so on.

In our case, we have two options:
ref: https://lingui.dev/ref/cli#extract-workers

  1. We calculate the number of threads ourselves, before starting, using --workers
    To avoid breaking the cli API and creating new flags, we can leave the same number of threads specified with --workers, and we guarantee this number of threads.
minThreads: poolSize,
maxThreads: poolSize,
  1. Or we calculate automatically.
    Use our current logic for calculating the number of threads and pass this value to maxThreads.
minThreads: 0,
maxThreads: poolSize,

Plus, we need to adjust all the messages, as there will be two cases:

Use worker pool of size ${poolSize} and Use worker pool with up to ${poolSize} workers

Is that okay?

@timofei-iatsenko
Copy link
Collaborator

It seems from the docs (from upstream piscina):

minThreads: (number) Sets the minimum number of threads that are always running for this thread pool. The default is the number provided by os.availableParallelism.
maxThreads: (number) Sets the maximum number of threads that are running for this thread pool. The default is the number provided by os.availableParallelism * 1.5.

That minThreads is not based on the number of tasks. So i'm confused now how that works. Reading this docs seems your's

minThreads: poolSize,
maxThreads: poolSize,

totally makes sense

@timofei-iatsenko
Copy link
Collaborator

@andrii-bodnar regarding coverage - disregard this report, since workers are not instrumented by covergae because they run in kinda separate process. The functionality is covered in the original implementation - this change doesn't add anything to logic we already have.

@timofei-iatsenko timofei-iatsenko changed the title chore: replace threads with tinypool chore(cli): replace threads with tinypool Jan 29, 2026
@andrii-bodnar andrii-bodnar merged commit 8bca419 into lingui:next Jan 29, 2026
8 of 10 checks passed
@andrii-bodnar andrii-bodnar added this to the v6 milestone Jan 29, 2026
@timofei-iatsenko timofei-iatsenko mentioned this pull request Feb 4, 2026
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants