Skip to content

New executor for Fujitsu Technical Computing Suite (TCS)#5928

Merged
pditommaso merged 26 commits intonextflow-io:masterfrom
exthnet:tcs
Sep 2, 2025
Merged

New executor for Fujitsu Technical Computing Suite (TCS)#5928
pditommaso merged 26 commits intonextflow-io:masterfrom
exthnet:tcs

Conversation

@exthnet
Copy link
Contributor

@exthnet exthnet commented Mar 27, 2025

This is a new Executor for Fujitsu Technical Computing Suite (TCS).

Some supercomputers build by Fujitsu are operated (mainly in Japan).
Most of they use Technical Computing Suite (TCS) job management system, they are not supported by current Nextflow.

Therefore, I wrote new Executor citing existing executors.
We tested simple jobs and confirmed the correct behaviors on on Fugaku (Riken), Flow (Naogya Univ.), and Genkai (Kyushu Univ.) supercomputers.

Note that I am not familiar with all the features of Nextflow, so there may be some lack of implementation.

@netlify
Copy link

netlify bot commented Mar 27, 2025 •
edited
Loading

✅ Deploy Preview for nextflow-docs-staging ready!

Name Link
🔨 Latest commit 519d02b
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-docs-staging/deploys/68b6f9b7e800340008877ac5
😎 Deploy Preview https://deploy-preview-5928--nextflow-docs-staging.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

pditommaso
pditommaso requested changes Mar 27, 2025
View reviewed changes
Copy link
Member

@pditommaso pditommaso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for this contribution. Please consider adding some unit tests. You can reference to other executors for some examples. Also please sign your commits as example in the contributing docs

@bentsherman bentsherman linked an issue Mar 28, 2025 that may be closed by this pull request
New feature: TCS Executor (for Supercomputers of Fujitsu) #5916
Closed
@exthnet exthnet requested a review from a team as a code owner April 3, 2025 04:49
@exthnet
Copy link
Contributor Author

exthnet commented Apr 3, 2025 •
edited
Loading

Thanks a lot for this contribution. Please consider adding some unit tests. You can reference to other executors for some examples. Also please sign your commits as example in the contributing docs

Thanks for the advice. I'll try to add unit tests and add sign.

@pditommaso
Copy link
Member

pditommaso commented May 24, 2025

This PR looks stalled. Feel free to reopen if you are keen to contribute to this feature.

@pditommaso pditommaso closed this May 24, 2025
@exthnet
Copy link
Contributor Author

exthnet commented Jun 10, 2025

I have been too busy to take the time.

I am going to resume my work. I have already successfully run a simple workflow job, but I am not sure what additional tests I should run.
I know that there are many test sets in the test directories (tests, tests-v1, test-e2e), but I can't find any explanations.
Could you please advise?

@pditommaso pditommaso reopened this Jun 10, 2025
@pditommaso
Copy link
Member

pditommaso commented Jun 10, 2025

Ideally it should be added a TcsExecutorTest class validating the executor logic. You can following the the SlurmExecutorTest as as template

@exthnet
Copy link
Contributor Author

exthnet commented Jun 11, 2025

Ideally it should be added a TcsExecutorTest class validating the executor logic. You can following the the SlurmExecutorTest as as template

Thanks, I'll try it.

pditommaso and others added 13 commits July 3, 2025 10:47
@pditommaso @exthnet
Revert "Fix JWT refresh and use TowerClient to handle Fusion token re…
470a055 
…quests nextflow-io#6207" [ci fast]

This reverts commit 9883b4d.

Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Signed-off-by: Satoshi OHSHIMA <ohshima@cc.kyushu-u.ac.jp>
@adamrtalbot @exthnet
Minor Azure Batch disk slot calculation demoted to debug (nextflow-io…
445813b 
…#6234) [ci skip]

Signed-off-by: adamrtalbot <12817534+adamrtalbot@users.noreply.github.com>
Signed-off-by: Satoshi OHSHIMA <ohshima@cc.kyushu-u.ac.jp>
@pditommaso @exthnet
Fix JWT token refresh to Fusion validation request (nextflow-io#6231)…
4fd5fcf 
… [ci fast]

Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Signed-off-by: Satoshi OHSHIMA <ohshima@cc.kyushu-u.ac.jp>
@ewels @exthnet
Fix docs rendering on config page [ci fast] (nextflow-io#6236)
5e42bbb 
Minor formatting bug when rendering the AWS config reference.

Signed-off-by: Phil Ewels <phil.ewels@seqera.io>
Signed-off-by: Satoshi OHSHIMA <ohshima@cc.kyushu-u.ac.jp>
@bentsherman @exthnet
Document secrets config variable (nextflow-io#6230) [ci fast]
c5384b9 
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Signed-off-by: Satoshi OHSHIMA <ohshima@cc.kyushu-u.ac.jp>
@pditommaso @exthnet
Remove deprecated nexus-staging plugin [ci fast]
1e89771 
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Signed-off-by: Satoshi OHSHIMA <ohshima@cc.kyushu-u.ac.jp>
@bentsherman @exthnet
Fix false warning in config schema validator (nextflow-io#6240)
175a617 
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
Signed-off-by: Satoshi OHSHIMA <ohshima@cc.kyushu-u.ac.jp>
@pditommaso @exthnet
Remove deprecated GsonHelper class
586c8d0 
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Signed-off-by: Satoshi OHSHIMA <ohshima@cc.kyushu-u.ac.jp>
@exthnet
add new executor for Fujitsu Technical Computing Suite (TCS)
b9c514e 
Signed-off-by: Satoshi OHSHIMA <ohshima@cc.kyushu-u.ac.jp>
@exthnet
update comment for TCS Executor
7d8e394 
Signed-off-by: Satoshi Ohshima <ohshima@cc.kyushu-u.ac.jp>>
Signed-off-by: Satoshi OHSHIMA <ohshima@cc.kyushu-u.ac.jp>
@exthnet
Update TcsExecutor
d1b3e34 
Add TcsExecutorTest

Signed-off-by: Satoshi OHSHIMA <ohshima@cc.kyushu-u.ac.jp>
@exthnet
Update TcsExecutor
a4edb55 
Add TcsExecutorTest

Signed-off-by: Satoshi OHSHIMA <ohshima@cc.kyushu-u.ac.jp>
@exthnet
Update test cases for TcsExecutor.
cd5935c 
Signed-off-by: Satoshi OHSHIMA <ohshima@cc.kyushu-u.ac.jp>
@exthnet exthnet requested a review from a team as a code owner July 3, 2025 02:24
christopher-hakkaart
christopher-hakkaart requested changes Jul 8, 2025
View reviewed changes
Copy link
Collaborator

@christopher-hakkaart christopher-hakkaart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've requested some changes in the documentation. It's mostly formatting and changing the voice and making the example more generic.

@bentsherman bentsherman mentioned this pull request Jul 10, 2025
Closed
pditommaso and others added 4 commits July 15, 2025 15:38
@pditommaso @christopher-hakkaart
Update docs/executor.md [ci skip]
1ace909 
Co-authored-by: Chris Hakkaart <chris.hakkaart@seqera.io>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
@pditommaso @christopher-hakkaart
Update docs/executor.md [ci skip]
e20a4a5 
Co-authored-by: Chris Hakkaart <chris.hakkaart@seqera.io>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
@pditommaso @christopher-hakkaart
Update docs/executor.md [ci skip]
e011736 
Co-authored-by: Chris Hakkaart <chris.hakkaart@seqera.io>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
@bentsherman
cleanup
eb6a226 
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
pditommaso
pditommaso requested changes Jul 15, 2025
View reviewed changes
bentsherman and others added 4 commits July 15, 2025 08:57
@bentsherman
cleanup
c8dd777 
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
@exthnet
Just add a brief description.
0a056d8 
Signed-off-by: Satoshi OHSHIMA <ohshima@cc.kyushu-u.ac.jp>
@exthnet
Merge branch 'master' into tcs
0be0dbc
@exthnet
Merge branch 'master' into tcs
6255f64
@manuel-g-castro manuel-g-castro mentioned this pull request Sep 1, 2025
Closed
12 tasks
@pditommaso
Merge branch 'master' into tcs
515d9e6
@pditommaso
Minor cleanup [ci skip]
519d02b 
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
@pditommaso pditommaso merged commit dd87b2f into nextflow-io:master Sep 2, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pditommaso pditommaso pditommaso approved these changes

@bentsherman bentsherman bentsherman approved these changes

@christopher-hakkaart christopher-hakkaart Awaiting requested review from christopher-hakkaart

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

New feature: TCS Executor (for Supercomputers of Fujitsu)

6 participants

@exthnet @pditommaso @bentsherman @christopher-hakkaart @adamrtalbot @ewels