Fix WaveClient sending Bearer token to public S3 URLs by pditommaso · Pull Request #6672 · nextflow-io/nextflow

pditommaso · 2025-12-18T09:04:05Z

Summary

Fix WaveClient sending Bearer token to external URLs (e.g., public S3 buckets) when fetching container configs
Add dedicated plainHttpClient() without authentication for external resource requests
S3 and similar services reject requests with unsupported Authorization headers

Changes

Added plainHttpClient() factory method with @Memoized annotation for lazy initialization
Updated fetchContainerConfig() to use the plain client instead of the authenticated client
Added Javadoc to explain the rationale for each HTTP client
Added unit test that verifies container config fetch works without Bearer token

Fixes #6671

Test plan

Unit test added that simulates S3 behavior (rejects Bearer tokens)
All existing nf-wave tests pass

🤖 Generated with Claude Code

When fetching container configs from external URLs (e.g., public S3 buckets), the WaveClient was sending the Tower/Platform Bearer token with the request. AWS S3 does not support Bearer authentication and returns a 400 error: "Unsupported Authorization Type". This fix adds a dedicated HTTP client (plainHttpClient) without Bearer token authentication for fetching external container configs, while the main client continues to use authentication for Wave/Tower API calls. Fixes #6671 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>

netlify · 2025-12-18T09:04:11Z

✅ Deploy Preview for nextflow-docs-staging canceled.

Name	Link
🔨 Latest commit	`1103109`
🔍 Latest deploy log	https://app.netlify.com/projects/nextflow-docs-staging/deploys/6943c388a161f40008a8912b

plugins/nf-wave/src/main/io/seqera/wave/plugin/WaveClient.groovy

pditommaso · 2025-12-18T11:51:34Z

@jorgee Good question. After reviewing both codebases, here's what I found:

The Bearer token IS used by Wave backend, but for a different purpose than fetching the container config URL.

How the token flow works:

Nextflow WaveClient sends the Tower access token to Wave API endpoints (like /v1alpha2/container)
Wave backend validates it against Tower/Platform - it forwards the token to the Tower /user-info endpoint to verify it's valid and retrieve the user identity (UserServiceImpl.getUserByAccessToken)
What the token enables: Once validated, the PlatformId (user info, workspace, token) is used for:
- Fetching registry credentials from Tower
- Accessing workflow launch info
- Rate limiting and metrics tied to user identity

Why `plainHttpClient()` is correct here:

The containerConfig URL returned by Wave points to an external resource (like a public S3 bucket), not to the Wave API itself. This URL doesn't need/accept the Tower Bearer token - S3 uses different auth mechanisms (signed URLs, IAM, etc.).

The Wave API request that returns this URL already used the authenticated client. The subsequent fetch of the config content from the external URL should be unauthenticated.

So to answer your question: No, the token is not required for fetching the container config content from external URLs. The token is only needed for Wave API calls, which still use httpClient() with authentication.

jorgee · 2025-12-18T12:31:29Z

plugins/nf-wave/src/main/io/seqera/wave/plugin/WaveClient.groovy

+    protected HxClient plainHttpClient() {
+        return HxClient.newBuilder()
+                .httpClient(newHttpClient0())
+                .retryConfig(config.retryOpts())


Suggested change

.retryConfig(config.retryOpts())

.retryConfig(config.retryOpts())

.followRedirects(HttpClient.Redirect.NORMAL)

Quite sure this is the default

jorgee · 2025-12-18T12:32:17Z

plugins/nf-wave/src/main/io/seqera/wave/plugin/WaveClient.groovy

+        final resp = plainHttpClient().sendAsString(req)
        final code = resp.statusCode()
        final body = resp.body()
        if( code>=200 && code<400 ) {


Suggested change

if( code>=200 && code<400 ) {

if( code>=200 && code<300 ) {

jorgee

I was testing the provided URL and it was answering with 301 it was considered a valid status code and then there was a failure because it was expecting a JSON but S3 respond with an XML

Caused by:
  Expected BEGIN_OBJECT but was STRING at line 1 column 1 path $
  See https://github.com/google/gson/blob/main/Troubleshooting.md#unexpected-json-structure

As it is not doing a Wave client request, I suggest to do the following changes:
1 - configure the plainHttpClient with followRedirection(NORMAL)
2 - manage redirection status code as error to avoid the JSON error problem.

Not sure for (1), but at least we should do (2)

pditommaso · 2025-12-18T13:00:38Z

The redirect is handled by the http client by default. Just tried my branch and it's ok

jorgee · 2025-12-18T14:10:42Z

Maybe I get confused, but HxClient is not HxClient a wrapper of HttpClient. In that libray, the default for followredirects is NEVER https://docs.oracle.com/en/java/javase/11/docs/api/java.net.http/java/net/http/HttpClient.html#followRedirects()

But, either NORMAL is the default or not, if the redirect has been followed, we shouldn't receive 3XX. So I think 3XX should be considered an error. This is the reason why I was getting the JSON parsing error when trying with "fusion.containerConfigUrl = 'https://s3.eu-west-2.amazonaws.com/bucket/path/fusion-amd64.json'

pditommaso · 2025-12-18T14:14:10Z

So why (integration) tests pass ?

jorgee · 2025-12-18T15:41:15Z

Ok, I see what is happening. The URL in the issue is fake. As it is a regional S3, the AWS is returning 301 to redirect to global endpoint. It is what produces the error in my case. With a correct URL, it is working.

pditommaso · 2025-12-19T11:34:04Z

I assume this is ok in the correct form then

jorgee · 2025-12-19T11:36:51Z

Yes

pditommaso requested a review from jorgee December 18, 2025 09:04

jorgee reviewed Dec 18, 2025

View reviewed changes

plugins/nf-wave/src/main/io/seqera/wave/plugin/WaveClient.groovy Show resolved Hide resolved

jorgee reviewed Dec 18, 2025

View reviewed changes

jorgee requested changes Dec 18, 2025

View reviewed changes

jorgee approved these changes Dec 19, 2025

View reviewed changes

pditommaso merged commit ffaef0b into master Dec 19, 2025
36 checks passed

pditommaso deleted the fix/wave-bearer-token-s3 branch December 19, 2025 11:37

bentsherman added the software/wave label Dec 19, 2025

	.retryConfig(config.retryOpts())
	.retryConfig(config.retryOpts())
	.followRedirects(HttpClient.Redirect.NORMAL)

Conversation

pditommaso commented Dec 18, 2025

Summary

Changes

Test plan

Uh oh!

netlify bot commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for nextflow-docs-staging canceled.

Uh oh!

Uh oh!

pditommaso commented Dec 18, 2025

How the token flow works:

Why plainHttpClient() is correct here:

Uh oh!

jorgee Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

pditommaso Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

jorgee Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

jorgee left a comment

Choose a reason for hiding this comment

Uh oh!

pditommaso commented Dec 18, 2025

Uh oh!

jorgee commented Dec 18, 2025

Uh oh!

pditommaso commented Dec 18, 2025

Uh oh!

jorgee commented Dec 18, 2025

Uh oh!

pditommaso commented Dec 19, 2025

Uh oh!

jorgee commented Dec 19, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

netlify bot commented Dec 18, 2025 •

edited

Loading

Why `plainHttpClient()` is correct here: