Skip to content

Migrate SraExplorer to use HxClient for HTTP operations#6467

Merged
pditommaso merged 2 commits intomasterfrom
migrate-sra-explorer-to-hxclient
Oct 10, 2025
Merged

Migrate SraExplorer to use HxClient for HTTP operations#6467
pditommaso merged 2 commits intomasterfrom
migrate-sra-explorer-to-hxclient

Conversation

@pditommaso
Copy link
Member

Summary

Replace legacy URL.getText() HTTP client with HxClient in the SRA datasource component for better retry handling and consistency with other HTTP clients in the codebase.

Changes

SraRetryConfig

  • Implement Retryable.Config interface for compatibility with HxClient
  • Add multiplier field (default: 2.0) for exponential backoff
  • Implement required interface methods: getMaxAttempts(), getJitter(), getMultiplier()
  • Add getDelay() and getMaxDelay() for HxClient integration

SraExplorer

  • Replace new URI(url).toURL().getText() with HxClient.sendAsString()
  • Add getHttpClient() method that configures HxClient with:
    • HTTP/1.1 protocol
    • Normal redirect following
    • Retry configuration from SraRetryConfig
    • Retry on HTTP status codes: 408, 429, 500, 502, 503, 504
  • Remove custom retry logic (~70 lines):
    • runWithRetry() method
    • retryPolicy() method
    • containsErrorCodes() method
    • ERROR_PATTERN constant
  • Simplify HTTP calls in makeDataRequest(), makeSearch(), and readRunUrl()

SraExplorerTest

  • Remove obsolete retry tests for custom implementation
  • Remove unused FailsafeException import

Benefits

  • Cleaner architecture: Uses HxClient's built-in retry mechanism
  • Better integration: Aligns with HTTP client usage in RepositoryProvider, WaveClient, etc.
  • Maintained compatibility: Existing retryPolicy configuration still works
  • More maintainable: Centralized retry logic in HxClient
  • Standardized: SraRetryConfig now compatible with broader Nextflow ecosystem

Testing

  • All existing tests pass
  • Retry functionality preserved through HxClient's automatic retry mechanism

Replace legacy `URL.getText()` with HxClient for better retry handling
and consistency with other HTTP clients in the codebase.

Changes:
- Replace legacy java.net.URL HTTP calls with HxClient
- Update SraRetryConfig to implement Retryable.Config interface
- Remove custom retry logic (~70 lines) in favor of HxClient's built-in retry
- Configure HxClient with retry policy and status codes (408, 429, 500, 502, 503, 504)
- Remove obsolete retry tests that tested custom implementation

Benefits:
- Cleaner architecture using standardized HTTP client
- Better integration with Nextflow's HTTP infrastructure
- Maintained backward compatibility with existing retry configuration
- More maintainable with centralized retry logic

Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
@pditommaso pditommaso requested a review from jorgee October 10, 2025 09:01
@netlify
Copy link

netlify bot commented Oct 10, 2025

Deploy Preview for nextflow-docs-staging canceled.

Name Link
🔨 Latest commit 9781440
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-docs-staging/deploys/68e906a63887ec00088724ab

@pditommaso
Copy link
Member Author

@claude review

@claude
Copy link

claude bot commented Oct 10, 2025

Claude finished @pditommaso's task —— View job


Code Review Completed ✅

Summary: This is a well-executed refactoring that successfully migrates SraExplorer from legacy HTTP handling to HxClient. The changes improve code maintainability while preserving existing functionality.


Strengths

Clean Architecture Improvement

  • Removes ~70 lines of custom retry logic in favor of HxClient's built-in retry mechanism
  • Eliminates custom runWithRetry(), retryPolicy(), and containsErrorCodes() methods
  • Significantly simplifies HTTP operations in makeDataRequest(), makeSearch(), and readRunUrl()

Proper HxClient Integration

  • getHttpClient() method correctly configures HxClient with HTTP/1.1 and normal redirects
  • Retry configuration properly uses SraRetryConfig with appropriate HTTP status codes (408, 429, 500, 502, 503, 504)
  • Follows established patterns from RepositoryProvider.groovy:232-240

Backward Compatibility

  • SraRetryConfig correctly implements Retryable.Config interface
  • Existing retryPolicy configuration continues to work seamlessly
  • All existing constructor and configuration patterns preserved

Clean Implementation

  • SraRetryConfig.groovy:33-78: Proper interface implementation with all required methods
  • SraExplorer.groovy:302-309: Clean HTTP request handling with HxClient
  • Appropriate test cleanup in SraExplorerTest.groovy

📋 Technical Notes

SraRetryConfig Changes

  • ✅ Correctly implements Retryable.Config interface methods: getMaxAttempts(), getJitter(), getMultiplier()
  • ✅ Adds multiplier field with sensible default (2.0) for exponential backoff
  • ✅ Maintains existing configuration map-based initialization pattern
  • ✅ Consistent with nextflow.util.RetryConfig implementation

SraExplorer Refactoring

  • getHttpClient() method lazy-initializes HxClient with proper retry configuration
  • ✅ Replaces new URI(url).toURL().getText() with cleaner HxClient.sendAsString() pattern
  • ✅ Removes complex custom retry logic while maintaining the same retry behavior
  • ✅ HTTP client configuration matches patterns used elsewhere in the codebase

Test Updates

  • ✅ Removes obsolete retry-specific tests that are no longer relevant
  • ✅ Core functionality tests remain intact and should continue passing

🔍 Minor Observations

  1. Line 296: The .retryConfig(config) call is correct, though the variable name shadows the class name HxConfig - this is acceptable in this context

  2. Consistency: The implementation aligns well with other HxClient usage patterns in the codebase (e.g., RepositoryProvider, TowerClient)

  3. Error Handling: The migration preserves existing error handling while letting HxClient handle transient HTTP failures


Verdict

This PR represents a high-quality refactoring that:

  • Reduces code complexity (net -43 lines)
  • Improves maintainability by centralizing retry logic
  • Maintains backward compatibility with existing configurations
  • Follows established patterns from the codebase
  • Preserves functionality while improving robustness

The changes are technically sound and ready for merge.


Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
@pditommaso pditommaso merged commit 84e3e78 into master Oct 10, 2025
10 checks passed
@pditommaso pditommaso deleted the migrate-sra-explorer-to-hxclient branch October 10, 2025 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant