Skip to content

Update trace observers to v2#6257

Merged
pditommaso merged 1 commit intomasterfrom
trace-observers-v2
Jul 8, 2025
Merged

Update trace observers to v2#6257
pditommaso merged 1 commit intomasterfrom
trace-observers-v2

Conversation

@bentsherman
Copy link
Member

@bentsherman bentsherman commented Jul 8, 2025

This pull request introduces significant updates to the Nextflow tracing and observer system, primarily focused on refactoring the observer classes to support a new TraceObserverV2 interface. It also includes code cleanup, optimizations, and enhancements to improve maintainability and functionality.

Refactoring and Interface Updates:

  • Refactored AnsiLogObserver, GraphObserver, and ReportObserver to implement the new TraceObserverV2 interface. This involved renaming methods (e.g., onProcessSubmit to onTaskSubmit) and updating method parameters to use the TaskEvent class. ([[1]](https://github.com/nextflow-io/nextflow/pull/6257/files#diff-483299acf7148ae9c19ae3276b024336bfa608f5d8f5352758a68e779fee4b7fL41-R42), [[2]](https://github.com/nextflow-io/nextflow/pull/6257/files#diff-32f9369643f5607fedb8abadeef4add6bafe8b03cf753924f44056cb27cc5b71L34-R40), [[3]](https://github.com/nextflow-io/nextflow/pull/6257/files#diff-f3013c28d519427f2bc730293ecb6a16a8ba9ab6f81f6151edb8437142effa36L41-R40), [[4]](https://github.com/nextflow-io/nextflow/pull/6257/files#diff-f3013c28d519427f2bc730293ecb6a16a8ba9ab6f81f6151edb8437142effa36L150-R195))
  • Updated DefaultObserverFactory to create instances of TraceObserverV2 implementations and introduced support for additional configuration options like overwrite for file-based observers. ([modules/nextflow/src/main/groovy/nextflow/trace/DefaultObserverFactory.groovyR1-R101](https://github.com/nextflow-io/nextflow/pull/6257/files#diff-f9b14e11087969ea12e554daa956fc203222087d3266b51e65c3a0635396518eR1-R101))

Code Cleanup and Optimization:

  • Removed redundant methods from GraphObserver and ReportObserver that were no longer necessary after the refactor. ([[1]](https://github.com/nextflow-io/nextflow/pull/6257/files#diff-32f9369643f5607fedb8abadeef4add6bafe8b03cf753924f44056cb27cc5b71L111-L136), [[2]](https://github.com/nextflow-io/nextflow/pull/6257/files#diff-f3013c28d519427f2bc730293ecb6a16a8ba9ab6f81f6151edb8437142effa36L150-R195))
  • Replaced mutable variables with final where possible for better immutability and thread safety. ([[1]](https://github.com/nextflow-io/nextflow/pull/6257/files#diff-483299acf7148ae9c19ae3276b024336bfa608f5d8f5352758a68e779fee4b7fL196-R197), [[2]](https://github.com/nextflow-io/nextflow/pull/6257/files#diff-483299acf7148ae9c19ae3276b024336bfa608f5d8f5352758a68e779fee4b7fL250-R251), [[3]](https://github.com/nextflow-io/nextflow/pull/6257/files#diff-f3013c28d519427f2bc730293ecb6a16a8ba9ab6f81f6151edb8437142effa36L275-R258), [[4]](https://github.com/nextflow-io/nextflow/pull/6257/files#diff-f3013c28d519427f2bc730293ecb6a16a8ba9ab6f81f6151edb8437142effa36L296-R272))

Functional Enhancements:

  • Enhanced createObserversV1 and createObserversV2 methods in the Session class to better organize observer initialization and ensure proper metrics enablement. ([[1]](https://github.com/nextflow-io/nextflow/pull/6257/files#diff-bee88a999f8eecd3cec5fcad24b76736162be86626f89d746932f64928f24374L442-R443), [[2]](https://github.com/nextflow-io/nextflow/pull/6257/files#diff-bee88a999f8eecd3cec5fcad24b76736162be86626f89d746932f64928f24374L466-R482))
  • Improved file handling in ReportObserver and GraphObserver by introducing helper methods for path resolution and ensuring directory creation. ([[1]](https://github.com/nextflow-io/nextflow/pull/6257/files#diff-f3013c28d519427f2bc730293ecb6a16a8ba9ab6f81f6151edb8437142effa36L275-R258), [[2]](https://github.com/nextflow-io/nextflow/pull/6257/files#diff-32f9369643f5607fedb8abadeef4add6bafe8b03cf753924f44056cb27cc5b71L55-R63))

Miscellaneous:

  • Updated imports to reflect the new TaskEvent class and removed unused imports. ([[1]](https://github.com/nextflow-io/nextflow/pull/6257/files#diff-483299acf7148ae9c19ae3276b024336bfa608f5d8f5352758a68e779fee4b7fL24-R29), [[2]](https://github.com/nextflow-io/nextflow/pull/6257/files#diff-f3013c28d519427f2bc730293ecb6a16a8ba9ab6f81f6151edb8437142effa36L28-R30), [[3]](https://github.com/nextflow-io/nextflow/pull/6257/files#diff-32f9369643f5607fedb8abadeef4add6bafe8b03cf753924f44056cb27cc5b71L19))

These changes collectively modernize the tracing system, improve code quality, and lay the groundwork for future enhancements.

@bentsherman bentsherman requested a review from pditommaso July 8, 2025 00:12
@netlify
Copy link

netlify bot commented Jul 8, 2025

Deploy Preview for nextflow-docs-staging ready!

Name Link
🔨 Latest commit f5d041e
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-docs-staging/deploys/686d1b8ba5e7e100087859ce
😎 Deploy Preview https://deploy-preview-6257--nextflow-docs-staging.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@bentsherman bentsherman force-pushed the trace-observers-v2 branch from 562508e to 913fbc1 Compare July 8, 2025 00:31
@pditommaso
Copy link
Member

Well done, but looks like many tests are failing

@bentsherman bentsherman force-pushed the trace-observers-v2 branch from dd7eb37 to 576cfd7 Compare July 8, 2025 13:19
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
@bentsherman bentsherman force-pushed the trace-observers-v2 branch from 576cfd7 to f5d041e Compare July 8, 2025 13:22
@bentsherman
Copy link
Member Author

I'm surprised the Groovy type checker didn't catch that, whatever. Tests are passing 👍

Copy link
Member

@pditommaso pditommaso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done

@pditommaso pditommaso merged commit 544b8c4 into master Jul 8, 2025
22 checks passed
@pditommaso pditommaso deleted the trace-observers-v2 branch July 8, 2025 15:15
@bentsherman bentsherman added this to the 25.10 milestone Jul 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants