Add additional test for hasher#6198
Merged
pditommaso merged 5 commits intonextflow-io:masterfrom Jun 19, 2025
Merged
Conversation
Co-authored-by: DriesSchaumont <5946712+DriesSchaumont@users.noreply.github.com> Signed-off-by: Robrecht Cannoodt <rcannood@gmail.com>
✅ Deploy Preview for nextflow-docs-staging canceled.
|
Co-authored-by: DriesSchaumont <5946712+DriesSchaumont@users.noreply.github.com> Signed-off-by: Robrecht Cannoodt <rcannood@gmail.com>
Contributor
Author
|
I added a fix to this PR as well. Let me know if it's acceptable. Check whether this fix solves the issue: $ ./gradlew :nf-commons:test
> Task :nextflow:compileGroovy
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
[Incubating] Problems report is available at: file:///home/rcannood/workspace/rcannood/nextflow/build/reports/problems/problems-report.html
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.14/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD SUCCESSFUL in 26s
25 actionable tasks: 9 executed, 16 up-to-date |
bentsherman
reviewed
Jun 18, 2025
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
bentsherman
reviewed
Jun 18, 2025
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
pditommaso
reviewed
Jun 19, 2025
Signed-off-by: Robrecht Cannoodt <rcannood@gmail.com>
pditommaso
added a commit
that referenced
this pull request
Aug 20, 2025
This implements improved order-independent hashing for directories and unordered collections, controlled by the NXF_PATCH_UNORDERED_DIR environment variable. Key improvements: - Directory traversal order no longer affects hash values - Unordered collections (Sets, Bags) produce consistent hashes - Addresses edge cases with similar directory contents (fixes #6198) - Uses commutative byte addition for order independence - Maintains backward compatibility (disabled by default) The patch can be enabled by setting NXF_PATCH_UNORDERED_DIR=true. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
pditommaso
added a commit
that referenced
this pull request
Sep 3, 2025
This implements improved order-independent hashing for directories and unordered collections, controlled by the NXF_PATCH_DIRECTORY_HASH environment variable. Key improvements: - Directory traversal order no longer affects hash values - Unordered collections (Sets, Bags) produce consistent hashes - Addresses edge cases with similar directory contents (fixes #6198) - Uses commutative byte addition for order independence - Maintains backward compatibility (disabled by default) The patch can be enabled by setting NXF_PATCH_DIRECTORY_HASH =true. Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is related to a bug that was introduced by merging #6197 in order to fix #6112.
By making
hashDirSha256invariant to the order in which data is processed, the link between the file path and its hash is lost.This PR adds a unit test where the filename of two files are swapped, resulting in hashDirSha256 returning the same hash (while it shouldn't).
Credits go to @DriesSchaumont for noticing the issue.
Running the tests added by this PR yields:
I tried going over the different versions of HashBuilder.java in the commits in #6113 but for all of these commits, the tests are passing: