Skip to content

fix: more normalization during comparisons#8046

Merged
jeremylong merged 1 commit intomainfrom
normalization
Oct 15, 2025
Merged

fix: more normalization during comparisons#8046
jeremylong merged 1 commit intomainfrom
normalization

Conversation

@jeremylong
Copy link
Copy Markdown
Collaborator

Description of Change

attempt to solve seemingly random test failures...

@boring-cyborg boring-cyborg Bot added the core changes to core label Oct 14, 2025
@chadlwilson
Copy link
Copy Markdown
Collaborator

The problem seems similar to what was discussed in #7338 - that one seemed to be resolved or mitigated by changing threading and otherwise simplifying it.

I tried at the time to find a reproducer for a "raw" comparison contract violation (without a threading issue) but couldn't find one. When I looked at the code in the CPE Parser I saw things that seemed "fishy" between equals/compareTo/hashCode, but never got any further than a hunch :-D

.append(versionEndIncluding)
.append(versionStartExcluding)
.append(versionStartIncluding)
.append(normalizeForComparison(versionEndExcluding))
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: this change seems to have no effect onto the hash code computation.

By implementing the following test in core/src/test/java/org/owasp/dependencycheck/dependency/VulnerableSoftwareTest.java

    @Test
    void test() throws CpeValidationException {
        VulnerableSoftware vuln1 = new VulnerableSoftware(Part.APPLICATION, "vendor", "product", "version", "update",
                "edition", "language", "swEdition", "targetSw", "targetHw", "other", "", "", "", "", true);
        VulnerableSoftware vuln2 = new VulnerableSoftware(Part.APPLICATION, "vendor", "product", "version", "update",
                "edition", "language", "swEdition", "targetSw", "targetHw", "other", null, null,
                null, null, true);

        assertEquals(vuln1.hashCode(), vuln2.hashCode());
    }

I observe this above test assertion to be always green, on normalization and main branches.

I suppose the goal here was to bring implementation consistency with equals() (whose change below does change the equals result) and compareTo() methods instead of aiming to change any behavior?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct - just making compareTo and equals behave the same.

@chadlwilson I'll go take a look at the CPE Parser to see if the issue is actually there.

Copy link
Copy Markdown
Collaborator

@chadlwilson chadlwilson Oct 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The timsort implementation never calls equals or hashCode so the issue is quite likely in compareTo somewhere assuming nothing is modifying any objects across threads (which I think is safe now?).

I’ve been trying to replicate a contract violation using some brute force but I haven’t found one so far. I was suspecting somewhere in compareVersions wasn’t conforming since most of the rest of the comparison is pretty simple.

There are some minor compareTo contract violations (CPE doesn’t throw an NPE on comparing to null which it should by spec and VulnerableSoftware breaks the spec if compared to a CPE object) but neither of those seem possible in reality or the root cause here.

If it’s happening reasonably regularly on the tests perhaps we could just catch and log the list of CPE strings during any IllegalArgumentException in getVulnerableSoftware to find a possible reproducer? I’ll try and see if I can replicate locally with that.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See stevespringett/CPE-Parser#285. There was a bug for large numeric version parts - but I can't believe that is the root cause of random failures....

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found at least one reproducer that is causing EngineIT to fail:

The version comparison is definitely flawed.

stevespringett/CPE-Parser#285 (comment)

@jeremylong jeremylong merged commit d900542 into main Oct 15, 2025
5 checks passed
@jeremylong jeremylong added this to the 12.1.9 milestone Oct 15, 2025
@jeremylong
Copy link
Copy Markdown
Collaborator Author

stevespringett/CPE-Parser#285 <- not convinced that will resolve the issue.

@nhumblot nhumblot deleted the normalization branch October 15, 2025 11:48
@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Nov 15, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

core changes to core

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants