Skip to content

Remove hppc from LinearizabilityChecker#84878

Merged
rjernst merged 1 commit intoelastic:masterfrom
rjernst:hppc/linearizability_checker
Mar 15, 2022
Merged

Remove hppc from LinearizabilityChecker#84878
rjernst merged 1 commit intoelastic:masterfrom
rjernst:hppc/linearizability_checker

Conversation

@rjernst
Copy link
Copy Markdown
Member

@rjernst rjernst commented Mar 10, 2022

The LinearizabilityChecker has one use of hppc, where it uses a Long to
Object map. Given that the rest of the maps in the Cache class are Java
Maps, and this is only for testing, hppc does not seem necessary.
This commit converts the usage to Map.

relates #84735

The LinearizabilityChecker has one use of hppc, where it uses a Long to
Object map. Given that the rest of the maps in the Cache class are Java
Maps, and this is only for testing, hppc does not seem necessary.
This commit converts the usage to Map.

relates elastic#84735
@rjernst rjernst added >test Issues or PRs that are addressing/adding tests :Distributed/Distributed A catch all label for anything in the Distributed Area. Please avoid if you can. v8.2.0 labels Mar 10, 2022
@rjernst rjernst requested a review from henningandersen March 10, 2022 16:56
@elasticmachine elasticmachine added the Team:Distributed Meta label for distributed team. label Mar 10, 2022
@elasticmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

@henningandersen
Copy link
Copy Markdown
Contributor

Unfortunately this is somewhat performance sensitive due to the brute-force nature of the linearizability checker. The tests risk timing out with this change. I wonder if there is an alternative map implementation available (i.e., that we already use perhaps) that avoids wrapping longs? Or else we will need to build or use one here.

Would it be a viable option to leave the dependency on hppc as a test only dependency?

@rjernst
Copy link
Copy Markdown
Member Author

rjernst commented Mar 10, 2022

The tests risk timing out with this change.

While I understand the concern, I question whether this change is that impactful. Java has become quite good at inlining boxed objects. Also, HashMap's table is a singular array, while hppc's map is two arrays (keys and values), so getting at a value still requires a second dereference once the index is retrieved.

I wonder if there is an alternative map implementation available (i.e., that we already use perhaps) that avoids wrapping longs? Or else we will need to build or use one here

Could we try out this change, and if it results in timeouts, then we can investigate how to fix?

Would it be a viable option to leave the dependency on hppc as a test only dependency?

This would not be ideal. It would still leave open for hppc use in many places, especially since this checker is in the test framework.

@rjernst rjernst mentioned this pull request Mar 10, 2022
43 tasks
@rjernst
Copy link
Copy Markdown
Member Author

rjernst commented Mar 10, 2022

@elasticmachine run elasticsearch-ci/part-2

@rjernst
Copy link
Copy Markdown
Member Author

rjernst commented Mar 14, 2022

@henningandersen I ran 20 iterations of ConcurrentSeqNoVersioningIT.testSeqNoCASLinearizability (with a fixed seed, though I know that doesn't matter much give the test implementation). Baseline on master took 5m17s. This PR took 4m43s. I don't claim this PR is somehow better, but it is certainly not leading to OOMs or slower checks.

@henningandersen
Copy link
Copy Markdown
Contributor

Thanks Ryan. Perhaps also run it 20 times with no fixed seed just to check that it does not OOM with a random seed. The seed does determine the disruption type, size of cluster, number of keys, number of threads and more so might reveal a different result.

@rjernst
Copy link
Copy Markdown
Member Author

rjernst commented Mar 15, 2022

Sorry, I was not clear. The fixed seed was the root seed. Each individual test iteration had a unique seed, based on the root seed. I did the fixed seed so that each iteration used the same seed, but they were all still unique.

@rjernst
Copy link
Copy Markdown
Member Author

rjernst commented Mar 15, 2022

@henningandersen I ran 100 more iterations and did not see any timeouts or OOMs with this PR.

Copy link
Copy Markdown
Contributor

@henningandersen henningandersen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for verifying this thoroughly.

@rjernst rjernst merged commit 68dc709 into elastic:master Mar 15, 2022
@rjernst rjernst deleted the hppc/linearizability_checker branch March 15, 2022 19:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Distributed/Distributed A catch all label for anything in the Distributed Area. Please avoid if you can. Team:Distributed Meta label for distributed team. >test Issues or PRs that are addressing/adding tests v8.2.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants