First look what´s breaking after upgrading Selenium to 4.x

On my dev-machine all tests pass.
But ... Selenium 4 seems to be an resource hug.
With Selenium 3 my dev-machine (8C/16T, 32 GB RAM) was still responsive during running PF-integration-tests. And it consumed something like 2-3 GB of RAM. With Selenium 4 my dev-machine is not responsive anymore. Running the exact same tests consumes about 14 GB of RAM.

With junit.jupiter.execution.parallel.enabled=false we are up and running on Selenium 4.
--> Looks like Selenium 4 has some concurrency-issues.

Great stuff!

I also wonder in our check for JavaScript errors if I remmeber it only worked for Chrome and we might have disabled it for Firefox but I thought Firefox had fixed its console errors check for 4.0

I also wonder in our check for JavaScript errors if I remmeber it only worked for Chrome and we might have disabled it for Firefox but I thought Firefox had fixed its console errors check for 4.0

There´s some kind of activity at Mozilla. But does not work yet. (Just tested it.)
Current comments on mozilla/geckodriver#284 sound like this may start working sometimes in 2022.

then we have to check parallel issues - thats a no go for me currently if this doesnt work

Same thoughts. We have to see what the selenium-guys say. No need to hurry.

The selenium-team provides a work-around. (implementing a custom JUnit5-strategy)
But does not work for Java8. Only Java 11. And it´s at least partial broken for Java 17.

Selenium 4.1.1 released.

Summary of further research and conversation with the selenium-team:

• Selennium 4 internally switched to async (eg AsyncHttpClient, ...)
• JUnit 5 uses ForkJoinPool for parallel execution
• ForkJoinPool has quite some magic - when a thread run´s into some blocking operation another thread (up to maxPoolSize) is spawned. ForkJoinPool-parallelism defines how much threads run concurrent. But up to maxPoolSize threads may be created. (in this usecase each Thread is one browser-instance)
• To limit maxPoolSize we have our own JUnit5Selenium4Strategy. This limit is not recognized for Java 8 because ForkJoinPool only allows since Java 9 to define maxPoolSize. --> Java 8 has to run serial.
• Java 17 did some internal changes to ForkJoinPool causing other issues. (java.util.concurrent.RejectedExecutionException: Thread limit exceeded replacing blocked worker) --> We need to set saturate-parameter of ForkJoinPool to true. There´s already a JUnit 5 - PR which enhances org.junit.platform.engine.support.hierarchical.ParallelExecutionConfiguration with this ability. (coming for JUnit 5.9)

--> We need a) wait for JUnit 5.9 and b) disable parallel test execution for Java 8.

thanks for your effort @christophs78 and your communication with the selenium team!

1. There´s some (additional) issue with Selenium 4.1.2 which generates
   java.lang.NoSuchMethodError: 'com.google.common.collect.ImmutableMap com.google.common.collect.ImmutableMap.of
   (so i temporary stepped back to 4.1.1)
2. @tkumar18 figured out setting MinimumRunnable to 0 (as part of ParallelExecutionConfiguration) helps to work around the Java 17 - specific issues.

Up and running on Java 11 and Java 17.
Some idea how i can set junit.jupiter.execution.parallel.enabled=false on Java 8 before JUnit kicks in?

Can we just disable the tests on Java8? I mean running Integration tests on 11 and 17 should be good enough, unit tests and compilation on Java8 should be good enough.

Good enough for now and ready to be merged. :-)

so, java11 and later runs paralell and as fast as with selenium 3?

so, java11 and later runs paralell and as fast as with selenium 3?

yes

Did you close this PR intentionally?

We have the same matrix than before; Java 8 serial, everthing else parallel; there´s a new maven profile (''integration-tests-enable-parallel-execution") to explizit enable parallel execution.

Side-note:

<activation>
    <jdk>[9,)</jdk>
</activation>

causes compile errors. So we have a new if as part of build.yml which adds 'integration-tests-enable-parallel-execution' for java != 8.

oh, wrong button, sorry :D

cant we just split into 2 profiles?
-Pintegration-tests, paralell-execution

cant we just split into 2 profiles? -Pintegration-tests, paralell-execution

it´s already splitted

<profile>
    <id>integration-tests-enable-parallel-execution</id>
    <properties>
        <junit.jupiter.execution.parallel.enabled>true</junit.jupiter.execution.parallel.enabled>
    </properties>
</profile>

should i remove the "integration-tests-" - prefix from the profile-name?

@tandraschko i think this is safe to merge.

will review and merge this week