Skip to content

Allow to run some tests isolated #2142

@pbludov

Description

@pbludov

We have one test that changes the global state, for example

@Test
void localeTest() {
  Locale.setDefault(Locale.FRENCH);
  assertEquals("Bunjour", getMessage());
}

and hundreds of tests that reads the state:

@Test
void someTest() {
  assertEquals("Hello", getMessage());
}

After enabling parallel test execution our tests start to fail at random locations with messages like that:
[ERROR] message 0 expected:<...Annotation.java:17: [Annotation 'AnnotationAnnotation' have incorrect indentation level 2, expected level should be 0].> but was:<...Annotation.java:17: [Die Annotation 'AnnotationAnnotation' hat eine unerwartete Einrückungstiefe von 2 (erwartet: 0)].>

@Execution(ExecutionMode.SAME_THREAD) for the test localeTest won't help us, since the all other tests are able to run concurrent with every other test and with localeTest too.

This can be solved with @ResourceLock:

@Test
@Execution(ExecutionMode.SAME_THREAD)
@ResourceLock(value="global", mode=READ_WRITE)
void localeTest() {
  Locale.setDefault(Locale.FRENCH);
  assertEquals("Bunjour", getMessage());
}

@Test
@Execution(ExecutionMode.CONCURRENT)
@ResourceLock(value="global", mode=READ)
void test1() {
}

// ...

@Test
@Execution(ExecutionMode.CONCURRENT)
@ResourceLock(value="global", mode=READ)
void test100500() {
}

but this is error prone, since adding a new Test without this annotation will produce a flaky tests which may fail.

It will be much better to mark some tests as "incompatible with parallel execution at all":

@Test
@Execution(ExecutionMode.ALONE) // no other this will be executed in parallel 
void localeTest() {
 Locale.setDefault(Locale.FRENCH);
 assertEquals("Bunjour", getMessage());
}

As a workaround, it is possible to turn such a test into Junit 4 test, which will be executed in other universe, isolated from all other.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions