The case for UniqueId over Legacy Name

Identifiers.md

TestIdentifier - UniqueID vs Legacy Name

Format

• @Test
  • Unique ID: [engine:<engine-name>][class:<quantified-class-name>]/[method:<method-name>(<quantified-class-name>...)]
  • Legacy Name: <method-name>(<simple-class-name>...)
• @ParameterizedTest
  • Unique ID: [engine:<engine-name>][class:<quantified-class-name>]/[test-timplate:<method-name>(<quantified-class-name>...)]/[test-template-invocation:<parameter-set-number>]
  • Legacy Name: <method-name>(<simple-class-name>...)[<parameter-set-number>]

For reference:

• quantified-class-name: package-name + . + class-name + ($ + inner-class-name)
• simple-class-name: class-name or inner-class-name

---

Notes

• Potential name conflicts:
  • @ParameterizedTest test(String) vs @ParameterizedTest test(Integer)
    • Two tests with the same name, but different parameterized arguments, is valid and may exist somewhere in someone's test code
    • How do we differentiate the tests if we are using DiscoverySelectors.selectMethod(class-name, method-name)?
    • Same name but different argument types
• Legacy Name uses simple names, meaning the packages of those classes are not given.
  • Simple @Test methods don't have arguments so its typically <method-name>() so this isn't a problem for @Test
  • Templated tests using @ParameterizedTest do have arguments, so resolving the simple name to a fully quantified one becomes a problem.
• Migrating from using DiscoverySelectors.selectMethod to DiscoverySelectors.selectUniqueId
  • Resolves the problems noted above
  • Simplifies our code since we do not need to parse TestIdentifier in order to select which tests to rerun while keeping the same behavior
  • Does not require new code coverage tests since no new control flow blocks are created (Unit tests and integration tests exist for this feature)

---

Samples

static class TestClass7 {
  static List<Object[]> params() {
    return Arrays.asList(
        new Object[] { "Always pass", true },
        new Object[] { "Always fail", false }  );
  }

  @ParameterizedTest 
  @MethodSource("params")
  void testParameterizedTestCases(String testName, boolean value) {
    assertTrue( value );
  }
}

The second array { "Always fail", false } is what will be re-run so the ID looks like:

UniqueID - testParameterizedTestCases(params[1]): [engine:junit-jupiter]/[class:org.apache.maven.surefire.junitplatform.JUnitPlatformProviderTest$TestClass7]/[test-template:testParameterizedTestCases(java.lang.String, boolean)]/[test-template-invocation:#2]

If it were the first array that used false then it would end with [test-template-invocation:#1]

---

static class TestClass4 {
  static int count;

  @DisplayName( "Always passes" )
  @Test
  void testPass() {}

  @Test
  void testAborted() {
    assumeFalse( true );
    throw new IllegalStateException( "this exception should never happen" );
  }

  @Test
  void testAlwaysError() {
    throw new Error( "some error" );
  }

  @Test
  void testAlwaysFail() {
    assertTrue( false );
  }

  @Test
  @Disabled
  void testAlwaysSkipped() {
    throw new IllegalStateException( "this test should be never called" );
  }

  @DisplayName( "Fails twice" )
  @Test
  void testFailTwice() {
    count += 1;
    assertTrue( count >= 3 );
  }
}

The following are the ID's of failing tests:

UniqueID - testAlwaysFail(): [engine:junit-jupiter]/[class:org.apache.maven.surefire.junitplatform.JUnitPlatformProviderTest$TestClass4]/[method:testAlwaysFail()]

UniqueID - testFailTwice(): [engine:junit-jupiter]/[class:org.apache.maven.surefire.junitplatform.JUnitPlatformProviderTest$TestClass4]/[method:testFailTwice()]

UniqueID - testAlwaysError(): [engine:junit-jupiter]/[class:org.apache.maven.surefire.junitplatform.JUnitPlatformProviderTest$TestClass4]/[method:testAlwaysError()]