PIT is a state of the art mutation testing system, providing gold standard test coverage for Java and the jvm. It's fast, scalable and integrates with modern test and build tooling. Visit https://pitest.org/ for more information.
mvn -e --no-transfer-progress -P"$PITEST_PROFILE" clean test-compile org.pitest:pitest-maven:mutationCoverageThis will generate the pit report in the /target folder. We use a custom made script to analyze this report.
Use Groovy version 2.4.21
groovy .ci/pitest-survival-check-xml.groovy "$PITEST_PROFILE"To see the values $PITEST_PROFILE can take, run:
groovy .ci/pitest-survival-check-xml.groovy --listThe default format for reporting violations by the report is:
New surviving mutation(s) found:
Source File: "XMLLogger.java"
Class: "com.puppycrawl.tools.checkstyle.XMLLogger$FileMessages"
Method: "getExceptions"
Line Contents: "return Collections.unmodifiableList(exceptions);"
Mutator: "org.pitest.mutationtest.engine.gregor.mutators.experimental.ArgumentPropagationMutator"
Description: "replaced call to java/util/Collections::unmodifiableList with argument"
Unnecessary suppressed mutation(s) found and should be removed:
Source File: "XMLLogger.java"
Class: "com.puppycrawl.tools.checkstyle.XMLLogger$FileMessages"
Method: "getExceptions2"
Line Contents: "return Collections.unmodifiableList(exceptions);"
Mutator: "org.pitest.mutationtest.engine.gregor.mutators.experimental.ArgumentPropagationMutator"
Description: "replaced call to java/util/Collections::unmodifiableList with argument"
The report can also be generated in XML format using the -g or
--generate-suppression flag.
groovy .ci/pitest-survival-check-xml.groovy "$PITEST_PROFILE" -gOutput (Hypothetical):
New surviving mutation(s) found:
<mutation unstable="false">
<sourceFile>XMLLogger.java</sourceFile>
<mutatedClass>com.puppycrawl.tools.checkstyle.XMLLogger$FileMessages</mutatedClass>
<mutatedMethod>getExceptions</mutatedMethod>
<mutator>org.pitest.mutationtest.engine.gregor.mutators.experimental.ArgumentPropagationMutator</mutator>
<description>replaced call to java/util/Collections::unmodifiableList with argument</description>
<lineContent>return Collections.unmodifiableList(exceptions);</lineContent>
</mutation>
Unnecessary suppressed mutation(s) found and should be removed:
<mutation unstable="false">
<sourceFile>XMLLogger.java</sourceFile>
<mutatedClass>com.puppycrawl.tools.checkstyle.XMLLogger$FileMessages</mutatedClass>
<mutatedMethod>getExceptions2</mutatedMethod>
<mutator>org.pitest.mutationtest.engine.gregor.mutators.experimental.ArgumentPropagationMutator</mutator>
<description>replaced call to java/util/Collections::unmodifiableList with argument</description>
<lineContent>return Collections.unmodifiableList(exceptions);</lineContent>
</mutation>Note: Always make sure you have the correct profile pit report present in \target folder to avoid unexpected results.
Note: Pit operates on bytecode, so if changes in source code don't allign with pit report then see the bytecode to figure out stuff. You can use any decompiler if you are not familiar with bytecode.
From the javadoc of the mutator
Remove switch statements. We get an array of labels to jump to, plus a
default label. We change each label to the default label, thus removing it.
Example: Source Code:
void method(int a) {
switch (a) {
case 1:
foo1();
break;
case 2:
foo2();
break;
case 3:
foo3();
break;
case 4:
foo4();
break;
case 5:
case 6:
break;
default:
throw new IllegalArgumentException("illegal");
}
} Decompiled bytecode:
void method(int a) {
switch (a) {
case 1:
this.foo1();
break;
case 2:
this.foo2();
break;
case 3:
this.foo3();
break;
case 4:
this.foo4();
case 5:
case 6:
break;
default:
throw new IllegalArgumentException("illegal");
}
} Mutation: RemoveSwitch 0 mutation
This mutation will change the 0th index case label and put it into the default label list.
Here is the 0th index case label is case 1:
Mutated bytecode (decompiled):
void method(int a) {
switch (a) {
case 1:
default:
throw new IllegalArgumentException("illegal");
case 2:
this.foo2();
break;
case 3:
this.foo3();
break;
case 4:
this.foo4();
case 5:
case 6:
}
} Similarily mutations like RemoveSwitch 3 mutation,
RemoveSwitch 10 mutation can be tackled.
From the javadoc of the mutator:
Mutator for non-void methods that have a parameter that matches the return
type: it replaces the result of the method call with a parameter.
E. g. the method call
public int originalMethod() {
int someInt = 3;
return someOtherMethod(someInt);
}
private int someOtherMethod(int parameter) {
return parameter + 1;
} is mutated to
public int mutatedMethod() {
int someInt = 3;
return someInt;
} Issue reported with this mutator:
From the javadoc of the mutator:
Mutator for non-void methods whos return type matches
the receiver's type that replaces the method call with the receiver.
E. g. the method call
public int originalMethod() {
String someString = "pit";
return someString.toUpperCase();
}is mutated to:
public int mutatedMethod() {
String someString = "pit";
return someString;
}Detailed information about other mutators is present at: https://pitest.org/quickstart/mutators/
Different decompilers may optimize the bytecode (eg- remove default branch in some cases). In case there is some problem, javap command should be utilized to view the bytecode.
Using the javap command:
$ javap -c -p -l SomeClass.classNote: For nested classes, add quotes around the class name.
$ javap -c -p -l 'SomeClass$NestedClass.class'