Game Test Framework on Forge

The Game Test Framework is a powerful tool provided by Mojang that is included with Minecraft in 1.17 and up. It allows you to declare game tests that can be run inside a Minecraft world and provide a success or fail state. Game tests are similar in concept to unit tests, but they allow you to interact with a running instance of Minecraft. This means you can test block interactions, entity interactions, item functionality, etc. Game tests use template structures to define the dimensions of the test and what blocks and entities will start in the test structure. This framework has useful applications for Continuous Integration (CI) and testing during development to ensure features are working as expected.

For a more in-depth explanation about the Game Test Framework itself and how Mojang uses it to test the base game, please see this video with contributions by Dinnerbone. This guide is tailored towards Forge modders by explaining how it can be used to test your own mods and general setup instructions.

Setting up your environment

Forge version

To make use of the Game Test Framework on Forge, you must be on Minecraft 1.18.1 or higher and Forge 39.0.88 or higher.

Double-check your Forge version in your build.gradle file: It should look something like this:

dependencies {
    minecraft 'net.minecraftforge:forge:1.18.1-39.0.88'

    // Other dependencies ...
}

`gameTestServer` run configuration

The GameTestServer is a special server implementation provided by Mojang that runs all registered game tests and then exits.

This is very useful for CI as the exit code actually represents how many required game tests failed! The GameTestServer is also useful for local development and testing to ensure no game tests are failing after a feature change. A game test is required to succeed by default unless required is set to false in the @GameTest annotation, see this section for more details. A failed optional test will still be logged and reported but will not contribute to the final count of failed required tests.

Adding to your `build.gradle`

NOTE: If you are using an MDK on Forge version `39.0.88` or newer, skip this step!

Older build.gradle files will not contain the gameTestServer run configuration by default.

Add the following to your build.gradle:

minecraft {
    // ...
    
    runs {
        // ...

        // This run config launches GameTestServer and runs all registered gametests, then exits.
        // By default, the server will crash when no gametests are provided.
        // The gametest system is also enabled by default for other run configs under the /test command.
        gameTestServer {
            workingDirectory project.file('run')

            // Recommended logging data for a userdev environment
            // The markers can be added/remove as needed separated by commas.
            // "SCAN": For mods scan.
            // "REGISTRIES": For firing of registry events.
            // "REGISTRYDUMP": For getting the contents of all registries.
            property 'forge.logging.markers', 'REGISTRIES'

            // Recommended logging level for the console
            // You can set various levels here.
            // Please read: https://stackoverflow.com/questions/2031163/when-to-use-the-different-log-levels
            property 'forge.logging.console.level', 'debug'

            // Comma-separated list of namespaces to load gametests from. Empty = all namespaces.
            property 'forge.enabledGameTestNamespaces', '{YOUR_MOD_ID_HERE}'

            mods {
                examplemod {
                    source sourceSets.main
                }
            }
        }
    }
}

Be sure to replace {YOUR_MOD_ID_HERE} with your actual mod id! After setting this up, regenerate your run configurations. his can be done by running one of the Gradle tasks named genEclipseRuns, genIntellijRuns, or genVSCodeRuns depending on your IDE.

You have now setup the gameTestServer run configuration! However, running it won't do much of anything yet. In fact, running gameTestServer with no registered game tests will exit with a crash.

Game test system with other run configurations

By default, the game test system will only be enabled for the client and server run configurations. Game tests can be accessed through the /test command.

If you want to enable the game test system for other non-standard run configurations, you can do so by adding the following line to your run configuration(s):

property 'forge.enableGameTest', 'true'

Writing game tests

Now for the fun part, actually writing game tests!

Game tests should be logically separate from all your other code. For this guide, we will make a DemoGameTests class.

A simple game test is a method that is annotated with @GameTest, takes in a GameTestHelper, and returns nothing.

This will be the general structure for all game tests you write:

public class DemoGameTests {
    @GameTest
    public static void doTest(GameTestHelper helper) {

    }
}

Scheduling actions

It is important to keep in mind that a game test's main code block is only run once during a given test run. However, game tests can schedule actions to occur at later ticks, like checking if an entity moved to a certain location!

Custom action scheduling can be done with methods provided by GameTestHelper including runAtTickTime, runAfterDelay, and succeedOnTickWhen.

There are also many assertions that GameTestHelper provides to check if a condition is true. These are methods that start with assert. These assertions are similar to JUnit assertions, where a pass will do nothing and a fail will throw an exception. For the game test system, this should always be an unchecked exception of the type GameTestAssertException. Throwing this will signify a test failure and will be logged accordingly.

Assertion methods can be combined with scheduling methods to craft a game test that will wait until after a certain delay to check a condition, then fail or succeed the test.

Success state

While actions and assertions can be scheduled to fail a game test, an explicit success state is required for a game test to pass. A game test with no success will time out after 100 ticks (or 5 seconds) by default and fail. You can configure the timeout amount for a specific gametest to be longer or shorter, see this section for more details.

Runnables can be passed to a few succeed methods in GameTestHelper to signify success if no GameTestAssertException is thrown. No thrown exceptions will signify a "pass". A thrown GameTestAssertionException will signify a failure for that specific tick.

succeedIf - Immediately succeeds the game test if the given Runnable passes on the first tick. If the Runnable passes on a later tick before (not after) the timeout, this will still count as a failure!
succeedWhen - Continually tries to check if the given Runnable passes each tick until the timeout is reached.
succeedOnTickWhen - Tries to check if the given Runnable passes on the exact given tick. If the Runnable passes on an earlier or later tick, this will count as a failure.

This example demonstrates using succeedIf and an assertion method to check if the block at 1, 1, 1 in the template structure is air.

public class DemoGameTests {
    @GameTest
    public static void doTest(GameTestHelper helper) {
        helper.succeedIf(() -> helper.assertBlock(new BlockPos(1, 1, 1), b -> b == Blocks.AIR, "Block was not air"));
    }
}

If our template structure contains a non-air block at (1, 1, 1), the game test will timeout and fail after 100 ticks. Using /test run demogametests.dotest in game, we would get an output like this: The lectern describes more information about the failure if available:

If our template structures correctly contains an air block at (1, 1, 1), the game test will succeed immediately. Using /test run demogametests.dotest in game, we would get an output like this:

Registering game tests

Writing game tests is an important step towards getting a functional test suite for your mod. However, an even more important step is actually registering them so the game test system knows where to find them! There are two ways to register game tests:

`@GameTestHolder`

@GameTestHolder is an annotation that can be applied to a class to automatically register all methods annotated with @GameTest, @GameTestGenerator, @BeforeBatch, or @AfterBatch. The annotations @GameTestGenerator, @BeforeBatch, and @AfterBatch are not completely explained within this guide and further investigation is left up to the reader. @GameTestHolder allows you to declare the default template namespace used for all your game test templates (see below). For our example game test, this would look like so:

@GameTestHolder(ExampleMod.MODID)
public class DemoGameTests {
    @GameTest
    public static void doTest(GameTestHelper helper) {
        // ...
    }
}

Now, any game test methods declared in this class will automatically be registered. This way should be preferred for a concise and easy setup.

`RegisterGameTestsEvent`

RegisterGameTestsEvent is fired on the MOD bus and can be used to register your game tests by passing in a Method instance or Class instance. Passing in a Class will automatically register all methods annotated with @GameTest, @GameTestGenerator, @BeforeBatch, or @AfterBatch. Passing in a Method will register it only if one of the 4 valid annotations is present.

This way to register can be used for dynamic registration or more advanced setups specific to a mod. Example usage would look like so:

@Mod.EventBusSubscriber(modid = ExampleMod.MODID, bus = Mod.EventBusSubscriber.Bus.MOD)
public class ModEventListener {
    @SubscribeEvent
    public static void onRegisterGameTests(RegisterGameTestsEvent event) {
        event.register(DemoGameTests.class);
    }
}

Template structures and you

As previously mentioned, game tests require structures or templates to be able to run. These templates define the physical dimensions of the game test in the world and what blocks/entities it will start out with.

A template always has a name so that it can be referenced and loaded for a game test. By default, this name will be {the_class_lowercase}.{the_method_lowercase}. For our example game test, this would be demogametests.dotest. This period is part of the filename and not a directory. An example structure should look something like this:

The template name for a game test can also be overwritten by declaring a value in the @GameTest annotation. It would look something like this: @GameTest(template = "my_test_template"). This will result in a template structure pointing to <namespace>:demogametests.my_test_template. You can also specify the namespace to load a template from. If you are using @GameTestHolder, this should already be set to your mod id. If you are using RegisterGameTestsEvent, the namespace will default to "minecraft" and needs to be changed. Using a custom template namespace and a template would look something like this: @GameTest(templateNamespace = ExampleMod.MODID, template = "my_test_template"). This will result in a template structure pointing to examplemod:demogametests.my_test_template.

It is important to understand that these examples will still prefix the template name with the name of the class in lowercase! If you want to disable this behavior, you can annotate the game test method with @PrefixGameTestTemplate(false). This annotation can also be used on the class to set the default for all game tests in that class. A method annotation for @PrefixGameTestTemplate will take precedence over a class annotation. Using false, this would result in a template structure pointing to examplemod:my_test_template for the previous example.

For an in-depth look at creating structure files and including them in your mod, please see this tutorial repo by TelepathicGrunt.

`@GameTest` annotation in depth

The @GameTest annotation has many properties that can be configured to control how a game test runs.

This is a complete list of all possible properties and their default values:

Property name	Purpose	Default value
`timeoutTicks`	Defines how long this game test will run without success until it declares a failed state. For any tests involving random block ticks or entities, you will probably want to set this to a high amount of ticks.	100 ticks (5 seconds)
`batch`	Defines a batch which can be used to group similar game tests together. A method annotated with `@BeforeBatch` can be registered like other game test methods to run code before a batch of game tests for setup. This method must take one `ServerLevel`. This also works with `@AfterBatch` which does the same thing but after a batch completes. Batches run sequentially until all batches are complete.	"defaultBatch"
`rotationSteps`	Can rotate a template structure. 0 = None, 1 = Clockwise 90 degrees, 2 = 180 degrees, 3 = Clockwise 270 degrees. Anything outside of 0-3 will throw an exception.	0 (No rotation)
`required`	If required is true, a failed state for this test will fail the entire batch. The exit code of `gameTestServer` is the value of how many required tests failed. If required is false, a failure will be logged but will not affect the batch or exit code.	true
`templateNamespace`	Added by Forge, this property defines what namespace to load the template structure from. This also determines if a game test will be loaded based on the system property `forge.enabledGameTestNamespaces`. If this system property is blank, then all gametests that are registered will be loaded. If not blank, the system property should have namespaces separated by comma. Only if the namespace is contained within this system property will the gametest be loaded.	"minecraft"
`template`	Defines the structure path relative to `data/<modid>/structures` used to load the template for the game test. Also see the relevant documentation for `@PrefixGameTestTemplate` here.	"" (empty string)
`setupTicks`	How many ticks to wait after spawning in the structure before the game test should be executed.	0
`attempts`	How many times to attempt to run the game test until it meets `requiredSuccesses`. A value greater than one means the game test is considered "flaky" by the game test system. This is only useful for tests that have an element of randomness to them.	1
`requiredSuccesses`	How many successful passes of the game test are required before the test is considered passed. This value does nothing if `attempts` is set to 1.	1

Wrapping it up

This guide went over the basics of how to setup the Game Test Framework with Forge by adding the run configuration to your build.gradle, writing a basic test, declaring your template structure, and explaining all the properties of @GameTest. However, this guide is not comprehensive and merely scratches the surface of the framework. You are highly encouraged to experiment, read the code yourself, and try to find your own uses! A good place to start would be with GameTestHelper to see what all helper methods are available to you in your game tests.

While this system is relatively fresh, there are not many other resources available to learn from. This list may be amended over time as more resources come out. You can find extra examples and information about the Game Test Framework from the following links:

More example game tests including an energy capability game test and dropped item in a hopper game test
Game tests in action in Compact Machines 4
Example Github CI configuration for running gameTestServer - requires setting forceExit false in gameTestServer {} as shown here

If you need more help or guidance, feel free to ask in the Forge discord.

SizableShrimp/README.md