Limitations & Requirements

So the limitations with the previous system were:

context arguments sometimes being mentioned and sometimes not is a bit confusing
every function's input / output types are set in stone at registration. this means that when making a function like if-else, you have to create and register it for both booleans and numbers
mostly due to the point above, the only things a function can output are booleans and numbers, and adding another type that a function can output would involve making another Registry
the design of the code weighed too heavily towards making datagen "easier" (i.e. making red squiggly lines appear if you screw up the types) by providing a factory function for each DFunctionType, resulting in massive type signatures where even the simplest looked like DFunctionType<Boolean, ? extends Function<ContextArg<Entity>, ? extends DFunction<Boolean>>>. This also resulted in the huge DFunctionImplementations class.
there was no way to create objects from JSON in a DFunction. I wanted this so that I could return 'actions' like 'set player on fire for x ticks' from my DFunctions.

So the new system:

Needed a way to create generic functions: if-else needs to be a <a> (a, a, boolean) -> a function, where a is inferred
Needed to do away with context arguments: instead, it just needs to have a variable system, so i can then do {"var": "entity"} and I know that it understands I mean the variable named entity
Needed its own type system separated from Java's, that can express:
template types like <T> (with bounds)
parameterised types like List<T> (this is required so that we can have things like an is_in_tag function that takes a TagKey<T> and a T and gives a boolean)
Still needed to be codec-able, of course
Needed lambdas (so we can have a function map_optional (Optional<A>, Function<A, B>) -> Optional<B>
Needed some way to construct objects

The Result

What I ended up doing was separating the actual language, which I'm calling 'vampilang', out from the data-functions mod. Here it is: https://github.com/williambl/vampilang

vampilang consists of two modules:

A lang module which handles the language itself: parsing, evaluating, etc.
A stdlib module which contains standard types like boolean and number, and functions like if-else and >=

data-functions then builds on top of this to add more types (entity, tag) and functions (boolean_game_rule, swimming)

data-functions also provides a VEnvironment. This is essentially the world that vampilang knows, containing all the types and functions that are available.

Using the new data-functions

Java-side

An expression in vampilang is represented with the VExpression class in Java.

Getting + Evaluating

To obtain a codec which will give you a VExpression which resolves to a particular type, use DFunctions#resolvedExpressionCodec. This takes the expected final type, and a specification of which variables are available (with which types). Standard variable specifications are available in DFunctions.

You can then evaluate your VExpression with VExpression#evaluate. You will need to pass a VEnvironment (there is one in DFunctions). This will give you the result wrapped in a VValue object. To get an unwrapped result, use the convenience method DFunctions#evaluate.

New functions

Normal functions

To create new functions, simply create a VFunctionDefinition object. For example:

// note - this is a builtin data-functions function so the name ('position') is bare - your functions should
// be in 'modid:name' format!
public static final VFunctionDefinition POSITION = new VFunctionDefinition("position", new VFunctionSignature(Map.of(
        "entity", DTypes.ENTITY),
        DTypes.VEC3),
        (ctx, sig, args) -> new VValue(sig.outputType(), args.get("entity").get(DTypes.ENTITY).position()));

Then, in your mod initaliser (note: there should be a dedicated entrypoint provided by data-functions for this!), register them to the VEnvironment. This makes the functions exist in the world that vampilang expressions live in.

Functions with interesting type signatures

Sometimes the signature of a function is complicated. Follow these rules:

the same instance of a template type will be treated as the same template (e.g. two uses of StandardVTypes.TEMPLATE_ANY will both become <a>. uniquise a template type to make another template which has the same bounds, but a different name.
parameterise parameterised types with your own templates, so the template names are the same. If you don't, you may end up having an (a, list<b>) -> a function when you wanted an (a, list<a>) -> a function.
lambdas are parameterised types - the zeroth type parameter is the output, the others are inputs

    public static final VFunctionDefinition MAP_OPTIONAL = create(() -> {
        var type = StandardVTypes.TEMPLATE_ANY.uniquise(new HashMap<>()); // <a>
        var output = StandardVTypes.TEMPLATE_ANY.uniquise(new HashMap<>()); // <b>
        return new VFunctionDefinition("map_optional",
                new VFunctionSignature(Map.of(
                        "optional", StandardVTypes.OPTIONAL.with(0, type), // optional: optional<a>
                        "mapping", StandardVTypes.OPTIONAL_MAPPING.with(0, output).with(1, type) // mapping: optional_mapping: (a) -> b
                ), StandardVTypes.OPTIONAL.with(0, output)), // output: optional<b>
                (ctx, sig, args) -> {
                    var optContainingType = ((VParameterisedType) sig.inputTypes().get("optional")).parameters.get(0);
                    Optional<Object> opt = args.get("optional").getUnchecked();
                    VExpression mapping = args.get("mapping").getUnchecked();
                    Optional<Object> res = opt.map(o -> mapping.evaluate(ctx.with("unwrapped_optional", new VValue(optContainingType, o))).value());
                    return new VValue(sig.outputType(), res);
                });
    });

New types

To create types, use the various static methods in VType.

When registering types, you can also provide a codec so that you can create objects of that type in vampilang scripts. You'll want to do this for most types. You can also register codecs for parameterised types with VEnvironment#registerCodecForParameterisedType.

Follow these rules:

if your type maps directly onto a Java type, use VType#create(TypeToken)
to create a parameterised type, first make a 'bare type' (you can keep this private, or not keep it around), with the zero-args Vtype#create. Then, use VType#createParameterised.
you can keep around parameterisations of other types, just don't register them! Parameterisations are not distinct types.
similarly, register the bare type of a parameterised type, not the parameterised version itself.
for everything that is a distinct type, register it! You should probably register it under a modid:name-style name.

The DTypes class in datafunctions has a good example of types:

public class DTypes {
    public static final TypedVType<Level> LEVEL = VType.create(TypeToken.of(Level.class));
    public static final TypedVType<Entity> ENTITY = VType.create(TypeToken.of(Entity.class));
    public static final VType OPTIONAL_ENTITY = StandardVTypes.OPTIONAL.with(0, ENTITY); // don't register this, as it's just a parameterisation of another type
    // -snip-
    public static final TypedVType<ItemStack> ITEM_STACK = VType.create(TypeToken.of(ItemStack.class));
    public static final TypedVType<Fluid> FLUID = VType.create(TypeToken.of(Fluid.class));
    // -snip-
    public static final VTemplateType TAGGABLE = VType.createTemplate(ENTITY, DAMAGE_SOURCE, ITEM_STACK, BLOCK_IN_WORLD, FLUID);
    private static final SimpleVType BARE_TAG = VType.create();
    public static final VParameterisedType TAG = VType.createParameterised(BARE_TAG, TAGGABLE);
    // -snip-
    public static void register(VEnvironment env) {
        env.registerType("level", LEVEL);
        env.registerType("entity", ENTITY);
        env.registerType("item_stack", ITEM_STACK, ItemStack.CODEC);
        env.registerType("fluid", FLUID, BuiltInRegistries.FLUID.byNameCodec());
        // -snip-
        env.registerType("tag", BARE_TAG);
        //noinspection unchecked
        env.registerCodecForParameterisedType(BARE_TAG, type -> TagKey.codec((ResourceKey<? extends Registry<Object>>) registryKeyForTaggable(type.parameters.get(0))));
    }
}

Code generation

You can create expressions with the static methods in VExpression. Pass these through a V-Expression codec to generate vampilang JSON scripts.

JSON-side

In JSON, you will be creating V-Expressions. Everything is an expression in vampilang, meaning everything returns something.

There are six types of V-Expression:

Function Application
Raw Value
Variable Reference
Object Construction
List Construction
Lambda

Function Application

A function application is represented as a JSON object. It has one core property: function. The function property must specify the name of the function being applied, such as if-else. Then, you must add a property to the JSON object for each argument of the function, providing another V-Expression as the value.

Example of function applications:

{
  "function": "if-else",
  "a": 1.25,
  "b": 1.0,
  "predicate": {
    "function": "not",
    "operand": true
  }
}

This evaluates to ||1.0||.

Raw Value

A raw value is just that, an unprocessed bit of JSON that's turned into a value by a codec. Here's some examples:

the number 2.0:

2.0

the string 'hi':

"hi"

maybe a string, maybe a block, maybe an item, maybe a fluid - need more context :)

"minecraft:air"

Variable Reference

A variable reference is a JSON object with one property (called var), which specifies the name of the variable to be retrieved from the context the expression is being evaluated in. The context often contains variables such as entity or level.

Some examples:

{
  "var": "damage_source"
}

{
  "var": "entity"
}

Object Construction

An object construction expression is represented as a JSON object with a v-type property. This property holds the name of a ConstructableVType in the VEnvironment the expression lives in.

The other properties of the object are passed through to the vtype to create the object.

Example:

{
  "cooldown_id": "haema:dash",
  "entity": {
    "var": "entity"
  },
  "length": 10.0,
  "v-type": "actions:set_cooldown"
}

This becomes a SetCooldownAction object in Java!

List Construction

A list construction expression is represented as a JSON array. Every element of that array must be another VExpression.

Examples:

a list of numbers

[1.0, 2.0, 3.0]

a list with the 'entity' variable

[{"var": "entity"}]

Lambda

A lambda expression can look just like any other expression. The only difference is that a lambda evaluates to the expression, rather than to the result of the expression.