Macro definition and compilation process

NOTE: this is a proposal, not the way thing are currently done.

For “global” macros

That is, macros that can be used in other modules, distributed as libraries, and so on.

Macros are just regular function. The module can contain all the usual constructs (struct, function, augment and so on).

module MacroLib

struct Foo = {a, b}

function regularFunction = ...

function macro1 = ...
function macro2 = {
	return quote {
		let a = Foo(1, 2)
		regularFunction(a)
	}
}

This code is compiled to MacroLib.class. Since all the code is in the same .class, there is no need to distinguish macros and regular functions, so the macro keyword is not needed (beside documentation)

The module .class file must be in the CLASSPATH_PREFIX in order to compile modules that use the macros. The module must be imported in order to use its macros, since macros can generate code using functions or structures defined in the module itself. The regular functions in the module can’t use macros defined in the same file (but macros can call recursively).

module MyModule

import MacroLib

&macro1()

function main = |arg| {
	&macro2()
}

If the code generated by a macro don’t use elements defined in the macro module (besides other macros), it does not need to be imported, but either the macro must be fully qualified &MacroLib.macro1() or the module must be given as a compilation option, so that the compiler knows where to look for the macros.

For macros “local” to a module

Solution 1

Same as global macros, with conventions and implicit import.

In file1.golo:

module MyModule.Macros

(define anything you want)

In file2.golo:

module MyModule

(use macros)

file1.golo compile to MyModule/Macros.class (as usual: the convention is on the name)
file2.golo is compiled to MyModule.class, with an implicit import MyModule.Macros added.
no need for the macro keyword

Solution 2

Macros are in a file of their own, that contains only macros (defined using macro keyword) and local functions. The module is the same as the module that will be using the macros.

In file1.golo:

module MyModule

macro m1 = ...
macro m2 = ...

local function lf = ...

macro m3 = {
	lf()
	return quote { ... }
}

In file file2.golo:

module MyModule

&m1()

function main = |args| {
	&m2()
	&m3()
}

file1.golo is compiled to MyModule/Macros.class, hence the need for the macro keyword, to differentiate from regular modules;
file2.golo can be compiled to MyModule.class, provided MyModule/Macros.class is in the CLASSPATH_PREFIX. One can than run golo run --module MyModule without dependency on the macro class, since they have been expanded at compile time.
file1.golo and file2.golo can be compiled in 1 pass if specified in that order;
file2.golo can be run with golo golo --files file2.golo, provided MyModule/Macros.class is in the CLASSPATH_PREFIX, thus with a dependency on the macro class.

The module MyModule.Macros is automatically added to the class to look for macros in, so there is no need to import it in file2.golo, since only macros are exposed, and so no runtime dependency (hence the restriction on the module exposing only macros)

Solution 3

Macros and normal code are in the same file

module MyModule

macro m1 = ...
macro m2 = ...

local function lf = ...

function f = ...

struct Foo = {a, b}

&m1()

function foo = {
	&m2()
}

While in my pov this is the more elegant for “local” macros, I can’t see how to do this, since the previous restrictions can apply.

I initially planned to first compile only macros in a MyModule.Macros submodule (see solution 2) in a first compiler pass, and then expand and compile the rest of the module. But if one of the macro use lf internally, this function should be put in the macro submodule. On one hand, it would require to inspect all macros IR to list required functions, which is hardly feasible since resolution can be complex if the said function is in an imported module), and on an other hand, if regular code also use the same function lf, it is no more visible. Same for non-local functions.

We can’t put only macros in the submodule and import the regular one, since it is not compiled yet.

We could maybe put macros and public functions not using macros in the macros submodule and import it implicitly in the regular one. But:

this adds a runtime dependency
this can be hard to know where to put each function
the semantic of a different class for macros is lost
how about local functions

Making the macro submodule an inner class can maybe solve some visibility issues, but that mean that the inner class must be created before the outer one (in a different compiler pass), which I don’t think possible.

May be it is just impossible or too cumbersome to have the macros and the code using them in the same file.

Any ideas?

Discussion

It is not yet possible to run golo golo without having previously compiled the module containing the macros: E.g.

$ golo compile macros.golo test.golo && golo run --module Test

and

$ golo compile macros.golo && golo golo --files test.golo

works, whereas

$ golo golo --files macros.golo test.golo

does not.

Options:

a different compile command / option to know if we must compile a Macros submodule instead of the use of macro keyword
a compile --macros option to specify where to look for macros.
a --classpath option to the compiler instead of changing CLASSPATH_PREFIX

yloiseau/Pb_compiling_macros.md