elm-native-proposal.md

Proposal: revised Elm Native format

Right now, Native doesn't play well with dead-code elmination. Additionally, Native review is a big bottleneck for producing and publishing new librairies. Finally, there is no way to unclude JS code which calls ports using the Elm compiler.

Example file

/**
--The top of the file declares the structure of the Native module
--and must be a valid Elm syntax declaration
module Native.SomeModule where

--Used Native modules must be explicitly imported
import Native.Utils

--We can import pre-existing library functions
--this includes them in the final output
--but they can't be explicitly called from Elm.
--A Native module must act as an interface.
import raw "Bootstrap.js"

-every used non-Native function must be explicitly imported
import Signal exposing (constant)

--Each exposed Native function is declared as a String literal in Elm
--with the name of the JS function that defines it
--This lets us statically check that no top-level functions are missing

nativeFun = "nativeFun"


otherFun = "otherFun"
**/

//The rest of the JS file is just a series of JS statements

var nativeFun = function(x)
{
  if (Elm.Utils.constructor(x) == "Nothing" )
  {
    Elm.Utils.construct("Nothing");
  }
  //We construct "type" (algebraic data) values using a library function
  return Elm.Utils.construct("Just", 3);
};

var otherFun = function(y)
{
  return nonExposedFunction(y);
};

//We are allowed to declare other functions
//that can't be imported by Elm
var nonExposedFunction(y)
{
  var ret = [];
  //We build records out of arrays
  //and access fields using a library function
  ret["field"] = Elm.Utils.field(y, "elmField");
  ret["field2] = 3;
  
  //a library function constructs the record out of the array
  return Elm.Signal.constant(Elm.Utils.record(ret));
};

What's different in this representation:

• The structure of the module is declared at the top, using normal Elm syntax
• The user doesn't write a .make() function, or return a .values object. That's all done mechanically, we would be guaranteed those functions/values would always be well formed.
• The imported Native modules, and imported and exported Elm functions, must be explicitly declared
• Elm ADT and Record values are not manipulated directly, but using a library function, so Native modules would not break if Elm changed its internal representations.
• There's new syntax for including a JS file in compilation, such as pre-existing libraries.

Pros:

• The module structure is defined using Elm syntax, so we can use existing parsers, and it flows nicely with the rest of the project.
• Much less Native review needed to ensure a module is well formed.
• More resistant to breaking changes.
• The import raw syntax can be used to include the JS side of Ports code.

Cons:

• Could be slower, since we're calling library functions to access Elm values. (Could be solved by inlining).
• Requires pre-processing of comments (could be solved by putting the structure Elm declaration in its own file).
• There are now 3 types of JS ffi (Native, Ports, import raw)

Including Ports code

I believe that there is no reason to include the JS side of Ports code now that tasks exist. Anything that needs to react on an outgoing signal can be written as a Native JS Task. For example, see my pull request to remove the JS Ports handling code from elm-io:
https://github.com/maxsnew/IO/pull/18/files

Explicit import of members from raw JS

I believe you should only be allowed to import other JS files with: import raw "Backbone.js" exposing (Backbone). This will generate code which wraps the included JS file in a local scope, and only returns scope.Backbone:

;var scope = Object.create(window); // read-only access to window
(function (window) {
// file Backbone.js goes here
// it will initialize either this.Backbone or window.Backbone
}).call(scope, scope);
var Backbone = scope.Backbone; // explicitly bring only 'Backbone' into our outer scope

This solves two problems:

• It prevents and tricky js code from overriding window functions either intentionally or unintentionally.
• It prevents Backbone from colliding with another version of Backbone (because both are in their own scopes). This is unlikely with Backbone, but imagine you use a native library that requires jQuery, and then want to run your Elm module on a site which also has (an older) jQuery.

@lazlo, those points both make sense. The second one is quite nice, as it might let us add a second level of static checks making sure the right values are defined. Hopefully things like this will help avoid spaghetti code...

This proposal makes a lot of sense.

However, I wonder whether it might be better to turn it inside-out -- that is, instead of embedding Elm in Javascript comments, why not embed Javascript in Elm? That is, for instance, the general approach that GWT uses for "native" javascript (embedding Javascript in Java comments). But we can make it even nicer than GWT, because we control the compiler -- it wouldn't even have to be 'comments', since we can invent new language-level keywords.

Here's how your example file might look in such a syntax, with some comments:

-- The module declaration is vanilla Elm.

-- If we want to distinguish between "native" modules
-- and other modules, we could require that "native" modules start with 'Native'. Or, we could
-- use a new `native` keyword at the language level. Or, we might not need to specially designate the
-- module as 'native' any longer ... the compiler can, after all, see whether it creates any
-- native functions.
module Native.SomeModule where

-- Used Native modules must be explicitly imported

-- In fact, this would not be a special case ... it's now just an ordinary Elm import
import Native.Utils

--We can import pre-existing library functions
--this includes them in the final output
--but they can't be explicitly called from Elm.
--A Native module must act as an interface.

-- This too could now be ordinary Elm syntax, by adding a `raw` keyword at the language level.
import raw "Bootstrap.js"

-- every used non-Native function must be explicitly imported

-- Again, this would now be just an ordinary Elm import
import Signal exposing (constant)

-- Then the actual functions would be defined inline, something like this. This assumes
-- two things: 
-- 
-- * we would add a `native` keyword at the language level
-- * inside native code, we can refer to things defined in Elm code by using an @{}
--
-- Of course, this implies some compiler changes, and one could invent different syntax.

-- Note that I'm not sure I understand what this function was meant to do,
-- but I'm pretty sure I've translated it to the new form
native nativeFun : Maybe a -> Maybe Int
native nativeFun x =
  if (@{Native.Utils.constructor}(x) == "Nothing")
  {
    return @{Native.Utils.construct}("Nothing");
  }
  else
  {
    return @{Native.Utils.construct}("Just", 3);
  }

otherFun : {elmField: a} -> Signal a
otherFun = nonExposedFun 

-- We are allowed to declare other functions
-- that can't be imported by Elm.

-- In the revised scheme, you can simply decline to expose the function, in the ordinary Elm
-- way. This does raise one additional issue. At the moment, it is possible for native code
-- access non-exposed functions. That is, at least currently, important, because there are
-- lots of cases where native functions exist to be accessed by other native functions in
-- other modules, but which should not be accessed by "regular" Elm code.
-- One way of dealing with this would be to allow the native @{} syntax to access anything,
-- even if it is not exposed. A second option would be to allow a module to expose something
-- to native code only, e.g. an `exposing native` kind of syntax. That way, a module could
-- either expose to Elm and native, expose to native only, or not expose at all.

native nonExposedFunction : {elmField: a} -> Signal a
native nonExposedFunction y =
  var ret = [];
  //We build records out of arrays
  //and access fields using a library function
  ret["field"] = @{Native.Utils.field}(y, "elmField");
  ret["field2] = 3;

  //a library function constructs the record out of the array
  return @{Signal.constant}(@{Native.Utils.record}(ret));

Note that the @{} syntax would need to be converted by the compiler to refer to the actual Elm reference, and use A2, A3 or whatever, depending on the number of arguments supplied. So, the Elm compiler would have to parse the Javascript, but there must be a Javascript parser available in Haskell. And, you could actually guarantee that the Javascript is reasonably well-formed, since the parser would have to understand it.

Here's another advantage of my revision.

Since the Javascript references to Elm things would all be via the @{} syntax, and since the compiler would have to parse the Javascript, there actually would be some opportunities for the compiler to type-check the Javascript itself. For instance, in the revised nonExposedFunction, the compiler, when parsing the Javascript, would know the type of y. It would also know the type signature for Native.Utils.field. So, in principle, a certain amount of type-checking would be possible.

Now, I'm not saying that it would be complete -- it would probably not be. And, some type errors detected in the Javascript code might have to be warnings instead of errors, since it might be part of the point of the native code to do some things that the compiler can't verify in terms of types. However, in principle it is potentially an advantage if the compiler can make some explicit connections between the Javascript code and the Elm types.

Here's another idea for native code -- it's a new idea, not related to my previous comments.

I've been experimenting with Purescript a bit, and there's an idiom they use with "native" code that might be helpful.

To back up, Elm native code currently can have its own dependencies on other Elm modules. So, there is a kind of subterranean dependency graph in the native code, which the Elm compiler (currently) doesn't know about. And, that's one of the reasons for the difficulty with dead code elimination etc. So, one of the purposes of the proposal above is to "surface" that dependency graph.

Purescript approaches this differently. Basically, native code doesn't set up its own dependencies. Instead, if a native function needs something from elsewhere, it must be supplied (from the caller) as a parameter.

Consider a native function which wants to return a Maybe. In Elm, currently, the native module would reach out and get the Just and Nothing constructors itself. In Purescript, the function would ask for the constructors as parameters (possibly a record-type parameter, if there are a number of things that need to be supplied).

The nice thing about this is that the native function is then "pure" (in a sense) -- it only relies on its parameters and whatever is natively available in the Javascript runtime. Thus, the compiler doesn't need to special-case the reasoning about what the native function is using -- it knows in the regular way, since it sees the parameters.

So, that's just grist for the mill -- it seems elegant to me, but I understand that there are a lot of considerations involved here which may lead to other approaches.