Thoughts on a PHP module system

module_demo.php-ish

////////////////
module My\Stuff\MyModule; // implies “namespace My\Stuff\MyModule”

export class Foo {  }

export function foo() {  }

function bar() {  } // not exported

export string $message = “your ad here”;

export const BEST_NUMBER = 69;

////////////////
module Other\Stuff\OtherModule ;

// import specific items.  Specific syntax like ordering is obviously not set in stone.
import foo as my_foo from My\Stuff\MyModule;
my_foo()

//// module objects, in the const namespace (like classes afaik)
import My\Stuff\MyModule as mod;

mod::Foo $foo = new mod::Foo(); // static things use static syntax
mod::BEST_NUMBER;               // consts also being static things

mod->foo();    // functions become instance methods
mod->message;  // variables become props

//// Above is the most basic stuff.  Nice-to-have features below.

//// dynamic imports
$modname = “My\\Stuff\\MyModule”; // yeah, friggin backslashes i know
$modref = import($modname);
$modref->foo();

//// ::module literals
$modname = My\Stuff\MyModule::module;
$modref = import($modname);

// My\Stuff\MyModule::module doesn’t *have* to be just a string like ::class is,
// but should probably stringify the same for interoperability.
// If ::module were fancier than a string, then perhaps auto-importing...
$modref = My\Stuff\MyModule::module;
$modref->foo();	// -> and :: operators trigger auto-import

//// Module metadata (not specified: how to declare or set it)

// you didn't think we'd use _global_ functions to manage a _module_ system, did you?
$meta = PHP\Module\Metadata::module;
$mm = My\Stuff\MyModule::module;
$meta = $meta->load($mm);

// alternatively, just this:
$meta = PHP\Module\Metadata->load(\My\Stuff\MyModule::module);

$meta->version;
$meta->source;
$meta->bytecode;  // ok maybe not :)
$meta->get('user-defined-key', 'defaultval');

//// random thoughts on a package system
$pkg = $meta->package;
$pkg->version;
$pkg->license;
$pkg->author->name;
$pkg->author->url;

// Now for the money shot.  The syntax is definitely up for grabs here.
import Symfony\Component\Messenger as m in "my-forked/messenger" with ['version' => '>=2.4'];

wilder-ideas.php-kinda

//// parameterized modules
module Greeter;

export readonly var $name;  // let's resurrect 'var' for declarations
\PHP\Module\Lifecycle->on_import(fn(array $args) => $name = $args['name']);

export function hello() {
  echo "hello $name!\n";
}

// on_import is arbitrary code.  the sky is the limit, as is the deepest level of hell.

module App;
import * as greeter from Acme\Greeter with ['name' => "Jeff"];

greeter->hello();
$dude = greeter->name;  // also exported

//// importing a legacy namespace as a module, then using that module as a namespace
import * as msgr from Symfony\Component\Messenger;

export $send = fn(msgr\MessageBusInterface $bus, ...$args) => $bus->dispatch(...$args)


//// anonymous modules.  with parameters too, why not.  it's basically a function with 162% More Fun.


$greeter = module(string $name, Writer $out) {

  // no special class member syntax, it's all top-level PHP in here.
  $name === 'donnie' and throw new STFUException();

  export $greet = fn() => $out->printLine("hello $name!");
};

$g = $greeter("dude", $some_out_thingie);
$g->greet();

//// parameterized modules
module Greeter;

export readonly var $name;  // let's resurrect 'var' for declarations
\PHP\Module\Lifecycle->on_import(fn(array $args) => $name = $args['name']);

export function hello() {
  echo "hello $name!\n";
}

+100 for using var

Less sure about using $args array. Before programming in Go I would have been all for it, but now that I have used typed a LOT more, I feel like it should be more typesafe: fn(MyClass $c) => $name = $c->name)

// on_import is arbitrary code.  the sky is the limit, as is the deepest level of hell.
module App;
import * as greeter from Acme\Greeter with ['name' => "Jeff"];

greeter->hello();
$dude = greeter->name;  // also exported

Although the part of me that likes cool things is excited by the above, I wonder if it would not just be a lot simpler for the language to delegate parameters to code in the package itself?

Here is how I have been thinking about packages, which leverages more of existing PHP features and required fewer new features. The only thing required here is to be able to hide the symbols in path/to/acme/api.php from other code:

$acmeAPI = require_once("path/to/acme/api.php");

$greeter = $acmeAPI->newGreeter("Jeff");
$dude = $greeter->name;

//// importing a legacy namespace as a module, then using that module as a namespace
import * as msgr from Symfony\Component\Messenger;

export $send = fn(msgr\MessageBusInterface $bus, ...$args) => $bus->dispatch(...$args)

Wait, what?!? Are you saying this would take Symfony\Component\Messenger as a namespace?

//// anonymous modules.  with parameters too, why not.  it's basically a function with 162% More Fun.
$greeter = module(string $name, Writer $out) {
  // no special class member syntax, it's all top-level PHP in here.
  $name === 'donnie' and throw new STFUException();
  export $greet = fn() => $out->printLine("hello $name!");
};

$g = $greeter("dude", $some_out_thingie);
$g->greet();

Okay, my head hurts again!

Also, feels like there should be a more generalized way to create a controlled scope of top-level PHP code than to make it module specific?

Currently we can do this, but it does not hide the class:

{
   class Foo {}
}

What if this did?

return {
   class Foo {
      public $name = "Jim";
   }
   return new Foo();
}
// OR
$foo = {
   class Foo {
      public $name = "Jim";
   }
   return new Foo();
}
echo $foo->name;    // Prints: Jim
$foo2 = new Foo();  // Generates an error

Thanks for all the feedback! I'm juggling a few other tasks ATM but I wanted to address a few general points, make some edits to the files up top, then put together some more detailed responses to some of the questions and points below.

Would it make everything package-private unless otherwise exported?

Correct, nothing is accessible unless exported. In MyModule, bar() is private to the module, or as I put it simply, "not exported". Same sort of package-private semantics though, everything in the module's namespace (modules are basically reified namespaces) can access it, nothing outside it can. I avoid saying "package-private" since I'm trying to keep the concept of "packages" with all their metadata like versions and source repos separate from the simpler reified namespaces that are modules.

The import and export keywords would only work within a module and be a syntax error anywhere else, much like trying to use public and private outside of a class.

//// This of course works too
namespace My\Stuff;
module MyModule;

This was of course a total brainfart with a now ironic comment, and it's deleted from the latest revision. Nested namespaces aren't a thing, and while they could perhaps become a thing through bracket syntax, I'm not proposing it. So just assume we're doing module My\Stuff\MyModule from now on.

If we had this, then what would happen is someone wrote this in another file?

namespace My\Stuff\MyModule
function example {}

Would example() then automatically become part of the module? If yes, would that mean I could force an existing namespace to become a module? Would that cause any problems?

It's a good question. Interoperability between modules and namespaces is a goal, but it's also a big can of worms. I'm thinking the behaviors should be something like this:

• It should be possible to import a namespace as a module.
• variables, functions, and consts are automatically exported when defined under namespace, but not under module. Somewhat analogous to the behavior of classes vs structs in C++.

If no, then I assume that means PHP would disallow a namespace once a module has already been created with the same name? Given there is nothing that currently stops someone from defining an existing namespace in a new file, would that cause any potential issues?

Part of me wants to say they would merge the same way namespaces do now, using the above semantics for auto-exporting namespaces promoted to modules... but I'm now leaning toward just forbidding it, or at least having the module only see what was declared under module.

We're definitely in the tall grass here and could spend days hammering out the details of module/namespaceinterop. That certainly needs to be done, but I'll be handwaving away much of it for the rest of the reply except of course where it's the only issue at hand.

I'll just note that while anyone can declare any namespace anywhere, PHP already prevents redefining functions (a behavior I hate BTW), so it's not like there isn't already a potential runtime landmine in doing so.

BTW, there appear to be ~3500 instances of export used as a symbol in PHP files on public GitHub, which is a bit of a BC concern:

3500 is actually not a whole lot all told. As Theoretical BDFL, I decree that PHP now have more context-sensitivity in its parser (lexer really) and that the keywords are only recognized within a module. Within a module, BC breaks are served for breakfast. There, all solved :)

There's a case for using the public and private keywords instead of export and the default of not exporting. I still think modules should be private by default, but I guess I could hold my nose and use public to mean export. There's still the module and import keywords to deal with though.

Overall, this omelette recipe calls for breaking a few eggs, so I want to make it worth it.

import My\Stuff\MyModule as Mod;

Mod\foo()
Mod\bar();  // error — exported members only

// import specific items.  Specific syntax like ordering is obviously not set in stone.
import foo as my_foo from My\Stuff\MyModule;
my_foo()

Those seem like they could be in conflict, Mod being a namespace alias and my_foo being a symbol? But it is twisting my brain so I cannot say for sure if those two are unambiguous or not.

I'm now strongly leaning toward disallowing Mod\foo() or using a module like a namespace at all, and I've deleted such uses from the file. It now looks like this instead

import My\Stuff\MyModule;
MyModule->foo();  // ok
MyModule\foo();   // now a syntax error

import My\Stuff\MyModule as mm; // replaces previous import * syntax 
mm->foo(); // ok
mm\foo();  // syntax error

mod->foo();    // functions become instance methods
mod->message;  // variables become props

I like where the above is headed, but it feels like mod should be an object variable here, as $mod:

I'm somewhat attached to imports being in the constant namespace for a few reasons:

• \Fully\Qualified\Module->foo() should work without an explicit import, similar to how \Fully\Qualified\Namespace\foo() does now.
• They're, well, constant. Yes there's readonly but that's extra noise and is specific to classes at any rate. It could be repurposed, but that seems like extra work for actually negative benefit.

You called out the friggin backslashes before I did. I think that is my biggest pet peeve about PHP, #fwiw.

The backslashes are a passing annoyance for me. Things like variables always being global are what summon my inner vengeance demon, and the global function namespace is always a rich mine of WTFs.

I wonder if it would be better to pass a filepath to the module vs. a module name?

All the examples should work when concatenated into a single file, and if multiple files are involved, then autoloading gets involved. include/require/require_once don't even enter into my vocabulary anymore: the only place I ever want to deal with raw source files is inside of an autoloader, and even then only as a necessary evil.

$meta = PHP\Module\Metadata->load(\My\Stuff\MyModule::module);

Hmm. So you are separating loading modules from importing?

A module's metadata should be loadable without importing the module, yes. Thinking \My\Stuff\MyModule::module would be some sort of lazy proxy that turns into the real deal on demand, but where it's not necessary to do so for just metadata. I gleefully handwave away further details and assume modules under \PHP have sufficient magic to pull it off :)

$meta->version;
$meta->source;

Are these new properties of an object? How and where would they be declared?

Stuff like $meta->file and $meta->source would be built-in, no declaration required. Not sure about $meta->version, which probably belongs in a package anyway. Maybe a MyModule.meta.php file alongside the main one, but that would get cluttered really fast. Maybe using declare() and only parsing the file for declarations? Or maybe just import the module, possible side effects be damned.

$meta->get('user-defined-key', 'defaultval');

Why a key-value and not just a property of a subclass?

A subclass might be better, but I don't want to have to deal with the prospect of autoloading a user-defined metadata class in what might be the middle of an autoload operation itself. I think it ultimately depends the specifics of how metadata gets declared.

//// random thoughts on a package system
$pkg = $meta->package;
$pkg->version;
$pkg->license;
$pkg->author->name;
$pkg->author->url;

Is this something Composer handles, or done some other way with a new package manager?

Modules should be able to take advantage of the existing autoloading infrastructure with very few changes, so Composer should be able to autoload them. I'd want a new entry point to the autoloader for modules, but spl_autoload_module could start off as an alias to spl_autoload.

I'm not enamored with how primitive autoloading is in PHP, but insofar as it can have literally any side effect, it gets the job done better than a more restrictive API would. Enhancements to autoloading are definitely on the table, but I'd definitely want the input of the Composer devs.

import * as m from Symfony\Component\Messenger in "my-forked/messenger" with ['version' => '>=2.4'];

Oh, now my head hurts. 🤕

Hehehe, obviously not for Version 1. I've also changed it to use as instead of import *

[concerning \PHP\Module\Lifecycle->on_import]

Less sure about using $args array. Before programming in Go I would have been all for it, but now that I have used typed a LOT more, I feel like it should be more typesafe: fn(MyClass $c) => $name = $c->name)

I'm thinking the callback should take any type actually, I just used an array for convenience.

// on_import is arbitrary code.  the sky is the limit, as is the deepest level of hell.
module App;
import * as greeter from Acme\Greeter with ['name' => "Jeff"];

greeter->hello();
$dude = greeter->name;  // also exported

Although the part of me that likes cool things is excited by the above, I wonder if it would not just be a lot simpler for the language to delegate parameters to code in the package itself?

That's what the hook does. Or are you saying it should be exposed within the module as __ARGS__ or something and let the top level deal with it? In perl, you process import args in the import sub ... there could be an __import magic function, but that would likely be frowned upon, even if scoped to the module.

Here is how I have been thinking about packages, which leverages more of existing PHP features and required fewer new features. The only thing required here is to be able to hide the symbols in path/to/acme/api.php from other code:

$acmeAPI = require_once("path/to/acme/api.php");

$greeter = $acmeAPI->newGreeter("Jeff");
$dude = $greeter->name;

As I mentioned above, I prefer to wrangle actual files only at the low level of the autoloader. But you can still do it with require_once if you want: just return an anonymous module and you're all set.

//// importing a legacy namespace as a module, then using that module as a namespace
import * as msgr from Symfony\Component\Messenger;

export $send = fn(msgr\MessageBusInterface $bus, ...$args) => $bus->dispatch(...$args)

Wait, what?!? Are you saying this would take Symfony\Component\Messenger as a namespace?

That is a namespace already, this is just an example of importing a namespace as a module. I'm backpedaling on using the namespace separator (that is, the bloody backslash) after a module to access values, but it seems types would have to keep using it.

Also, feels like there should be a more generalized way to create a controlled scope of top-level PHP code than to make it module specific?

The idea of modules is to have a controlled scope, with hopefully as little extra overhead as possible

return {
   class Foo {
      public $name = "Jim";
   }
   return new Foo();
}
// OR
$foo = {
   class Foo {
      public $name = "Jim";
   }
   return new Foo();
}
echo $foo->name;    // Prints: Jim
$foo2 = new Foo();  // Generates an error

I would absolutely love for PHP to have real lexical scope in every block, but unless scopes specifically opted in to such mechanics, I can see it breaking the whole world. An anonymous module is that opt-in, an imperfect solution for an imperfect world.