namespaces.md

Namespaces 101

Introduction

Ruby has recently merged namespaces as an experimental feature that is disabled by default, as of this writing.

This is a non-trivial development driven by @matz himself, and mainly implemented by @tagomoris, who just became a Ruby committer (🎉).

The feature has been cooking for a long time, with a first ticket opened a couple of years ago (#19744) and a revised one opened just last week (#21311).

There is already some documentation here, and the tickets have tons of discussions. About details, understanding the proposal, performance considerations, objections, ..., it is a bit hard to follow.

In this post I'd like to present a digested summary of namespaces.

I would like to give you a mental model for them so you know what is coming.

The main idea

Let me show you some code:

require 'nokogiri'

module M
end

class String
  def blank? = ...
end

ns = Namespace.new
ns.require('foo')

OK, the main idea behind namespaces is that the code in foo.rb runs isolated from the parent context:

• As if Nokogiri had not been loaded.
• As if the M constant had not been defined.
• As if String had not been monkey-patched.

Therefore:

• foo.rb can load Nokogiri without conflict, even a different version.
• foo.rb can define its own top-level M without conflict.
• foo.rb can define its own core extensions without conflict.

It all runs in the same process, but the things reachable from within and outside the namespace are kind of shielded, that is the point.

Transparency

The author of foo.rb wrote it to be executed normally, like gems are written right now.

This feature allows users of foo.rb to load foo.rb normally, or through a namespace. It is up to the caller. The code in foo.rb should work correctly as-is in both scenarios.

I have personally seen some discrepancies (#21316, #21318, #21320), but this is all preliminary and they seem to be fixable.

Global variables are per-namespace

We said before that foo.rb is loaded as if Nokogiri had not been loaded.

Cool, in particular, if foo.rb needed Nokogiri too, it could load it:

# foo.rb
require 'nokogiri' # => true

But isn't Kernel#require idempotent?

Well, in this new paradigm global variables are per-namespace, so $LOADED_FEATURES (the array of required files Kernel#require bases its idempotency on) does not have the mutation performed by the main file.

Note that means we have Nokogiri loaded twice in memory. Not very much unlike what would happen if you forked for isolation. So you have the ability to do that, and it has a cost, as everything (that has a cost :).

Builtin constants, classes, and modules

In the previous section we saw that M does not exist when foo.rb is loaded. However, other top-level constants like String or Object do exist. How does that work?

Namespaces introduces a clear separation between builtin constants and user-defined ones. Hash is a builtin constant, and M is not. The builtin ones get preserved.

This is consistent with the main idea: When you start a program, those constants exist, right? So that is what you see under a namespace too.

But, we saw also that the monkey patch in String does not leak into the namespace, how is that possible?

Conceptually, the objects these constants store when the interpreter boots are considered to be builtin classes and modules that the feature keeps in their original form, so to speak, before they got a chance to be mutated.

Simply put, the corresponding builtin constants reachable by foo.rb have the objects in their original state.

(Technically, the reference is the same, the object ID is the same, but the builtin object state is per-namespace now.)

User-level constants

And that is why the M in main.rb is not reachable from within foo.rb, because Object is "reset" to its original state in the execution context of foo.rb (the ns namespace), and that state has no M. (Remember, top-level constants are stored in Object).

So, if foo.rb defines a new module

# foo.rb
module M
end

as far as foo.rb is concerned, it just created a new constant in Object and that constant stores a brand new module object, unrelated to the module object in main.rb, stored in a constant of the same name there.

Communication cross-namespace

Now, things get interesting, main.rb can see and interact with stuff in the namespace:

# foo.rb
module Foo
  def self.m = 1
end

# main.rb
ns = Namespace.new
ns.require('foo')

ns::Foo.m # => 1

What? So there is code crossing namespaces now? I defined Foo.m in the context of a namespace, and it is being executed outside of it? What are the rules of the game here? Is the isolation gone?

I believe a good mental model for this is remote procedure call (RPC).

While the invocation of Foo.m is happening in main.rb, the code being executed, the 1, was loaded in the namespace and runs in that context.

For instance, let's imagine this slightly more interesting scenario:

# foo.rb
M = 1

module Foo
  def self.m = M
end

# main.rb
ns = Namespace.new
ns.require('foo')

M = 2

ns::Foo.m # => 1

Question is, the body of the method Foo.m has a constant reference. When we invoke it as ns::Foo.m from the main namespace, how is that M resolved? In the context of main, or in the context of ns?

Think RPC.

While we are invoking Foo.m from the main namespace, the body of the method lives in foo.rb, which was loaded under ns. Therefore, the M in main does not exist there, the only M that exists for that method is the one in ns.

Same the other way around, you can pass references from main to namespaced method calls. If they are not builtin, they will respond to their regular methods, and if those methods are called from within the namespace, they get executed in the main context, RPC-style.

This opens the door to a potential gotcha several people have pointed out: It technically allows you to pass objects coming from one version of a gem into namespaces that use a different version of that gem. We'll see how this goes, we are in the super early days anyway.

Multiple namespaces

You can spawn as many namespaces as you want. And you can spawn namespaces from within namespaces.

Namespaces do not form a hierarchy, however, it is all conceptually flat, since all of them start with the same clean slate and nothing inherited.

Use cases

Use cases revolve around the idea of running code in an isolated, controlled manner. Like, I don't want this test suite to pollute my global state, I need to run this and that code with different versions of a certain gem, that kind of thing.

However, I have the hunch that when we have this feature at the tips of our fingers, new applications will arise. Probably, even creative ones we cannot foresee right now.

Try them!

In order to try namespaces you need to fetch and compile Ruby.

Then set RUBY_NAMESPACE=1 in the environment.

Closing

There is more to namespaces than this, but hope this post helped getting the gist of it.

Great write-up! Thanks @fxn!

Will this allow us to require code like Python?

from namespace import Class, OtherClass

@wilsonsilva no, it is a different feature (I think the name may be a bit confusing).