Issues with NPM's conceptual model

gistfile1.md

Let's say someone has implemented a module for sets that's being used in two modules.

a.js:

var set = require("set");

export.thing = set(1, 2, 3);

b.js:

var set = require("set");

export.thing = set(4, 5, 6);

You're using both:

var thing_a = require("a").thing
  , thing_b = require("b").thing

Imagine that b depends on a newer version of set which has a completely new (faster!) internal structure. This structure uses a new (public) API for computing intersects/unions between sets. This means that's it's impossible to use thing_a and thing_b together:

thing_a.intersect(thing_b) // => Err!
thing_a.union(thing_b))    // => Err!

It's easy to to incidentally creates these sort of dependencies inside your app. If a and b did use the same version of set, you might have used the code above. By upgrading either a or b there's a chance that your code might mysteriously break.

More concretely:

Every object that comes from module O, version A through module A can only be safely handled by a module X if (and only if) module X also has a dependency on module O, version A. (By version A I don't necessarily mean the exact same version, but the same version of the public API).

This turns out to be the exact same constraint as single-namespace (Ruby, Python, Perl etc.) language imposes.

Conclusion:

For modules that are only used internally in a module and never used across modules: NPM trumps. There's no way your dependencies can interact with other modules.

For modules that are used across several modules: You need to (globally) use the same module. Technically, you don't have to, but it's going to be a lot of pain and horrible for encapsulation.

@jcoglan:

That's mostly a problem with leaking implementation details, which is (or should be) separate from the package manager.

Me:

No. The problem is that two packages can say "I'm returning a set", and you need to worry about the differences. The package didn't "leak" anything. it specified clearly that it returns an object from another module. There are many cases where this is wanted (do you really want x different versions of set in your code?).

Example: jQuery. Having multiple versions of jQuery is not going to work well. Passing a DOM element from one jQuery to another is going to have subtle issues (e.g. unbinding delegated events won't work across jQuery instances). This is a case where you want your package manager to scream at you and say "nope, these two packages depends on different versions of jQuery and therefore cannot sanely work together.

I think your original gist makes a poor point, because it has nothing to do with different versions of a package in specific. It's a general problem when you assume too much ability based on identity in a dynamically typed language. Your example could be solved with duck typing.

However, the jQuery example is better, and you are on to something: in some cases being able to talk about the same instance across packages would make some things easier. But I suspect this really says something about the design (ie smells of singletons).

In any case, the NPM approach can still be used to emulate the RubyGems approach, but not the other way around.

This does make some sense -- the are concepts that appear common enough that you think it's okay to emit instances of them from your APIs. In theory a library should not allow objects other than those in the language/stdlib or that it created itself to pass through its interface, but I doubt I stick to that.

In Node, this seems to be translating into people using uniform interfaces, e.g. programming with streams, always yielding an error as the first callback argument, etc.