npm-cli-api-architecture.md

• my job is to make npm the most useful tool it can be
• to me, "useful" encompasses values:
  • pragmatism > rigor
    • people don't want their tools to surprise them
  • simplicity > completeness
    • do a few things well
    • let the community fill in the cracks
  • problems > solutions
    • developers love to skip over problems
    • having a clear understanding of several different problems often suggests an approach that will cover many or most of them
    • going straight to a (technical) solution predetermines a set of constraints that may not actually make sense for the problem at hand
    • familiarity vs correctness
  • (things that make npm money > things that will never make npm money) ~[digression on the importance of keeping revenue in mind for OSS]
• in effect, maintaining an open-source product requires you to become a product manager
  • prioritization is key
  • must balance competing demands
  • the more popular the product, the more product management it requires
• let's look at a specific example: making a package manager more extensible
  • npm is a tool that sits at the heart of many developers' workflows
  • workflows tend to be particular to individual teams, and it can be counterproductive to presuppose that there's a standard workflow that works for everybody
  • so everybody wants hooks
  • npm dealt with this the first time around using a lifecycle that will run user scripts at prescribed points in the process
  • but that isn't really flexible enough, and people asked for more
• looking for solutions leads to an infinitely generic and infinitely extensible miracle pegacorn that will either never work or be completely unusable
• instead, look at the model provided by dpkg and apt
  • small set of core primitives
  • a whole bunch of APIs that can be consumed by other tooling to build higher-level tools
  • supports many different workflows by doing relatively little itself
• so let's turn this idea of an extensible workflow inside out
  • look at npm as a set of logical components that together perform its various activities
    • fetching
    • caching
    • building
    • installing
  • break them out into separate physical components (that is, packages)
    • decoupled from npm itself
    • export APIs that deal only with their own responsibility
    • require only the configuration necessary to do their particular job
    • keep them independent – one doesn't depend on another
  • voila! you've got a set of pieces you can compose together into your own tools that run your workflow
    • don't include stuff you don't need
    • don't have to follow too much of somebody else's conventions
    • don't need to convince a large number of people to accommodate your needs
  • how this is working out in practice at npm
    • npm-registry-client (fetching, needs further decoupling)
    • npm-cache (in progress)
    • npm-install?
    • npm-build?
    • npm-deploy???
  • how third-party tools do it today vs future
    • atom / apm
    • yapm
    • bowser
    • Yeoman generators