RPS-SPEC

THIS DOCUMENT MOVED TO http://chneukirchen.github.com/rps/
AND http://github.com/chneukirchen/rps

= Ruby Packaging Standard

The aim of this document is two-fold.  First, to specify a common
structure of how a Ruby package distributed as source (that is, but
not limited to, development directories, version-controlled
repositories, .tar.gz, Gems, ...) should conform to.

Second, to document common and proven ways to structure Ruby packages,
and to point out certain anti-patterns that sneaked into common use.
It is by intent not to innovate.

(See RFC 2119 for use of MUST, SHOULD, SHALL.)

== Library files

Library code MUST reside in lib/.

Libraries SHOULD use a directory as namespace, e.g. lib/foo.rb and lib/foo/**.

Libraries SHOULD NOT require code of the project that are outside of lib/.

Libraries MUST not require 'rubygems' or modify the $LOAD_PATH.

Ruby library files MUST end with .rb.

Library files SHOULD be installed with mode 0644.

== Executables

Executables MUST reside in bin/.

Ruby executables SHOULD have a shebang line using env:

  #!/usr/bin/env ruby

Executables SHOULD NOT require code of the project that are outside of lib/.

Executables SHOULD NOT require 'rubygems' or modify the $LOAD_PATH,
unless they are specifically made for doing that (e.g. package managers).

Executable files SHOULD NOT end with .rb.

Executable files SHOULD be installed with mode 0755.

== Extensions

Extensions are directories which contain a extconf.rb.

Extensions SHOULD reside in ext/.

Extensions SHOULD be buildable with "ruby extconf.rb; make".

XXX How to install, how to find out what was built.

== Data files

Data files and resources of the project belong to data/.

XXX Or data/<projectname>?

XXX Where to install to, how to find out that place from the program?

== Tests

Tests SHOULD reside in test/ or spec/.

== History

09apr2010: First initial draft.
10apr2010: Fix binary permissions.
10apr2010: Add data files.

@drbrain That naming convention fits nicely with the current practice of prefixing libraries that have FFI bindings with ffi-.

a few things

• no mention of ancillary files like README, NEWS, LICENSE ? Maybe a tiny little SHOULD clause?
• where do config files go for an executable?
• what is the preferred directory for sample code? I've seen plenty of /(ex|s)amples?/
• test data goes in test/data/, test/fixtures or just fixtures/?

But good work thanks for sharing :)

@riffraff good point on the doc files

Which readme formats are allowed? README? README.md? README.txt? GitHub seems to be pushing us towards ones that allow some markup.

And what about the doc/ folder? My convention so far has been to put hand-written documentation in doc/ and generated documentation in doc/rdoc/, but I don't love it.

GNU Conventions:

COPYING http://www.gnu.org/prep/standards/standards.html#Releases
README http://www.gnu.org/prep/standards/standards.html#index-g_t_0040file_007bREADME_007d-file-144
NEWS http://www.gnu.org/prep/standards/standards.html#NEWS-File
ChangeLog http://www.gnu.org/prep/standards/standards.html#Change-Logs

The GNU packaging conventions have been long established (http://www.gnu.org/prep/standards/standards.html). Ruby core uses many of these conventions. We should be following the packaging model of ruby core itself not gems.

Checkout ruby's own file structure http://github.com/ruby/ruby

This is very good stuff, especially because your first draft is tightly focused. While there's a ton of stuff that could be added, I encourage you to avoid spending too much time talking about meta-files (README, ChangeLog, etc): They're not nearly as important as the basic library and extension guidelines you've already hit.

@jbarnette I agree. Though READMEs are a great thing, they don't fall under the "MUST" category. Its really annoying that autoconf barks at you if these files are missing.

Regarding setup.rb:

• data/ is copied in #{PREFIX_DIR}/share (like /usr/share). Why not name it "share" then ?
• conf/ is copied in /etc. Why not name it "etc" then ?
• setup.rb multi-project layout is probably a bad idea. Package management should handle this

@zimbatm I agree about the multi-project layout thing :P

data/ is already an established convention:

>> Config.datadir("foo")
=> "/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/share/foo"

If the lib is a gem installed by rubygems, it returns the gemdir.

>> Config.datadir("rake")
=> "/Users/josh/.gem/ruby/1.8/gems/rake-0.8.7/data/rake"

@josh

Where is that defined?

@josh: good to know

I also propose the following sections:

== Metadata

The project SHOULD include it's ruby and system compile and runtime dependencies in a metadata file.

(whom format is to be defined, maybe .gemspec or something more general)

== setup.rb

The project SHOULD include setup.rb at the root if it's hierarchy

(ideally, setup.rb should just contain "require 'setup.rb'" and additional hooks)

@chneukirchen rubygems monkey patches rbconfig.rb :P

@zimbatm I don't think rubygem specific config such as "gemspec" belongs in this document.

@chneukirchen We do it in http://github.com/jbarnette/rubygems/blob/master/lib/rbconfig/datadir.rb

@zimbatm's suggestions are understandable, but I think they're feature creep. Pretty please keep this lean and pragmatic.

@jbarnette I'm curious how you think we can make "data/" and Config.datadir more standardized.

styling: s/Libraries MUST not/Libraries MUST NOT/

There's RbConfig::CONFIG['datadir'].

Please think of ways future package managers can control the datadir.

Personally, I'm ok with overriding Config.datadir.

Another option would be a fixed path relative to the lib/, but I think that's messy.

I’d like to see a one-liner clarification mentioning that vendor/ belongs under lib/ and maybe a comment about vendoring practices in general.

I also totally agree with @drbrain about having a standard library naming convention; it still drives me up the wall whenever I see libraries that CamelCase or hyphenate their names such that figuring out how to require them involves guesswork.

Regarding this rule:

Executables SHOULD NOT require 'rubygems' or modify the $LOAD_PATH,

What's wrong with modifying $LOAD_PATH in the executable? In my executables, I use a line at the top like this:

$LOAD_PATH.unshift File.expand_path('../../lib', __FILE__)
require 'my_library'

Otherwise, I cannot rely on the executable's shebang to run the executable because I need to specify ruby -I some_path/lib/ some_path/bin/my_executable or I need to remember to set $RUBYLIB to some_path/lib.

In my mind, when you run an executable, you know which executable you're running, and which library that executable belongs to. So why not make the computer understand your intention? It should Just Work! :-)

Also, I think of executables as a top-level entry point into your library's code. Nobody requires an executable, so there's no harm done.

sunaku: the issue with modifying the load path in that way is that you're assuming the library files are in a location relative to your executable. When your program is installed by a package manager like apt/dpkg, Rip, or setup.rb, the lib directory may be in an entirely different place. When installed with Rubygems, that relative path location is correct but rubygems installs wrapper executables that setup the load path for you, so you're just duplicating that effort.

All that being said, modifying the load path in the way you've described is mostly harmless in my estimation. Worst case is you end up with a duplicate or invalid load path entry. It's also nice to have executables work automatically when developing out of a checkout / working copy. I prefer not modifying the load path in executables, or using a slightly more stringent set of checks to make sure I'm modifying it correctly.

I think the SHOULD NOT language is the right thing in this case. From RFC 2119:

SHOULD NOT This phrase, or the phrase "NOT RECOMMENDED" mean that there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behavior described with this label.

The key is understanding how the code you gave as an example will act in various scenarios and then making a good judgement based on that understanding and the specific case.

The alteration of the $LOAD_PATH in order to exercise a project's command-line utilities seems like it's a problem that needs a better solution. If only there was something like bundle exec but for enabling lib/ or even ext/?

See ruby-wrapper at http://github.com/chneukirchen/rup

@postmodern, @sunaku: This is a problem that's very easily solved with aliases. It's not completely transparent, but the disadvantages are outweighed by the fact that your executables aren't making any assumptions about the relative directory structure. When I'm developing, I'm generally executing scripts from the root of the project directory, so I use something like this:

$ alias rb='ruby -rubygems -Ilib:test'
$ rb bin/johnson

It's also pretty darn easy to make a project-specific alias (using absolute paths) for any given executable as well.

@jbarnette I have this in my env

export RUBYOPT="-Ilib:test"

or

export RUBYOPT="-rubygems -Ilib:test"

@josh Absolutely, the only reason I don't have that turned on all the time is because I'm dumb and it's deeply confused me a couple of times when I've been in the root directory of a project that contributes, say, RubyGems plugins via the load path. :)

@jbarnette Aliases seem like a good compromise, but they don't solve the problem completely for me: I like to use the development versions of my projects right from my source area (e.g. I run ~/src/my_project/bin/my_project directly from anywhere in the file system).

I had to write a little wrapper script (could also be a shell function if you prefer) to address this:

> cat ~/bin/rubin
#!/bin/sh -x
dir="$(dirname "$(dirname "$1")")"
ruby -rubygems -I "$dir/lib:$dir/test" "$@"

And then run it this way:

rubin ~/src/my_project/bin/my_project

I guess I can live with this. :-/

@chneukirchen Your ruby-wrapper script is awesome! Totally solves my problem. Thank you.

I've only ever used RubyGems for packaging my projects, so I'm kind of scared of setup.rb's style of separating my project's directories (i.e. lib/ is no longer next to bin/ and so on). In particular, I am confused about the following:

I want to run the man executable (pointing it to my project's man/ directory) from my project's executable in bin/. With RubyGems, I can rely on bin/ being next to man/. But with setup.rb I don't know how to find my project's man/ directory from my project's executable in bin/!

Any suggestions? Thanks.

Similar to josh and jbarnette, I use this rbdev shell function to activate bin, lib, and ext dirs when I'm working on a project:

# develop out of the current directory.
rbdev() {
    local useful dir
    test -d ./bin && {
        useful=yes
        PATH="$(pwd)/bin:$PATH"
        echo "./bin on \$PATH"
    }
    test -d ./ext && {
        useful=yes
        for dir in ./ext/*
        do
            test -d "$dir" || continue
            RUBYLIB="$(pwd)/$dir:$RUBYLIB"
            export RUBYLIB
            echo "$dir on \$RUBYLIB"
        done
    }
    test -d ./lib && {
        useful=yes
        RUBYLIB="$(pwd)/lib:$RUBYLIB"
        export RUBYLIB
        echo "./lib on \$RUBYLIB"
    }
    test -z "$useful" && {
        echo "no ./lib or ./bin detected. help me help you."
        return 1
    }
}

When I'm working on e.g., RDiscount, I do something like:

$ cd ~/git/rdiscount
$ rbdev
./bin on $PATH
./ext on $RUBYLIB
./lib on $RUBYLIB

Not setting up the load path from executables has been a problem, though. People assume everything is broken when they get a LoadError + backtrace after checking out a project for the first time and running an executable. I really wish these techniques for setting up development environments would become common practice. I'm thinking a template HACKING file that explained basic environment prep would probably go a long way.

@chneukirchen Would you mind moving this discussion to ruby-talk? It's hard to quote and inline-reply to the various tangents here. And it's easy for questions to get lost in the noise. Also, I'm sure the vast number of Rubyists on ruby-talk would have valuable feedback on this spec. Thanks.

Please discuss now at:

• http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/361459
• ruby-talk:361459
• http://groups.google.com/group/comp.lang.ruby/browse_thread/thread/ab8b8725494fb492
• <[email protected]>