NOTE: this is a draft

TODO comments are in the source

This spec must be validated by implementing it in at least one library. I may do this in Haskell's optparse-applicative.

Auto-Completion for Executables and Shells Protocol (ACES Protocol)

Considering that the auto completions for any program are highly coupled with the program, many libraries and programs implement some degree of shell auto- completion logic in the command's main executable. These include Python optcomplete, Nix's completion.sh and Haskell's optparse-applicative.

Each of these solutions implements some kind of protocol between the executable and a helper script. This leads to a situation where each combination of autocompletion library × shell variant needs to reimplement much of the same logic.

By formalizing the interface between the executable and the shell, both the executables and the shells only need to implement one interface.

Process Architecture

When a shell requests completions through the ACES protocol, the following happens:

shell
-----
  |     --invokes-->
  |                     wrapper script
  |                     --------------
  |                           |       --invokes-->
  |                           |                 main executable
  |                           |                 ---------------
  |                           |                       |
  |                           |              parse ACES command line
  |                           |                       |
  |                           |              write completions in ACES format
  |                           |       <--stdout--     |
  |                         reads                   exit 0
  |                           |
  |                    return completions
  |                       to the shell
  |   <--commands,            |
  |   <--stdout, etc.         |
  |                         exit
show completions

Shells may choose to avoid the wrapper script and implement the ACES protocol directly.

ACES Command Line Options

When the executable runs, it needs to know its context. This is provided through command line arguments. This is an implementation detail that the user of the executable need not be aware of.

--aces-completion-index INDEX: this option signifies that the executable must not perform its main role, but return possible completions using ACES. It takes one base 10 integer argument (INDEX) describing the argument where completions are requested.

--aces-completion-argument ARG: this option must be repeated for each of the entered arguments.

--aces-*: any other options with the prefix --aces- must be ignored when --aces-completion-index is provided.

ACES Stdout format

The ACES Stdout format is a custom line-based text format, in order to ease shell script based implementation, as most shells do not have good support for semi-structured formats.

The executable prints lines to stdout. It should print only a carriage return. Carriage returns should be ignored when reading the format.

Each line is either an instruction or a completion. Each completion must be preceded by a %value instruction line.

An instruction line consist of %, an extension word and optionally:

a space ( ) followed by a sequence of arbitrary characters terminated by the line break.

The extension word consists of a nonempty sequence of alphanumeric characters and/or -.

Instruction lines with an unknown instruction must be ignored by the shell.

Shells and libraries may independently agree on extensions that make use of instruction lines. Such extensions should prefix the extension word with x-; e.g. %x-no-socket-highlight.

Standard instruction words:

%value: Indicates that the next line is a completion. The next line will be read as a completion, even if it starts with %.

%addspace: Indicates that the following completion forms a complete argument. The shell should add a space, to indicate that the argument is complete and to help the user enter the next argument. This is different from an actual space trailing the completion, because such a space should be escaped by the shell, in order not to serve as an argument boundary.

%files: Indicates that the following completion contains a file or directory that the shell may highlight as such. The shell may append a / if it determines that the completion is a directory.

Glossary

ACES protocol: the entire protocol, including the process architecture
ACES format: the format written to stdout
ACES command line: the flags that are used by the executable to perform completion instead of the main program.

Besides the current solutions for registering shell completion logic in a shell, with a shim to call the program with ACES flags, shells (or their support scripts) could perform the following, to solve the problem of each command's installation having to be aware of all possible shells.

Suppose the current command is mycmd and PATH resolves its location to /opt/foo/bin,

look for /opt/foo/bin/.aces/mycmd and call that for ACES instead
look for /opt/foo/bin/._aces_mycmd and call that for ACES instead

The prior has the benefit of polluting bin less.
The latter might be considered "more FHS" because everything is executable in bin
The prior should be preferred.

The invocation of the ACES command must have argv[0] matching the absolute path, such as /opt/foo/bin/.aces/mycmd.

The ACES executable lookup does not exit the example bin directory (ie dirname of the command), because doing so may be unexpected and potentially unsafe when the PATH item does not point to a typical FHS-like structure.

In practice, packages that support ACES may install a symlink pointing to the command, so that the main command can behave as what could be considered a multi-call executable, except the conditional may be based on the presence of ACES arguments as described.
Hence, both integrated and external completion logic is supported.
Whether .aces executables may be third-party is a matter of package management policy.

roberth/ACES-Protocol.md