This experiment is deprecated. Go see block adverbs instead.
This document is an experiment, exploring
pipe adverbs as a potential way to allow developers
to extend the Hack pipe operator |> in JavaScript.
Pipe adverbs probably would not be formally proposed until we gain more real-world experience with the pipe operator in the ecosystem. But it is worth looking toward the future.
inputExpr |> bodyExpr
inputExpr |@adverbExpr> bodyExpr
inputExpr |@adverbExpr:> bodyExpr
inputExpr |@adverbExpr: topicIdent> bodyExpr
-
inputExpr -
The pipe’s input expression, which will be evaluated first.
-
adverbExpr -
The pipe’s adverb expression, which has the same syntax as decorator expressions: a variable chained with property access (
.only, no[]) and calls(). To use an arbitrary expression as a decorator,|@(expression)>is an escape hatch. -
topicIdent -
An optional new topic variable to which the topic value will be bound, in addition to
%. -
bodyExpr -
The pipe’s body expression, which will be passed as a callback argument to the pipe’s adverb function.
The Hack-pipe operator |> is still right-associative
and has the same precedence as =>.
With this proposal, the operator |> would optionally contain,
between the | and >,
an adverb expression, which must evaluate to
a reference to an adverb function (or null,
which is the same as omitting the adverb).
An adverb is any function that –
when given (input, body) arguments,
where input is some type of value
and body is a callback function –
will somehow apply the body to the input
(or to something created by input).
When the adverb calls the body,
the argument that the body receives
becomes the body’s topic value
(to which any % in the body will refer).
The entire pipe expression
inputExpr |@adverbExpr> bodyExpr will evaluate
to the result of adverb(input, body),
where input is the result to which inputExpr evaluates,
and where body is a function enclosing the bodyExpr,
with the topic reference % bound to body’s argument.
For example, x |@adverb> % + 1
would be roughly equivalent to
adverb(x, topic => topic |> % + 1).
In fact, the body function that the operator passes to the adverb
is a new kind of function: a transparent function.
Transparent functions can only be created by |>.
When they are called, they evaluate their body
within its outer function context,
which affects the await and yield operators.
(this and super are also not affected by transparent functions,
as with arrow functions.)
For example, async () => x |@adverb> await f(%)
would be roughly equivalent to
async () => adverb(x, topic => topic |> await f(%)),
except that the await will evaluate
within the context of the async arrow function.
In addition, a colon : then a new variable identifier may optionally follow the adverb,
such as value in |@each: value> or in |@(A.each ?? B.each): value>.
Like %, this variable is bound to the topic value,
except that it is in scope for any following expression in the same pipeline.
Unlike %, the variable does not have to be used somewhere in the pipe body.
If a colon is included after the adverb
but without a new topic variable identifier,
then the pipe body does not have to use %;
the developer is opting out
of the topic-is-not-used early error.
Examples of useful adverbs include:
| Adverb | input |
adverb(input, body) |
Topic value (i.e., body’s argument) |
Result of body(topic) |
|---|---|---|---|---|
null |
Any value | Returns body(input) |
input |
Any value |
maybe |
Nullish values versus any other value | Returns body(input) – unless input is nullish, in which case returns input |
input but only when it is not nullish |
Any value |
each |
Iterable value | Returns a new iterator, which applies body to each value yielded by input, and which yields the resulting new values |
The values yielded by input |
An iterable object, whose values will be yielded by the new iterator |
asyncEach |
Async-iterable value | Returns a new async generator, which applies body to each value yielded by input, and which yields the resulting new values |
The values yielded by input |
An iterable or async-iterable object, whose values will be yielded by the new iterator |
awaiting |
Promise or other thenable value | Returns another thenable, which will resolve to body(await input) – or which will reject if input or body(await input) throws an error |
The value to which input resolves |
Any value |
cat |
Parser-rule function | Returns another parser-rule function that concatenates input and body(successfulInputMatch.value), whenever it is called with a str |
input(str).value |
Another parser-rule that will be called with input(str).remainder |
An Adverb built-in global object would have several static methods
with some of these adverbs:
globalThis.Adverb = {
maybe (input, body) {
return input != null ? body(input) : input;
},
each (input, body) {
function generate * () {
for (const value of input)
yield * body(value) ?? [];
}
return generate();
},
asyncEach (input, body) {
async function asyncGenerate * () {
for (const value of input)
yield * body(value) ?? [];
}
return generate();
},
awaiting (input, body) {
return input.then(body);
},
};The pipe operator |> allows JavaScript developers
to untangle deeply nested expressions
into linear pipelines (i.e., pipe expressions),
which may often be more humanly readable and editable.
Pipe adverbs would allow developers to extend the pipe operator. A pipe operator augmented with an adverb could automatically check its input for special values; conditionally short-circuit the pipeline; implicitly extract values from the input, bind them to a variable, and then rewrap them in a container object; and tacitly transfer ancillary state information.
Pipe adverbs could flatten many kinds of callback hell.
Adverbs would be to nested callbacks
as await would be to promise.then.
However, long pipelines (and flows of data in general) may show
recurring, repetitive patterns
that mix essential details of the program
together with boilerplate logic
at every step of these pipelines.
This boilerplate logic might include
checking for null, failures, or other special values,
extracting values from containers such as iterators,
or passing state data.
For example, the pipeline in processMaybeNumber
transforms an input (which may be a number or a nullish value),
but the first two steps of this pipeline
wrap the essential logic % + 1 and %.toString(16)
with the repetitive boilerplate % != null ? … : %.
Such repetitive handling of special values is common.
// `input` may be a number or a nullish value.
// Returns a JSON string or the nullish value.
function processMaybeNumber (input) {
return input
|> (% != null ? % + 1 : %)
|> (% != null ? %.toString(16) : %)
|> (% != null ? JSON.stringify(%) : %);
}
// This is `'3c'`:
processMaybeNumber(59);
// This is `'null'`:
processMaybeNumber(null);With a helper function maybe, we could rewrite this
less repetitively as:
// `input` may be a number or a nullish value.
// Returns a JSON string or the nullish value.
function processMaybeNumber (input) {
return maybe(input, x =>
return x + 1
|> %.toString(16)
|> JSON.stringify(%));
}
function maybe(input, callback) {
if (input != null)
return callback(input);
else
return input;
}…but this breaks up our linear pipeline into a small pyramid of nested callbacks, compromising its readability and editability. If we ever need to add more steps that require checking for nullish inputs, then each of those steps would require a callback.
Likewise, the pipeline in processItemsA
transforms an input iterable value
but wraps much of the essential logic
(like value + 1 and value > 1)
with repetitive callback boilerplate.
Likewise, the iterator-helper functions map, flatMap, and filter
perform repetitive extraction of values from input iterators
(i.e., their for loops) before processing
and then repackaging the values into output iterators.
// `input` may be an iterable value.
// Returns a new iterator.
function * map (input, callback) {
for (const value of input)
yield callback(value);
}
// `input` may be an iterable value.
// Returns a new iterator.
function * flatMap (input, callback) {
for (const value of input)
yield * callback(value);
}
// `input` may be an iterable value.
// Returns a new iterator.
function * filter (input, callback) {
for (const value of input)
yield * callback(value) ? [ value ] : [];
}
// `input` may be an iterable value.
// Returns a new iterator.
function processItemsA (input) {
return input
|> map(%, value => value + 1)
|> filter(%, value => value > 1)
|> flatMap(%, value => [ value * 2, value * 3 ]);
}
function * generate () {
yield * [ 0, 1, 2, 3, 4, 5 ];
}
// This is `[ 4, 6, 6, 9, 8, 12, 18 ]`:
[ ...processItemsA(generate()) ];The next example is similar to the previous example – it must use the same iterator-helper functions to repetitively extract values from input iterators and to repackage processed values back into output iterators. And, in this case, because a downstream processing step depends on the values extracted from multiple preceding pipeline steps, the pipeline must be split up into nested callbacks, one for each pipeline step.
// `input` may be an iterable value.
// Returns an iterator of iterators.
function processItemsB (input) {
return input |> flatMap(%, arr =>
arr |> flatMap(%, number =>
[ arr.length * number ]));
}
function * generate () {
yield [ 0, 1, 2 ];
yield [ 3, 4 ];
}
// This is `[ 0, 3, 6, 6, 8 ]`:
[ ...processItemsB(generate()) ];This problem of repetitive extraction/repackaging becomes all the more starker the longer a flow of data becomes, the more complicated the data extraction becomes, and the more dependent processing steps become on upstream steps.
Consider this function that combines several simple parser rules.
(The either, $zero, $int, $fracPart, etc. functions used here
are defined in an accompanying document.)
// This rule matches a JSON number.
// Its match’s value is its numerical value.
const $number =
$optSign |> cat(%, sign =>
either($zero, $int) |> cat(%, intPart, =>
opt($fracPart) |> cat(%, fracPart =>
opt($expPart) |> cat(%, expPart =>
meaning(
intPart + fracPart
|> % ** expPart
|> sign * %)))));This function – and the other parser rules that it uses –
are just functions that accept input strings
and which return match objects that look like { value, remainder }.
value can be any value representing the “meaning” of the match,
but remainder must be a substring of the input string.
Also, when value is not nullish, then the match is considered to be successful;
otherwise when value is nullish, then the match is considered to have failed.
These parser rules form a data flow made of sequential steps:
first applying $optSign,
then either($zero, $int),
then $opt($expPart),
and lastly meaning(sign * (intPart + fracPart) ** expPart).
The value and remainder from each consecutive rule’s match
must flow into each consecutive parser rule.
There is a lot of boilerplate involved
in correctly extracting values and remainders from each step.
The cat function hides this boilerplate:
cat combines an input rule with a callback that returns a rule.
The callback receives the value of the input rule’s match.
Theoretically, this sequential data flow should be able
to become a linear pipeline of sequential steps.
In spite of this, a deeply nested callback structure results from our need
to refer to the values of multiple previous steps
before combining them in the final step,
meaning(sign * (intPart + fracPart) ** expPart).
The longer the data flow becomes,
the deeper the pyramid of callbacks becomes.
These deeply nested callbacks are difficult to read and difficult to edit –
in a way much analogous to the callback hell
of asynchronous JavaScript before await syntax.
In each processing step from the previous examples, the same operations need to be repetitively applied to input data before the important data inside can be processed. Likewise, other operations need to be repetitively applied after the processing, before the output data from that step is returned.
A flow of data might need to repeatedly testing the input data of each step for nullish values, errors, failure objects, or other special types of value, before performing the important details on that input data.
A flow of data might need to also repeatedly extract data from the input data of each step, when that input data are iterators, promises, futures, state functions, or other container types that wrap the important data. Furthermore, after performing the important details on the important values, each step of the pipeline may need to rewrap the data back into a new instance of that container type.
This repetitive boilerplate can be extracted into helper functions
(such as map/filter/flatMap for iterators and cat for parser functions).
However, oftentimes these helper functions require deeply nested callbacks
when any step of the data flow is dependent on multiple previous
In a flow of data, many steps of the flow might need to refer
to the results of previous steps.
When this occurs, linear flows of data must become deeply nested callbacks:
the “callback hell” of pre-await asynchronous JavaScript
persists in these other contexts.
| Regular pipes only | With pipe adverbs |
|---|---|
// `input` may be a number or a nullish value.
// Returns a JSON string.
function processMaybeNumber (input) {
return maybe(input, x =>
x
|> % + 1
|> %.toString(16));
}
// This is `'3c'`:
processMaybeNumber(59);
// This is `'null'`:
processMaybeNumber(null); |
// `input` may be a number or a nullish value.
// Returns a JSON string or the nullish value.
function processMaybeNumber (input) {
return input
|@maybe> % + 1
|> %.toString(16)
|> JSON.stringify(%);
}
// This is `'3c'`:
processMaybeNumber(59);
// This is `'null'`:
processMaybeNumber(null);The first |
// `input` may be an iterable value.
// Returns a new iterator.
function processItemsA (input) {
return input
|> map(%, value => value + 1)
|> filter(%, value => value > 1)
|> flatMap(%, value =>
[ value * 2, value * 3 ]);
} |
// `input` may be an iterable value.
// Returns a new iterator.
function processItemsA (input) {
return input
|@each> [ % + 1 ]
|@each> % > 1 ? [ % ] : []
|@each> [ % * 2, % * 3 ];
}With |
// `input` may be an iterable value.
// Returns an iterator of iterators.
function processItemsB (input) {
return input |> flatMap(%, arr =>
arr |> flatMap(%, number =>
[ number * arr.length ]));
} |
// `input` may be an iterable value.
// Returns an iterator of iterators.
function processItemsB (input) {
return input |@each: arr>
arr |@each>
[ % * arr.length ];
}A downstream pipe step here
( |
// This rule matches a JSON number.
// Its match’s value is its numerical value.
const $number =
$optSign |> cat(%, sign =>
either($zero, $int) |> cat(%, intPart, =>
opt($fracPart) |> cat(%, fracPart =>
opt($expPart) |> cat(%, expPart =>
meaning(
intPart + fracPart
|> % ** expPart
|> sign * %))))); |
// This rule matches a JSON number.
// Its match’s value is its numerical value.
const $number =
$optSign |@cat: sign>
either($zero, $int) |@cat: intPart>
opt($fracPart) |@cat: fracPart>
opt($expPart) |@cat: expPart>
meaning(
intPart + fracPart
|> % ** expPart
|> sign * %);The difference is particularly dramatic
with these concatenated parser rules.
When |
In the abstract, adverbs actually act like duck-typed monads
(or more specifically the bind operation of various monadic types),
arranged in a syntax similar to Haskell’s do comprehension syntax.
Monads are a recurring pattern from category theory that appear in many functional programming languages, and which savvy developers may combine in many ways. This essay tries to avoid dwelling on monads’ abstract theory, in favor of focusing on concrete examples of adverbs. Nevertheless, the pattern is useful to simplify real-world code, by separating pipelines’ essential details from boilerplate logic (such as checking for failures or explicitly passing state data) at every step of each pipeline.
Each pipe adverb takes two arguments – a pipe input and a pipe body callback –
and somehow converts the input into a topic value,
before calling body(topic) (or not, if the adverb’s conditions are not met)
and returning the result as its output.
This is analogous to both the bind operation (>>= in Haskell)
and the unit operation (return in Haskell) of monads.
The bind operation has a type signature 𝑀𝑥 → (𝑥 → 𝑀𝑦) → 𝑀𝑦;
with function adverb (input, body),
𝑀𝑥 corresponds to the pipe input,
𝑥 → 𝑀𝑦 corresponds to the body callback,
𝑥 is the topic value that the adverb feeds to body,
and 𝑀𝑦 is the output of body (and of the adverb).
Many existing systems with monads are statically typed and use static type dispatch on a single polymorphic monad bind operator and a static “monadic type”. In contrast, much of JavaScript is dynamically typed and generally duck typed. Types are often only informally extended ad hoc, using duck-typed functions. We have therefore adapted the monadic operations into duck-typed functions that extend the pipe operator. This duck typing requires specifying the monadic bind function with each step, rather than relying on an implicit polymorphic method call on a data type.
For example, rather than relying on a Maybe data type,
pipe adverbs use a maybe function
that covers all data types but treats nullish values specially.
This design allows pipe adverbs to be mixed with regular pipe operations in the same pipeline. It also allows multiple adverbs to be applied to the same data types, even in the same pipeline.