Giter VIP home page Giter VIP logo

proposal-pipeline-operator's Introduction

Pipe Operator (|>) for JavaScript

(This document uses % as the placeholder token for the topic reference. This will almost certainly not be the final choice; see the token bikeshedding discussion for details.)

Why a pipe operator

In the State of JS 2020 survey, the fourth top answer to “What do you feel is currently missing from JavaScript?” was a pipe operator. Why?

When we perform consecutive operations (e.g., function calls) on a value in JavaScript, there are currently two fundamental styles:

  • passing the value as an argument to the operation (nesting the operations if there are multiple operations),
  • or calling the function as a method on the value (chaining more method calls if there are multiple methods).

That is, three(two(one(value))) versus value.one().two().three(). However, these styles differ much in readability, fluency, and applicability.

Deep nesting is hard to read

The first style, nesting, is generally applicable – it works for any sequence of operations: function calls, arithmetic, array/object literals, await and yield, etc.

However, nesting is difficult to read when it becomes deep: the flow of execution moves right to left, rather than the left-to-right reading of normal code. If there are multiple arguments at some levels, reading even bounces back and forth: our eyes must jump left to find a function name, and then they must jump right to find additional arguments. Additionally, editing the code afterwards can be fraught: we must find the correct place to insert new arguments among many nested parentheses.

Real-world example

Consider this real-world code from React.

console.log(
  chalk.dim(
    `$ ${Object.keys(envars)
      .map(envar =>
        `${envar}=${envars[envar]}`)
      .join(' ')
    }`,
    'node',
    args.join(' ')));

This real-world code is made of deeply nested expressions. In order to read its flow of data, a human’s eyes must first:

  1. Find the initial data (the innermost expression, envars).

  2. And then scan back and forth repeatedly from inside out for each data transformation, each one either an easily missed prefix operator on the left or a suffix operators on the right:

    1. Object.keys() (left side),
    2. .map() (right side),
    3. .join() (right side),
    4. A template literal (both sides),
    5. chalk.dim() (left side), then
    6. console.log() (left side).

As a result of deeply nesting many expressions (some of which use prefix operators, some of which use postfix operators, and some of which use circumfix operators), we must check both left and right sides to find the head of each expression.

Method chaining is limited

The second style, method chaining, is only usable if the value has the functions designated as methods for its class. This limits its applicability. But when it applies, thanks to its postfix structure, it is generally more usable and easier to read and write. Code execution flows left to right. Deeply nested expressions are untangled. All arguments for a function call are grouped with the function’s name. And editing the code later to insert or delete more method calls is trivial, since we would just have to put our cursor in one spot, then start typing or deleting one contiguous run of characters.

Indeed, the benefits of method chaining are so attractive that some popular libraries contort their code structure specifically to allow more method chaining. The most prominent example is jQuery, which still remains the most popular JS library in the world. jQuery’s core design is a single über-object with dozens of methods on it, all of which return the same object type so that we can continue chaining. There is even a name for this style of programming: fluent interfaces.

Unfortunately, for all of its fluency, method chaining alone cannot accommodate JavaScript’s other syntaxes: function calls, arithmetic, array/object literals, await and yield, etc. In this way, method chaining remains limited in its applicability.

Pipe operators combine both worlds

The pipe operator attempts to marry the convenience and ease of method chaining with the wide applicability of expression nesting.

The general structure of all the pipe operators is value |> e1 |> e2 |> e3, where e1, e2, e3 are all expressions that take consecutive values as their parameters. The |> operator then does some degree of magic to “pipe” value from the lefthand side into the righthand side.

Real-world example, continued

Continuing this deeply nested real-world code from React:

console.log(
  chalk.dim(
    `$ ${Object.keys(envars)
      .map(envar =>
        `${envar}=${envars[envar]}`)
      .join(' ')
    }`,
    'node',
    args.join(' ')));

…we can untangle it as such using a pipe operator and a placeholder token (%) standing in for the previous operation’s value:

Object.keys(envars)
  .map(envar => `${envar}=${envars[envar]}`)
  .join(' ')
  |> `$ ${%}`
  |> chalk.dim(%, 'node', args.join(' '))
  |> console.log(%);

Now, the human reader can rapidly find the initial data (what had been the most innermost expression, envars), then linearly read, from left to right, each transformation on the data.

Temporary variables are often tedious

One could argue that using temporary variables should be the only way to untangle deeply nested code. Explicitly naming every step’s variable causes something similar to method chaining to happen, with similar benefits to reading and writing code.

Real-world example, continued

For example, using our previous modified real-world example from React:

Object.keys(envars)
  .map(envar => `${envar}=${envars[envar]}`)
  .join(' ')
  |> `$ ${%}`
  |> chalk.dim(%, 'node', args.join(' '))
  |> console.log(%);

…a version using temporary variables would look like this:

const envarString = Object.keys(envars)
  .map(envar => `${envar}=${envars[envar]}`)
  .join(' ');
const consoleText = `$ ${envarString}`;
const coloredConsoleText = chalk.dim(consoleText, 'node', args.join(' '));
console.log(coloredConsoleText);

But there are reasons why we encounter deeply nested expressions in each other’s code all the time in the real world, rather than lines of temporary variables. And there are reasons why the method-chain-based fluent interfaces of jQuery, Mocha, and so on are still popular.

It is often simply too tedious and wordy to write code with a long sequence of temporary, single-use variables. It is arguably even tedious and visually noisy for a human to read, too.

If naming is one of the most difficult tasks in programming, then programmers will inevitably avoid naming variables when they perceive their benefit to be relatively small.

Reusing temporary variables is prone to unexpected mutation

One could argue that using a single mutable variable with a short name would reduce the wordiness of temporary variables, achieving similar results as with the pipe operator.

Real-world example, continued

For example, our previous modified real-world example from React could be re-written like this:

let _;
_ = Object.keys(envars)
  .map(envar => `${envar}=${envars[envar]}`)
  .join(' ');
_ = `$ ${_}`;
_ = chalk.dim(_, 'node', args.join(' '));
_ = console.log(_);

But code like this is not common in real-world code. One reason for this is that mutable variables can change unexpectedly, causing silent bugs that are hard to find. For example, the variable might be accidentally referenced in a closure. Or it might be mistakenly reassigned within an expression.

Example code
// setup
function one () { return 1; }
function double (x) { return x * 2; }

let _;
_ = one(); // _ is now 1.
_ = double(_); // _ is now 2.
_ = Promise.resolve().then(() =>
  // This does *not* print 2!
  // It prints 1, because `_` is reassigned downstream.
  console.log(_));

// _ becomes 1 before the promise callback.
_ = one(_);

This issue would not happen with the pipe operator. The topic token cannot be reassigned, and code outside of each step cannot change its binding.

let _;
_ = one()
  |> double(%)
  |> Promise.resolve().then(() =>
    // This prints 2, as intended.
    console.log(%));

_ = one();

For this reason, code with mutable variables is also harder to read. To determine what the variable represents at any given point, you must to search the entire preceding scope for places where it is reassigned.

The topic reference of a pipeline, on the other hand, has a limited lexical scope, and its binding is immutable within its scope. It cannot be accidentally reassigned, and it can be safely used in closures.

Although the topic value also changes with each pipeline step, we only scan the previous step of the pipeline to make sense of it, leading to code that is easier to read.

Temporary variables must be declared in statements

Another benefit of the pipe operator over sequences of assignment statements (whether with mutable or with immutable temporary variables) is that they are expressions.

Pipe expressions are expressions that can be directly returned, assigned to a variable, or used in contexts such as JSX expressions.

Using temporary variables, on the other hand, requires sequences of statements.

Examples
Pipelines Temporary Variables
const envVarFormat = vars =>
  Object.keys(vars)
    .map(var => `${var}=${vars[var]}`)
    .join(' ')
    |> chalk.dim(%, 'node', args.join(' '));
const envVarFormat = (vars) => {
  let _ = Object.keys(vars);
  _ = _.map(var => `${var}=${vars[var]}`);
  _ = _.join(' ');
  return chalk.dim(_, 'node', args.join(' '));
}
// This example uses JSX.
return (
  <ul>
    {
      values
        |> Object.keys(%)
        |> [...Array.from(new Set(%))]
        |> %.map(envar => (
          <li onClick={
            () => doStuff(values)
          }>{envar}</li>
        ))
    }
  </ul>
);
// This example uses JSX.
let _ = values;
_= Object.keys(_);
_= [...Array.from(new Set(_))];
_= _.map(envar => (
  <li onClick={
    () => doStuff(values)
  }>{envar}</li>
));
return (
  <ul>{_}</ul>
);

Why the Hack pipe operator

There were two competing proposals for the pipe operator: Hack pipes and F# pipes. (Before that, there was a third proposal for a “smart mix” of the first two proposals, but it has been withdrawn, since its syntax is strictly a superset of one of the proposals’.)

The two pipe proposals just differ slightly on what the “magic” is, when we spell our code when using |>.

Both proposals reuse existing language concepts: Hack pipes are based on the concept of the expression, while F# pipes are based on the concept of the unary function.

Piping expressions and piping unary functions correspondingly have small and nearly symmetrical trade-offs.

This proposal: Hack pipes

In the Hack language’s pipe syntax, the righthand side of the pipe is an expression containing a special placeholder, which is evaluated with the placeholder bound to the result of evaluating the lefthand side's expression. That is, we write value |> one(%) |> two(%) |> three(%) to pipe value through the three functions.

Pro: The righthand side can be any expression, and the placeholder can go anywhere any normal variable identifier could go, so we can pipe to any code we want without any special rules:

  • value |> foo(%) for unary function calls,
  • value |> foo(1, %) for n-ary function calls,
  • value |> %.foo() for method calls,
  • value |> % + 1 for arithmetic,
  • value |> [%, 0] for array literals,
  • value |> {foo: %} for object literals,
  • value |> `${%}` for template literals,
  • value |> new Foo(%) for constructing objects,
  • value |> await % for awaiting promises,
  • value |> (yield %) for yielding generator values,
  • value |> import(%) for calling function-like keywords,
  • etc.

Con: Piping through unary functions is slightly more verbose with Hack pipes than with F# pipes. This includes unary functions that were created by function-currying libraries like Ramda, as well as unary arrow functions that perform complex destructuring on their arguments: Hack pipes would be slightly more verbose with an explicit function call suffix (%).

(Complex destructuring of the topic value will be easier when do expressions progress, as you will then be able to do variable assignment/destructuring inside of a pipe body.)

Alternative proposal: F# pipes

In the F# language’s pipe syntax, the righthand side of the pipe is an expression that must evaluate into a unary function, which is then tacitly called with the lefthand side’s value as its sole argument. That is, we write value |> one |> two |> three to pipe value through the three functions. left |> right becomes right(left). This is called tacit programming or point-free style.

Real-world example, continued

For example, using our previous modified real-world example from React:

Object.keys(envars)
  .map(envar => `${envar}=${envars[envar]}`)
  .join(' ')
  |> `$ ${%}`
  |> chalk.dim(%, 'node', args.join(' '))
  |> console.log(%);

…a version using F# pipes instead of Hack pipes would look like this:

Object.keys(envars)
  .map(envar => `${envar}=${envars[envar]}`)
  .join(' ')
  |> x=> `$ ${x}`
  |> x=> chalk.dim(x, 'node', args.join(' '))
  |> console.log;

Pro: The restriction that the righthand side must resolve to a unary function lets us write very terse pipes when the operation we want to perform is a unary function call:

  • value |> foo for unary function calls.

This includes unary functions that were created by function-currying libraries like Ramda, as well as unary arrow functions that perform complex destructuring on their arguments: F# pipes would be slightly less verbose with an implicit function call (no (%)).

Con: The restriction means that any operations that are performed by other syntax must be made slightly more verbose by wrapping the operation in a unary arrow function:

  • value |> x=> x.foo() for method calls,
  • value |> x=> x + 1 for arithmetic,
  • value |> x=> [x, 0] for array literals,
  • value |> x=> ({foo: x}) for object literals,
  • value |> x=> `${x}` for template literals,
  • value |> x=> new Foo(x) for constructing objects,
  • value |> x=> import(x) for calling function-like keywords,
  • etc.

Even calling named functions requires wrapping when we need to pass more than one argument:

  • value |> x=> foo(1, x) for n-ary function calls.

Con: The await and yield operations are scoped to their containing function, and thus cannot be handled by unary functions alone. If we want to integrate them into a pipe expression, await and yield must be handled as special syntax cases:

  • value |> await for awaiting promises, and
  • value |> yield for yielding generator values.

Hack pipes favor more common expressions

Both Hack pipes and F# pipes respectively impose a small syntax tax on different expressions:
Hack pipes slightly tax only unary function calls, and
F# pipes slightly tax all expressions except unary function calls.

In both proposals, the syntax tax per taxed expression is small (both (%) and x=> are only three characters). However, the tax is multiplied by the prevalence of its respectively taxed expressions. It therefore might make sense to impose a tax on whichever expressions are less common and to optimize in favor of whichever expressions are more common.

Unary function calls are in general less common than all expressions except unary functions. In particular, method calling and n-ary function calling will always be popular; in general frequency, unary function calling is equal to or exceeded by those two cases alone – let alone by other ubiquitous syntaxes such as array literals, object literals, and arithmetic operations. This explainer contains several real-world examples of this difference in prevalence.

Furthermore, several other proposed new syntaxes, such as extension calling, do expressions, and record/tuple literals, will also likely become pervasive in the future. Likewise, arithmetic operations would also become even more common if TC39 standardizes operator overloading. Untangling these future syntaxes’ expressions would be more fluent with Hack pipes compared to F# pipes.

Hack pipes might be simpler to use

The syntax tax of Hack pipes on unary function calls (i.e., the (%) to invoke the righthand side’s unary function) is not a special case: it simply is explicitly writing ordinary code, in the way we normally would without a pipe.

On the other hand, F# pipes require us to distinguish between “code that resolves to an unary function” versus “any other expression” – and to remember to add the arrow-function wrapper around the latter case.

For example, with Hack pipes, value |> someFunction + 1 is invalid syntax and will fail early. There is no need to recognize that someFunction + 1 will not evaluate into a unary function. But with F# pipes, value |> someFunction + 1 is still valid syntax – it’ll just fail late at runtime, because someFunction + 1 isn’t callable.

TC39 has rejected F# pipes multiple times

The pipe champion group has presented F# pipes for Stage 2 to TC39 twice. It was unsuccessful in advancing to Stage 2 both times. Both F# pipes (and partial function application (PFA)) have run into strong pushback from multiple other TC39 representatives due to various concerns. These have included:

This pushback has occurred from outside the pipe champion group. See HISTORY.md for more information.

It is the pipe champion group’s belief that any pipe operator is better than none, in order to easily linearize deeply nested expressions without resorting to named variables. Many members of the champion group believe that Hack pipes are slightly better than F# pipes, and some members of the champion group believe that F# pipes are slightly better than Hack pipes. But everyone in the champion group agrees that F# pipes have met with far too much resistance to be able to pass TC39 in the foreseeable future.

To emphasize, it is likely that an attempt to switch from Hack pipes back to F# pipes will result in TC39 never agreeing to any pipes at all. PFA syntax is similarly facing an uphill battle in TC39 (see HISTORY.md). Many members of the pipe champion group think this is unfortunate, and they are willing to fight again later for an F#-pipe split mix and PFA syntax. But there are quite a few representatives (including browser-engine implementers) outside of the Pipe Champion Group who are generally against encouraging tacit programming (and PFA syntax), regardless of Hack pipes.

Description

(A formal draft specification is available.)

The topic reference % is a nullary operator. It acts as a placeholder for a topic value, and it is lexically scoped and immutable.

% is not a final choice

(The precise token for the topic reference is not final. % could instead be ^, or many other tokens. We plan to bikeshed what actual token to use before advancing to Stage 3. However, % seems to be the least syntactically problematic, and it also resembles the placeholders of printf format strings and Clojure’s #(%) function literals.)

The pipe operator |> is an infix operator that forms a pipe expression (also called a pipeline). It evaluates its lefthand side (the pipe head or pipe input), immutably binds the resulting value (the topic value) to the topic reference, then evaluates its righthand side (the pipe body) with that binding. The resulting value of the righthand side becomes the whole pipe expression’s final value (the pipe output).

The pipe operator’s precedence is the same as:

  • the function arrow =>;
  • the assignment operators =, +=, etc.;
  • the generator operators yield and yield *;

It is tighter than only the comma operator ,.
It is looser than all other operators.

For example, v => v |> % == null |> foo(%, 0)
would group into v => (v |> (% == null) |> foo(%, 0)),
which in turn is equivalent to v => foo(v == null, 0).

A pipe body must use its topic value at least once. For example, value |> foo + 1 is invalid syntax, because its body does not contain a topic reference. This design is because omission of the topic reference from a pipe expression’s body is almost certainly an accidental programmer error.

Likewise, a topic reference must be contained in a pipe body. Using a topic reference outside of a pipe body is also invalid syntax.

To prevent confusing grouping, it is invalid syntax to use other operators that have similar precedence (i.e., the arrow =>, the ternary conditional operator ? :, the assignment operators, and the yield operator) as a pipe head or body. When using |> with these operators, we must use parentheses to explicitly indicate what grouping is correct. For example, a |> b ? % : c |> %.d is invalid syntax; it should be corrected to either a |> (b ? % : c) |> %.d or a |> (b ? % : c |> %.d).

Lastly, topic bindings inside dynamically compiled code (e.g., with eval or new Function) cannot be used outside of that code. For example, v |> eval('% + 1') will throw a syntax error when the eval expression is evaluated at runtime.

There are no other special rules.

A natural result of these rules is that, if we need to interpose a side effect in the middle of a chain of pipe expressions, without modifying the data being piped through, then we could use a comma expression, such as with value |> (sideEffect(), %). As usual, the comma expression will evaluate to its righthand side %, essentially passing through the topic value without modifying it. This is especially useful for quick debugging: value |> (console.log(%), %).

Real-world examples

The only changes to the original examples were dedentation and removal of comments.

From jquery/build/tasks/sourceMap.js:

// Status quo
var minLoc = Object.keys( grunt.config( "uglify.all.files" ) )[ 0 ];

// With pipes
var minLoc = grunt.config('uglify.all.files') |> Object.keys(%)[0];

From node/deps/npm/lib/unpublish.js:

// Status quo
const json = await npmFetch.json(npa(pkgs[0]).escapedName, opts);

// With pipes
const json = pkgs[0] |> npa(%).escapedName |> await npmFetch.json(%, opts);

From underscore.js:

// Status quo
return filter(obj, negate(cb(predicate)), context);

// With pipes
return cb(predicate) |> _.negate(%) |> _.filter(obj, %, context);

From ramda.js.

// Status quo
return xf['@@transducer/result'](obj[methodName](bind(xf['@@transducer/step'], xf), acc));

// With pipes
return xf
  |> bind(%['@@transducer/step'], %)
  |> obj[methodName](%, acc)
  |> xf['@@transducer/result'](%);

From ramda.js.

// Status quo
try {
  return tryer.apply(this, arguments);
} catch (e) {
  return catcher.apply(this, _concat([e], arguments));
}

// With pipes: Note the visual parallelism between the two clauses.
try {
  return arguments
    |> tryer.apply(this, %);
} catch (e) {
  return arguments
    |> _concat([e], %)
    |> catcher.apply(this, %);
}

From express/lib/response.js.

// Status quo
return this.set('Link', link + Object.keys(links).map(function(rel){
  return '<' + links[rel] + '>; rel="' + rel + '"';
}).join(', '));

// With pipes
return links
  |> Object.keys(%).map(function (rel) {
    return '<' + links[rel] + '>; rel="' + rel + '"';
  })
  |> link + %.join(', ')
  |> this.set('Link', %);

From react/scripts/jest/jest-cli.js.

// Status quo
console.log(
  chalk.dim(
    `$ ${Object.keys(envars)
      .map(envar => `${envar}=${envars[envar]}`)
      .join(' ')}`,
    'node',
    args.join(' ')
  )
);

// With pipes
Object.keys(envars)
  .map(envar => `${envar}=${envars[envar]}`)
  .join(' ')
  |> `$ ${%}`
  |> chalk.dim(%, 'node', args.join(' '))
  |> console.log(%);

From ramda.js.

// Status quo
return _reduce(xf(typeof fn === 'function' ? _xwrap(fn) : fn), acc, list);

// With pipes
return fn
  |> (typeof % === 'function' ? _xwrap(%) : %)
  |> xf(%)
  |> _reduce(%, acc, list);

From jquery/src/core/init.js.

// Status quo
jQuery.merge( this, jQuery.parseHTML(
  match[ 1 ],
  context && context.nodeType ? context.ownerDocument || context : document,
  true
) );

// With pipes
context
  |> (% && %.nodeType ? %.ownerDocument || % : document)
  |> jQuery.parseHTML(match[1], %, true)
  |> jQuery.merge(%);

Relationships with other proposals

Function helpers

Hack pipes can and would coexist with the Function helpers proposal, including its pipe and flow functions. These simple (and commonly downloaded) convenience functions manipulate unary functions without extra syntax.

TC39 has rejected the F# pipe operator twice. Given this reality, TC39 is considerably more likely to pass pipe and flow helper functions than a similar syntactic operator.

Standardized pipe and flow convenience functions may also obviate some of the need for a F#-pipe infix operator. (They would not preclude standardizing an equivalent operator later. For example, TC39 standardized binary ** even when Math.pow existed.)

Partial-function-application syntax

Hack pipes can coexist with a syntax for partial function application (PFA). There are two approaches with which they may coexist.

The first approach is with an eagerly evaluated PFA syntax, which has already been proposed in proposal-partial-application. This eager PFA syntax would add an …~(…) operator. The operator’s right-hand side would be a list of arguments, each of which is an ordinary expression or a ? placeholder. Each consecutive ? placeholder would represent another parameter.

Ordinary expressions would be evaluated before the function is created. For example, f~(g(), ?, h(), ?) would evaluate f, then g(), then h(), and then it would create a partially applied version of f with two arguments.

An optional number after ? placeholder would override the parameter’s position. For example, f~(?1, ?0) would have two parameters but would switch them when calling f.

The second approach is with a lazily evaluated syntax. This could be handled with an extension to Hack pipes, with a syntax further inspired by Clojure’s #(%1 %2) function literals. It would do so by combining the Hack pipe |> with the arrow function => into a pipe-function operator +>, which would use the same general rules as |>.

+> would be a prefix operator that creates a new function, which in turn binds its argument(s) to topic references. Non-unary functions would be created by including topic references with numbers (%0, %1, %2, etc.) or .... %0 (equivalent to plain %) would be bound to the zeroth argument, %1 would be bound to the next argument, and so on. %... would be bound to an array of rest arguments. And just as with |>, +> would require its body to contain at least one topic reference in order to be syntactically valid.

Eager PFA Pipe functions
a.map(f~(?, 0)) a.map(+> f(%, 0))
a.map(f~(?, ?, 0)) a.map(+> f(%0, %1, 0))
a.map(x=> x + 1) a.map(+> % + 1)
a.map(x=> x + x) a.map(+> % + %)
a.map(x=> f(x, x)) a.map(+> f(%, %))

In contrast to the eagerly evaluated PFA syntax, topic functions would lazily evaluate its arguments, just like how an arrow function would.

For example, +> f(g(), %0, h(), %1) would evaluate f, and then it would create an arrow function that closes over g and h. The created function would not evaluate g() or h() until the every time the created function is called.

No matter the approach taken, Hack pipes could coexist with PFA.

Eventual sending / pipelining

Despite sharing the word “pipe” in their name, the pipe operator and the eventual-send proposal’s remote-object pipelines are orthogonal and independent. They can coexist and even work together.

const fileP = E(
  E(target).openDirectory(dirName)
).openFile(fileName);

const fileP = target
|> E(%).openDirectory(dirName)
|> E(%).openFile(fileName);

Possible future extensions

Hack-pipe syntax for if, catch, and forof

Many if, catch, and for statements could become pithier if they gained “pipe syntax” that bound the topic reference.

if () |> would bind its condition value to %,
catch |> would bind its caught error to %,
and for (of) |> would consecutively bind each of its iterator’s values to %.

Status quo Hack-pipe statement syntax
const c = f(); if (c) g(c); if (f()) |> g(%);
catch (e) f(e); catch |> f(%);
for (const v of f()) g(v); for (f()) |> g(%);

Optional Hack pipes

A short-circuiting optional-pipe operator |?> could also be useful, much in the way ?. is useful for optional method calls.

For example, value |> (% == null ? % : await foo(%) |> (% == null ? % : % + 1))
would be equivalent to value |?> await foo(%) |?> % + 1.

Tacit unary function application syntax

Syntax for tacit unary function application – that is, the F# pipe operator – has been rejected twice by TC39. However, they could still eventually be added to the language in two ways.

First, it can be added as a convenience function Function.pipe. This is what the function-helpers proposal proposes. Function.pipe may obviate much of the need for an F#-pipe operator, while still not closing off the possibility of an F#-pipe operator.

Secondly, it can be added as another pipe operator |>> – similarly to how Clojure has multiple pipe macros ->, ->>, and as->.
For example, value |> % + 1 |>> f |> g(%, 0)
would mean value |> % + 1 |> f(%) |> g(%, 0).

There was an informal proposal for such a split mix of two pipe operators, which was set aside in favor of single-operator proposals. This split mix might return as a proposal after Hack pipes.

proposal-pipeline-operator's People

Contributors

advaith1 avatar alhadis avatar brian-gates avatar btoo avatar dead-claudia avatar felixfbecker avatar framemuse avatar fxfactorial avatar gilbert avatar hraban avatar js-choi avatar k644606347 avatar kerrick avatar kidonng avatar kittygiraudel avatar linbudu599 avatar littledan avatar loreanvictor avatar maadhattah avatar macabeus avatar mariozig avatar mfunkie avatar mjethani avatar nem035 avatar pokute avatar pygy avatar tabatkins avatar tehshrike avatar thenavigateur avatar vp2177 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

proposal-pipeline-operator's Issues

What high level improvement does this operator add to the language?

At a high level I'm failing to see this do either of two things that make a useful improvement to the language:

Reduce errors or not introduce any more chance for them while improving functionality
Save typing

It fails on both of these counts because eventually those pipelines only become useful in functional composition in the end otherwise they're just a line of code so in libraries a developer would see a ton of the following in any semi advanced program:

const pipe1 = val => val |> fn1 |> fn2 // 24 characters
const pipe2 = val => val |> pipe1 |> fn3 // 26 characters
const pipe3 = val => val |> pipe2 |> fn4 // 26 characters

const pipe1 = v=>v|>fn1|>fn2 // 16 characters, 8 characters saved by uglification

which would much rather be done with composition to save typing and errors so instead the library would have

const pipe1 = o(fn2,fn1) // 10 characters, 0 characters saved by uglification, 60% less characters
const pipe2 =  o(fn3,pipe1) // 12 characters
const pipe3 = o(fn7,o(fn6,fn5)) // 17 characters

Composition doesn't have the chance to misspell val because it doesn't need to spell it in the first place. Less letters, less errors. Less repetition, less error.

Read order composition is something that can be done now with 1 line of code for composition, and can extend to an in order pipe with 2 more lines of code, or done directly in 1 line of code:

// compose 2 functions
const o = (fn2,fn1) => (...args) => fn2(fn1(...args))
// flip a two Arity function
const flipA2 = (fn) => (a,b) => fn(b,a)
// pipe
const p = flipA2(o)

// single line pipe
const p = (fn2, fn1) => (...args) => fn2(fn1(...args))

//split a string
const split = c => s => s.split(c)
//join an array
const join = c => as => as.join(c)

const tabsToCSV = p(split('\t'), join(','))

tabsToCSV('test\tlist')

After removing comments and white space that's 251 characters to change a Tab separated value list to a CSV in a reusable and extendable way with the longer method. In fact someone can now easily define list changes and extend functionality easily. After 251 character being able to go from CSV to SSV(;) in const CSV2SSV = p(split(','),join(';')) of 39 characters means there is little room for error. And from tab separated to SSV is just const TSV2SSV = p(tabsToCSV,CSV2SSV) of just 37 characters more with inconsistent naming conventions. To be fair this is limited to every following function being a single parameter, but that's an EXTREMELY small price to pay for this sort of agility and the same price paid with the suggested operator. With the pipeline operator developers are even more limited in that they can only do a SINGLE argument to the first function unless they want a weird footprint of const pipe1 = (...args) => fn1(...args) |> fn2. In fact in terms of o they're the same length

const o=(a,b)=>(...args)=>b(a(...args)) // 39 characters uglified, pipe is the exact same length
const o=(a,b)=>(...args)=>a(...args)|>b // 39 characters uglified, reverse order is impossible

which causes it to fail the Save Typing test as well.

As is evident with proper composition the proposed pipeline operator and actual composition end up being the same, and at a high level it has introduced nothing to the language and only added complexity, confusion, and more room for errors with developing developers. With types returns(Promise, Functor, Array) it gets even worse

Pipeline with spread operator?

As mentioned in #23 (comment), this proposal will be more powerful when it supports spread operator .... In short, make something like the following

...[100, 200] |> Math.max

equivalent to the following.

Math.max(...[100, 200]) //=> 200

This change will fantastically make the pipeline able to pass more than one argument to the callee.

I suspect whether this would be legal when it chained more than one times. Obviously the following is valid,

...(...[10, 20] |> ((a, b) => [a + 5, b + 5])) |> console.log

but it would be nice to have something like the following

...[10, 20] |> ...((a, b) => [a + 5, b + 5]) |> console.log

and it's equivalent to the following.

console.log(...(((a, b) => [a + 5, b + 5])(...[10, 20]))) // prints '15 25'

Note that, since the spread operator is not a general unary operator and something like foo = ...bar; is not permitted syntactically, I think this kind of extension will result in a syntax extension of spread operator itself.

Use a promise chain instead?

You can simply return a promise from each of such functions, and then write the following:

let result = await doubleSay('hello').then(capitalize).then(exclaim);

or:

let result = doubleSay('hello').then(capitalize).then(exclaim).catch(ops);
// + resolution with an error handler

This creates even better code, IMO, because:

  • syntax .catch for the error handler is nicer/shorter than try{}catch(e){}
  • the chain logic is while just as obvious and short, it is already a standard, with regards to the callback function passed into each .then call.
  • This approach is compatible with ES6 Generators
  • Now this is a big one: No extra effort for asynchronous code, as you get it by default.

I think the existing ES7 is already verbose enough, and not lacking anything, except ahem, better support for classes, closer to C++.


As a bonus, you can simplify it even further, by implementing a function that takes a random value + a list of functions to do .then for each of them.

There are already implementations like this in some promise libraries, so you can do:

let result = await Promise.chain('hello', [doubleSay, capitalize, exclaim]);

This also lets you mix together synchronous and asynchronous code, which is priceless.

Promise sample: no, not significant

You say about the last example that

If we wanted to add catchError-like functionality using ES6 and stay fluent, we would have to either extend Promise.prototype, or re-implement the Promise interface (as bluebird and others have done).

But it's not true at all, in reality the exact same goal can be achieved in the same way with standard API:

fetchPlayers()
  .then( players => players.filter( p => p.score > 100 ).map( fetchGames ) )
  .then( games => Promise.all(games) )
  .then( processGames )
  .catch( catchError( ProcessError, err => [] ) )
  .then( forEach(g => console.log(g)) );

function catchError (ErrorClass, handler) {
  return function catcher (error) {
    if (error instanceof ErrorClass) return handler(error);
    else throw error;
  }
}

function forEach (fn) {  return array => array.forEach(fn);  }

It's even shorter and simpler this way. I really like this operator, but this example is clearly not a "significant" one ;)

Composition operator

Pipeline operator is good, but I thinks composition operator is more expected by the functional paradigm.

Suppose we define * as the composition operator, then

f(g(h(x))) 

could be written as

 f*g*h*x

Further more, an issue of proposal-decorators and react could be solved gracefully by this operator.

e.g. for react hoc composition:

const Comp = compose(
  hoc1,
  hoc2,
  ...
)((props) => <div>...</div>)

it could be written as:

const Comp =
ho1*
hoc2*
hoc3*
...
(props) => <div>...</div>

babel-plugin-transform-function-composition

I have written a Babel plugin that enables predominantly equivalent functionality to that described in this proposal.

https://github.com/gajus/babel-plugin-transform-function-composition

However, unlike the pipeline operator, babel-plugin-transform-function-composition enables function composition of functions with multiple parameters. For the most part, this implementation captures the requirements of the https://github.com/mindeavor/es-papp proposal too.

First class support for Promises

The pipeline operator could work like an infix version of R.pipeP, allowing asynchronous pipelines to have (nearly?) the same syntax as synchronous ones:

fs.readFile('./index.txt')
|> .split('\n')
    .map(fs.readFile)
|> Promise.all
|> .join("\n")
|> process.stdout.write
|> .catch(process.stderr.write)

New, optimized version of Pipeline Operator

After much mulling and consideration, I think I have managed to combine everyone's ideas to create a version of the pipeline operator that is optimal for JavaScript. In short, the rule is:

  • Pipe the left-hand side of the operator into the right-hand side as the last argument.

In other words, x |> f(10, 20) is equivalent to f(10, 20, x).

Why not the first argument like Elixir? A couple reasons:

  • When I read x |> f(10), I expect 10 to be the first argument to f. Elixir's style changes this to f(x, 10).
  • Piping to the last argument plays well with partial-application-friendly functions. This allows code such as let g = f.bind(null, 10); x |> g(20)

Syntactically, this also lets us code really useful flows that still solve the prototype extension problem. Check out the full README of the new proposal version for details.

I would love everyone's feedback on this, both negative and positive. Thanks for reading.

P.S. The pull request for this version is here.

Babel plugin

I have tried to use your plugin but I got this error:

ReferenceError: Unknown plugin "transform-pipeline-operator" specified in "/Users/gromm/tmp/babel-pipeline-app/.babelrc" at 0, attempted to resolve relative to "/Users/gromm/tmp/babel-pipeline-app"
    at /Users/gromm/.nvm/versions/node/v5.3.0/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/options/option-manager.js:191:17
    at Array.map (native)
    at Function.normalisePlugins (/Users/gromm/.nvm/versions/node/v5.3.0/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/options/option-manager.js:167:20)
    at OptionManager.mergeOptions (/Users/gromm/.nvm/versions/node/v5.3.0/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/options/option-manager.js:289:36)
    at OptionManager.addConfig (/Users/gromm/.nvm/versions/node/v5.3.0/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/options/option-manager.js:219:10)
    at OptionManager.findConfigs (/Users/gromm/.nvm/versions/node/v5.3.0/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/options/option-manager.js:425:16)
    at OptionManager.init (/Users/gromm/.nvm/versions/node/v5.3.0/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/options/option-manager.js:473:12)
    at File.initOptions (/Users/gromm/.nvm/versions/node/v5.3.0/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/index.js:206:75)
    at new File (/Users/gromm/.nvm/versions/node/v5.3.0/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/index.js:124:22)
    at Pipeline.transform (/Users/gromm/.nvm/versions/node/v5.3.0/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/pipeline.js:44:16)

Consistent Piping Behavior

Coming from an Elixir, I would expect this operator to always "pipe" the previous function's return value to the current function's first parameter.

In fact, it's more consistent this way, even within the examples provided.

For example, all the pipes to double are already assuming that the previous value is double's first/only param. This assumption is the case for all (currently proposed) single-param functions. It should, naturally, extend to functions of all parity.

The pipe operator is excellent and (I assume) designed for functional programming. If the cost of using it is "repetitive" parameters, then that's a fantastic price to pay!

function doubleSay (str) {
  return str + ", " + str;
}
function capitalize (str) {
  return str[0].toUpperCase() + str.substring(1);
}
function exclaim (str) {
  return str + '!';
}

let result = "hello"
  |> doubleSay // assumed 1st
  |> capitalize // assumed 1st
  |> exclaim; // assumed 1st

Single parity: Expected, nice, consistent

Current Validation example:

function bounded (prop, min, max) {
  return function (obj) {
    if ( obj[prop] < min || obj[prop] > max ) throw Error('out of bounds');
    return obj;
  };
}
function format (prop, regex) {
  return function (obj) {
    if ( ! regex.test(obj[prop]) ) throw Error('invalid format');
    return obj;
  };
}

function createPerson (attrs) {
  attrs
    |> bounded('age', 1, 100)
    |> format('name', /^[a-z]$/i)
    |> Person.insertIntoDatabase;
}

Instantiating new functions per call
Auto-pass to first param is gone
Different behavior entirely from single-parities

Proposed Alternative

function bounded (obj, prop, min, max) {
  if ( obj[prop] < min || obj[prop] > max ) throw Error('out of bounds');
  return obj;
}
function format (obj, prop, regex) {
  if ( ! regex.test(obj[prop]) ) throw Error('invalid format');
  return obj;
}

function createPerson (attrs) {
  attrs
    |> bounded('age', 1, 100)
    |> format('name', /^[a-z]$/i)
    |> Person.insertIntoDatabase;
}

Function mutators instantiated only once
Consistent behavior across all parity
Usage remains the same

I can provide more examples if needed, or try to explain myself further if I did a poor job. Just let me know.

Thanks!

async function example

// Assume fs.readFile is an `async` function
async function runTask () {
  './index.txt'
    |> await fs.readFile
    |> file => file
       .split('\n')
       .map(fs.readFile)
    |> await Promise.all
    |> all => all.join("\n")
    |> console.log
}

Is a really bad example, it could be more easily be done without it as:

async function runTask () {
    let file = await fs.readFile('./index.txt');
    let joined = await Promise.all(file.split("\n").map(fs.readFile));
    console.log(joined.join("\n"));
}

Constant stack usage

The current proposal desugars

"hello"
  |> doubleSay
  |> capitalize
  |> exclaim;

to

exclaim(capitalize(doubleSay("hello")));

If I want to reuse this pipeline I can write

const sayItAgainLouder = s => s |> doubleSay |> capitalize |> exclaim;

Now I get reuse, but I'm allocating three stack frames. This isn't a big deal unless I want to compose sayItAgainLouder with some other functions.

A simple alternative desugaring

[doubleSay, capitalize, exclaim].reduce((value, f) => f(value), "hello");

This decreases stack usage, but I imagine at the cost of speed.

It would be useful if this proposal required the compiler to maintain constant stack usage through either some sort of stream fusion or stack frame reuse.

Spread version?

Would it make sense to include spread pipeline operator as well? Applies the value instead of calls? For instance, ~>.

let add = (a, b) => a + b
let x = [1, 2, 3]
  ~> (_, b, c) => [b + c, b * c]
  ~> add
// x = (2 + 3) + (2 * 3) = 5 + 6 = 11

Babel plugin

Is it possible to have this as a Babel plugin? It's one thing to see snippets but I'd like to test it out and see how I like it in practice too.

Operator overload instead

What about instead of one fixed operator add a way to add function that map to operator that is a symbol tadded to the objects.

it will execute left hand side function for the operator. I can be something like

let pipe = Symbol('|>');
Number.prototype[pipe] = function(value, fn) {
    return fn(value);
};

And the operators can be executed from left to right like reduce.

The list of operators that are supported can be in some repository so you will need to register the symbol first to use it.

window.registerOperator(pipe);

that way a user may implement + operator for it's own objects like:

function Foo(a, b, c) {
   this.a = a;
   this.b = b;
   this.c = c;
}
Foo.prototype.toString = function() {
  return "<a:" + this.a + ", b:" + this.b + ", c:" + this.c + ">";
};
let plus = Symbol("+");
window.registerOperator(plus);
Foo.prototype[plus] = function(x, y) {
  return new Foo(x.a + y.a, x.b + y.b, x.c + y.c);
};
x = new Foo(1,2,3);
y = new Foo(2,3,4);
z = x + y;
console.log(z);
//> <a:3, b:5, c:7>

Details of draft specification

In PR #51 , I wrote some spec text. I'd be interested in feedback on the details here. In particular:

  • The right argument of |> is treated directly as a function; even if it is a |CallExpression|, the result of evaluating that |CallExpression| is treated as the function, rather than slotting the left argument into the arguments list. That is x |> y(z) is similar in semantics to y(z)(x). (see #4)
  • The precedence of |> sits between the ternary operator and ||. It is left-associative. (see #23)
  • All calls to eval through pipeline are indirect eval.
  • The pipeline operator is eligible for proper tail calls, when appropriate.
  • The left argument to |> is evaluated before the right one.
  • If the right argument to |> has a receiver, it is used for the call, with rules similar to a method call. That is x |> y.z gets y as a receiver, just as if it were written y.z(x).

Any thoughts?

Note to positive or negative reacts: tell me why in this thread!

Are there no better alternatives to `|>` character combination

Is |> the best character combination available for the operator? | - the ideal - is already taken by bitwise operators, but I worry that with the proposed syntax it's too easy to miss a | or > and still have syntactically valid code, leading to hard to trace bugs e.g.

var isValid = score
  |> isNumeric
  > isGreaterThanZero

Would generally give false every time.

Have any other character combinations been considered?

(Note: I've read #4, but it mostly concerns syntax for multi-parametered functions)

function composition syntax

I am loving this pipeline proposal. In the future I wonder if there is also space to allow for combining this with the partial application proposition for function composition

For example instead of this:

let add_mult = (_) => _ |> add |> mult

Do this:

let add_mult = ? |> add |> mult

Pipeline + arrow function grammar

In the explained and babel implementation, cases like x |> y => z are permitted. The current grammar does not permit these, throwing a syntax error. It seems like a useful case, but I haven't figured out how to spec it.

Polyfill with Lodash/Underscore

I've been using a modified version of thru with underscore/lodash chaining to use this in production. I've been planning on making a separate Babel plugin (https://github.com/AdamBrodzinski/pipes-js) but would love to collaborate if you're planning on making one instead.

I really don't have time to extract this but I have it running on a sideproject, RedScript an Elixir like lang that compiles to JS.
https://github.com/AdamBrodzinski/RedScript

Essentially you just need to do a bit of regex work to replace the pipes into this:

var res = '555 121-7878'
  |> stripChars '.'
  |> stripChars ' '
  |> stripChars('-')
// str - string to change
// char - the character to strip out
// returns transformed string
function stripChars(str, char) {
   return str.replace(char, ' ');
}

var res = _.chain('555 121-7878')
  .pipesCall(stripChars, '.')
  .pipesCall(stripChars, ' ')
  .pipesCall(stripChars, '-').value();

console.log(res);

The ES6 version of pipesCall is:

function pipesCall(firstParam, func, ...restParams) {
  return func(firstParam, ...restParams);
}

I also tried to get lo-dash to add this but he wasn't interested.

Functions with multiple arguments

Currently

The spec suggests function with multiple arguments be used like this:

Base functions:

function double(x) {
  return x * 2;
}

function add(x, y) {
  return x + y;
}

Pipe operator usage:

const score = 10;

const newScore = score
  |> double
  |> _ => add(7, _)

console.log(newScore); // 27;

Proposed change

Instead of writing extra function wrappers for passing arguments, the operator can always make sure the output from previous function as fed as the first argument of next function.

const newScore = score
  |> double // output of `score` is first argument: `double(10)`
  |> add(7) // output of `double(10)` is first argument: `add(20, 7)`

console.log(newScore); // 27

This is how Elixir works: https://hexdocs.pm/elixir/Kernel.html#%7C%3E/2


If TC39 is interested in this, I am happy to send a PR for improving this spec further.

Function calls that expect more than one argument?

This question came up when I saw the following example from the slides:

async function runTask () {
  './index.txt'
    |> await fs.readFile
    |> _ => _.split('\n')
    |> _ => _.map(fs.readFile)
    |> await Promise.all
    |> _ => _.join("\n")
    |> console.log
}

Let's remove the parts that aren't relevant to my question:

async function runTask () {
  './index.txt'
    |> await fs.readFile
    |> console.log
}

Which approximately translates to:

const fs = require("fs");

async function runTask () {
  // './index.txt'
  //   |> await fs.readFile
  //   |> console.log

  console.log(await fs.readFile("./index.txt"));
}
runTask()

Having the following result:

$ node pipeline-functions-expect-more-than-one-argument.js
(node:8877) [DEP0013] DeprecationWarning: Calling an asynchronous function without callback is deprecated.

Given that this particular example actually stands up as an example of all async fs.* functions, I'm wondering how that will affect the usefulness of |>? I'm going to spend more time exploring this issue and will report back here as I find anything else of interest.

Changing the function composition operator link

Edit: Added links.
Edit 2: Here's a comment clarifying some of the high points of the below thread.

This thread pretty much ruined any and all hopes I had in that particular proposal, where the OP doesn't seem particularly receptive to the criticism he's receiving of his proposal (and I'm not quite convinced he has much command of the subject matter).

Would it be wise at this point to change the function composition operator link to point to another proposal? (Shameless plug: I have my own separate proposal, but mine isn't the only alternative - there's others that are effectively the same thing, including function-based and method-based variants.)

Backwards pipe operator?

How about a backwards pipe operator as well?

const squareThenDouble = x => double(square(x))

// or use forward pipe
const squareThenDouble = x => x  |> square |> double

// or use backward pipe
const squareThenDouble = x => double <| square <| x

yield / await

For checking, since all information I found in other issues are more than 18 months old, about the yield and await keywords:

The following functions:

function* g() {
"input" |> yield foo |> bar;
}
async function a() {
  "input" |> await foo |> bar;
}

Would be equivalent to:

function* g() {
  const input = "input";
  const f = yield foo;
  bar( f( input ) );
}
async function a() {
  const input = "input";
  const f = await foo;
  bar( f( input ) );
}

And there would be no easy way to do the following using the pipeline operator:

function* g() {
  bar(yield foo("input"));
}
async function a() {
  bar(await foo("input"));
}

What is this for?

As I stated in another thread: #23 (comment)

Can play devils advocate and ask based purely on all of this discussion is it really worth being added to the language (The discussion indicates developer cognitive load when being used)?
As much as I syntactic sugar can be useful there isn't much this adds that a simple function could. This would add an extra function call in the expression though.

Some explanation here:
https://gist.github.com/jonathanKingston/4df71289a2cd8dd8306a

And also: #23 (comment)

The proposal isn't invalidated by having functional equivalents in current Javascript.

I agree totally as functionally most new features can be written in JavaScript pretty simply with the exception of features like Workers and Crypto.

But this is all besides the point — the proposal is about developer ergonomics, isn't it?

I agree also, I'm worried however that we end up with more operators used that languages like Perl which are often criticised in having far too many operators making the language harder to read like lengthy RegEx.
I'm also aware of the argument "if you don't want it - don't use it", I'm not sure how much that helps personally.
I would much rather browsers concentrate on fixing current hard issues like for example RegEx character classes which requires large libraries to use: http://xregexp.com/plugins/

Again sorry for playing devils advocate etc 😅

Can we carry on this discussion here?

Alternative pipeline strategies and operators

Bringing over a summary of some of the insane amounts of discussion that occurred in the bind operator proposal's repo regarding function pipelining, since it's far more relevant here than there now (since this is a TC39 repo now dedicated to the topic), and I think we could use a historical record and starting point for this. Apologies for the long length.


Pipeline strategies

There are three primary strategies you could use to pipeline function calls:

(For expository purposes, I'm using @ for all three variants to make them less visibly different while still making it seem like a fitting operator. Note that I'm not actually proposing it, since it conflicts with decorators.)

  1. foo@bar(...)bar.call(foo, ...)

    This was what was initially proposed in the bind operator proposal. It is fairly object-oriented in that it views callees more like extension methods than just functions, and calls them accordingly – the context (foo in this case) is passed as this, and the arguments as you would expect.

  2. foo@bar(...)bar(foo, ...)

    This was proposed by @gajus in this particular comment. It is more procedural, and can be viewed as threading a value through several functions much like Clojure's thread-first macro.

  3. foo@bar(...)bar(...)(foo)

    This was proposed by @gilbert in this repo. It is more functional in nature, since it involves piping through a series of single-arg functions, usually returned from calling another function (in this case, from bar(...)).

Pros and Cons

Of course, these are not all equal, and they each have their strengths and weaknesses. None of these are magic silver bullets, and they all have their shortcomings. I did go into some detail on this previously, but I thought I'd relay it here, too.

  1. foo@bar(...)bar.call(foo, ...)

    Pros:

    • It plays well with the philosophy that it's an extension method of sorts. (This could also be considered a con by some.)
    • It lends itself well to borrowing generic native prototype methods like Array.prototype.join and Object.prototype.toString.
    • It reduces the need for func.call, func.apply, and Reflect.apply.

    Cons:

    • Existing third-party library support is fairly low. (This could change depending on if this variant is accepted.)
    • The heavily object-oriented appearance is likely to be contentious, for similar reasons to why ES6 classes have met some sharp criticism.
    • It naturally begets a binding variant like foo@bar being equivalent to bar.bind(foo). This kind of behavior is much more contentious, and feel free to explore the bind operator proposal's issues to see this in action.
    • Due to its reliance on this, it's impossible to define such utilities using arrow functions, and inline utilities are pretty verbose (which could be rectified):
      foo()
      @function () { return this + 1 }()
      @bar()
  2. foo@bar(...)bar(foo, ...)

    Pros:

    • It lends itself well to concise helpers defined via arrow functions.
    • It avoids most of the issues with this, including much of the contention.
    • It encourages function-based APIs without becoming functional enough to be contentious.
    • It works well with many existing libraries such as Three.js, Lodash, and the like. (This is a pretty weak plus, since libraries can adapt to any of these.)

    Cons:

    • Use of native prototype methods have to be wrapped somehow. (This is a pretty weak minus, since var dethisify = Function.bind.bind(Function.call) can trivially wrap them.)
    • It looks somewhat object-oriented even though it avoids this. (This could turn off some functional programming fans.)
    • It may surprise newcomers already familiar with object-oriented languages, due to it not using this.
    • Defining inline helpers is slightly awkward:
      foo()
      @(x => x + 1)()
      @bar()
  3. foo@bar(...)bar(...)(foo)

    Pros:

    • It lends itself well to functional idioms. (This could be considered a con to some.)
    • It avoids this and related problems entirely.
    • It allows easy use of inline helpers:
      foo()
      @(x => x + 1)
      @bar()

    Cons:

    • The functional nature of it is likely to be contentious.
    • Existing library support exists, but is limited. (This could change depending on if this variant is selected.)
    • It requires currying for calls to not depend on this specific form for ease of use. (This is a similar issue to the this dependence of the first option.)
    • It's substantially harder for engines to optimize, since it enforces an observable closure allocation at least, and currying is hard to optimize without native support. (Engines can adapt for the former, and a language-level curry proposal would solve the latter.)

    In addition, this could be simulated by using a native composition operator/method and immediately calling it with the value to pipe through, although it would cause the functions to all create their closures before any of them could be evaluated. (This does make optimization a bit more difficult without inlining first.)

Alternate operators

As expected, there have been numerous other operators already suggested for signaling pipelining. Here's a list of many of them:

  • :: – proposed with the original bind proposal
  • ~> – proposed in this issue
  • -> – proposed in this comment
  • |> – proposed originally in this repo
  • .. – proposed in this issue
  • & – proposed in this issue, although it's obviously a no-go

I've noted that ~> is a bit awkward to type, and a few others pointed out that .. runs the risk of confusion with spread, a potential future range syntax. The remaining 3 (::, ->, and |>) are likely more viable, although I'll point out that |> is itself a little awkward to type in my personal experience (small hands 👐 😄).

Expand Promise example

Sorry, I'm not following at all how promise ends up available to catchError() and then(). If all of that example code exists under the aegis of a larger object implementation, it would be clearer to the reader if he or she could see that.

It's also not completely clear to me how this is better than just capturing the exception via an ordinary Promise#catch and vetting the exception type inside the function. If it isn't what you want, just re-throw from there. No extension to the Promise prototype would be required. What am I missing here?

The `|>` operator would be great for function composition instead. Explanation...

The existing code

const doubleThenSquare = value=>square(double(value))

would be rewritable as:

const doubleThenSquare = double |> square

Reasons to prefer this instead of |> being a function call operator:

  1. Genuinely saves code
  2. The tersely composed functions can be called any time and be tersely used to compose other functions
  3. Has logical consistency (FunctionExpression |> FunctionExpression instead of InputExpression |> FunctionExpression |> FunctionExpression)

Fuller explanation here:

https://github.com/TheNavigateur/proposal-pipeline-operator-for-function-composition
https://esdiscuss.org/topic/native-function-composition

Operator Precedence - Higher or Lowest?

At first I imagined the pipeline operator to be the lowest precedence, but now that the question has been raised, I'm wondering – should it instead have higher precedence?

obj.x || defaults.x |> process
//
// With lowest precedence:
//    process(obj.x || defaults.x)
//
// With higher precedence:
//    obj.x || process(defaults.x)
//

arg |> m.f || m.g |> h
//
// With lowest precedence:
//    h( (m.f || m.g)(arg) )
//
// With higher precedence:
//    m.f(arg) || m.g()
//

(Extended proposal?) Ghost Methods

So I got this idea while fixing the promises example in response to #10. In short, here's a truncated example of the code I was editing:

fetchPlayers()
  .then( games => Promise.all(games) )
  |> catchError( ProcessError, err => [] )
  |> then( forEach(g => console.log(g)) )

function then (handler) {  return promise => promise.then(handler)  }
function forEach (fn) {  return array => array.forEach(fn)  }

I noticed the bottom boilerplate functions had something in common – they both received a parameter, just to call that parameter on a future object. Then I thought, what if you could do this concisely?

fetchPlayers()
  .then( games => Promise.all(games) )
  |> catchError( ProcessError, err => [] )
  |> .then( .forEach(g => console.log(g)) )

// No more boilerplate functions!

In other words, .then(x) is equivalent to obj => obj.then(x).

Is this crazy? Useful? Grammatically possible?

Just had to throw it out there.

Alternate syntax

Although I wrote the proposal, I do recognize a potential objection: functions with multiple arguments.

Although to me using arrow functions is perfectly explicit and clear, for the sake of argument let's explore two possible alternatives to handling this situation.

1. Elixir-style Inlining

Elixir's pipeline operator is actually different than the original proposal you see in this repo (of which we will call F# style). Instead of simply calling the right-hand side with the left-hand side, it instead inserts the left-hand side as the first argument to the right:

// F# style
run("hello") |> withThis(10);
// is equivalent to
withThis(10)( run("hello") );

// Elixir style
run("hello") |> withThis(10);
// is equivalent to
withThis( run("hello"), 10 );

//
// More complicated example
//
// F# style
run("hello") |> withThis(10) |> andThis(20);
// is equivalent to
andThis(20)( withThis(10)( run("hello") ) );

// Elixir style
run("hello") |> withThis(10) |> andThis(20);
// is equivalent to
andThis( withThis( run("hello"), 10 ), 20 );

Pros / Cons

  • 👍 Less function calls (2 vs 3)
  • 👍 Compatible with much more of the JavaScript ecosystem
  • 👎 Gives a new semantic meaning to what looks like a normal function call. For example, withThis(10) is no longer just calling that function with one argument
  • 👎 Putting the left-hand side as the first argument to the right-hand side is pretty arbitrary.

2. Placeholder Arguments

Another alternative is to have a new syntax for "placeholder arguments" – basically, slots waiting to be filled by the next function call. For example:

// Placeholder style
run("hello") |> withThis(10, #);
// is equivalent to
withThis( 10, run("hello") );

//
// More complicated example
//
run("hello") |> withThis(10, #) |> andThis(#, 20);
// is equivalent to
andThis( withThis( 10, run("hello") ), 20 );

Pros / Cons

  • 👍 Very explicit (no surprises)
  • 👍 Less function calls
  • 👍 Compatible with even more of the JavaScript ecosystem
  • 👎 Requires more new syntax
  • 👎 Usage of the hash operator (#) would probably be hard to define outside coupled use with pipeline operator (this problem could be avoided by simply making it illegal)

Any thoughts and feedback would be greatly appreciated. Even if you are reiterating some of the points I've made, it's important to express them so we can see what the community thinks. Thanks for reading.

The Polyfill

I made a simple pipe func, it returns a function, but can be adjusted to be executed instantly.

let pipe = (...funcs) => (...args) => funcs.slice(1,funcs.length).reduce((v, f) => f(v), funcs[0](...args))

Tested here

Incorrect promise example

The example

fetchPlayers()
  .then(players => players.filter( p => p.score > 100 ).map( fetchGames ))
  |> Promise.all
  |> then(processGames)

seems to use Promise.all incorrectly. It takes an array of promises, not a promise for one. You'd have to do either of

  .then(games => Promise.all(games))
// or
  |> then(games => games |> Promise.all) // does that even use the correct this context?
// or
  .then(Promise.all.bind(Promise))
// or
  .then(Promise::all)

Struggling with the await syntax

I'm wondering what the reason is behind this syntax:

const userAge = userId |> fetchUserById |> await |> getAgeFromUser

I would expect the piping syntax to either implicitly handle awaitable code:

const userAge = userId |> fetchUserById |> getAgeFromUser

Or explicitly but with this cleaner variant:

const userAge = userId |> await fetchUserById |> getAgeFromUser

The current proposal confuses me. Can you make a comment on the reason behind it? Or on the alternatives I suggested?

Run-time impact

The expressed solution for functions with more than one required argument looks to me like a serious performance sink because of the way JavaScript handles in-lined functions at run-time (at least as I understand it):

let newScore = person.score
  |> double
  |> _ => add(7, _)
  |> _ => boundScore(0, 100, _);

While this would work reasonably well in C++, because we can think of it as a form of template meta-programming that only impacts compiler performance, but in JavaScript there ain't no such animal and this is going to hurt. My jsPerf testing appears to support the idea that arrow functions are also slower than ordinary functions (http://jsperf.com/interior-function-performance), so unless current implementations get better, I wouldn't sacrifice performance on the altar of enhanced readability (which, to me, is a mileage-may-vary concept at best). [For some reason Opera has arrow functions out-performing the others...but it's not widely used. Ditto Vivaldi.] See http://www.incaseofstairs.com/2015/06/es6-feature-performance/ for further data. I understand that this situation will not remain static and that we'll probably get better performance in the future. (But maybe we won't. So far forEach() is still a performance sink and there's been a lot of time to work on that.)

Shortcut for method call?

|>. cannot conflict with anything so we could consider |> .identifier(...) is the same as |> _ => _.identifier(...).
Same for [ by the way.

This would greatly simplify the example:

// Assume fs.readFile is an `async` function
async function runTask () {
  './index.txt'
    |> await fs.readFile
    |> .split('\n')
    |> .map(fs.readFile)
    |> await Promise.all
    |> .join("\n")
    |> console.log
}

Internally supporting thenables

So I've seen a bunch of suggestions for something like this:

let value = 'hello'
  |> sync_thing
  |> other_sync_thing
  |> await async_thing
  |> sync_thing

I'm thinking rather than supporting the ambiguous expression await in the middle of a pipe, why not instead just support thenable propagation internally and simply use an async pipeline like this:

let value = await 'hello'
  |> sync_thing
  |> other_sync_thing
  |> async_thing
  |> sync_thing

The internals of the pipeline operator could special case that when a function returns a thenable it would instead start using value.then(next) rather than next(value), thus handling otherwise sync continuation functions automatically.

It'd transform down to something like:

let value = await [
  sync_thing,
  other_sync_thing,
  async_thing,
  sync_thing
].reduce((value, fn) => {
  return value.then ? value.then(fn) : fn(value)
}, 'hello')

Pipelines with no left-hand argument? (error or composition without immediate invocation)

What should happen if an argument to the pipeline is not provided at the left-hand side of the expression?
eg:

w = |> x |> y |> z

Does it produce a syntax error, or a functional composition?
The latter could be useful in some situations, eg:

result = x.map( |> y |> z )
.catch(|>logToFile |> retry(fetchFunc, --numOfAttempts))

Of course it's not very verbose to create a composition with an arrow function:
rather than

result = x.map( i => i |> y |> z)

But this would allow us to do away with a little boiler plate in the same way that Ghost Methods #13 would.

Example seems incorrect (or confusing?)

Thanks for this proposal; seems really cool. I was confused by this example, though:

var person = { score: 75 };

var newScore = person.score
  |> double
  |> score => add(7, score)
  |> validateScore

newScore //=> 107

// As opposed to: var newScore = add(7, validateScore( double(person.score) ))

Now I don't know if there's a mistake in the example or if the proposal itself confuses me, but it seems really odd that the functions nest in a different order than they are piped. I would think that they would nest in the same order, with the first piped function as the innermost one, namely:

var newScore = validateScore(add(7, double(person.score) ));

Am I misunderstanding the example? Why do they neat in a different order for this case?

Thanks for all your great work!

Find a champion to get this proposal to stage 0

There seems to be consensus on the question of operator precedence, and I really want to see this pull request merged so that I can start using the operator!

However, on that pull request it looks like the operator needs a "TC39 champion". I'm not up on the protocol, so I'm opening this issue so that other people who have a better idea what's up can list the next steps for finding a champion and getting this proposal brought up at whatever meetings are appropriate.

Reverse pipeline?

This is regular:

"hello" |> doubleSay |> capitalize |> exclaim

How about reverse pipeline with a new operator: <|

exclaim <| capitalize <| doubleSay <| "hello"

There were previous talks about Elixir-like syntax

// Elixir style
run("hello") |> withThis(10) |> andThis(20)
// is equivalent to
andThis( withThis( run("hello"), 10 ), 20 )

👎 Gives a new semantic meaning to what looks like a normal function call. For example, withThis(10) is no longer just calling that function with one argument
👎 Putting the left-hand side as the first argument to the right-hand side is pretty arbitrary.

With reverse pipline (and additional feature of applying arguments from the right) it could be written as

run("hello") |> withThis <| 10 |> andThis <| 20

When both operators are used in an expression like this, the resverse pipe accepting arguments from the right would apply them _after_ the arguments coming from the regular pipeline from the left.

Regular pipeline takes higher precedence than reverse pipeline.

run("hello") |> withThis <| 10 |> andThis <| 20
____________    ______________
______________________________    _____________

Thoughts?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.