Giter VIP home page Giter VIP logo

pipe's Introduction

Infix programming toolkit

Module enabling a sh like infix syntax (using pipes).

Introduction

As an example, here is the solution for the 2nd Euler Project exercise:

Find the sum of all the even-valued terms in Fibonacci which do not exceed four million.

Given fib a generator of Fibonacci numbers:

euler2 = (fib() | where(lambda x: x % 2 == 0)
                | take_while(lambda x: x < 4000000)
                | add)

Installing

To install the library, you can just run the following command:

# Linux/macOS
python3 -m pip install pipe

# Windows
py -3 -m pip install pipe

To install the development version, do the following:

$ git clone https://github.com/JulienPalard/Pipe
$ cd Pipe
$ python3 -m pip install .

Deprecations of pipe 1.x

In pipe 1.x a lot of functions were returning iterables and a lot other functions were returning non-iterables, causing confusion. The one returning non-iterables could only be used as the last function of a pipe expression, so they are in fact useless:

range(100) | where(lambda x: x % 2 == 0) | add

can be rewritten with no less readability as:

sum(range(100) | where(lambda x: x % 2 == 0))

so all pipes returning non-iterables are now deprecated and will be removed in pipe 2.0.

Vocabulary

  • A Pipe: a Pipe is a 'pipeable' function, something that you can pipe to, In the code '[1, 2, 3] | add' add is a Pipe
  • A Pipe function: A standard function returning a Pipe so it can be used like a normal Pipe but called like in : [1, 2, 3] | concat("#")

Syntax

I don't like import * but for the following examples in an REPL it will be OK, so:

>>> from pipe import *

The basic syntax is to use a Pipe like in a shell:

>>> sum(range(100) | select(lambda x: x ** 2) | where(lambda x: x < 100))
285

Some pipes take an argument, some do not need one:

>>> sum([1, 2, 3, 4] | where(lambda x: x % 2 == 0))
6

>>> sum([1, [2, 3], 4] | traverse)
10

A Pipe as a function is nothing more than a function returning a specialized Pipe.

Constructing your own

You can construct your pipes using Pipe class initialized with lambdas like:

stdout = Pipe(lambda x: sys.stdout.write(str(x)))
select = Pipe(lambda iterable, pred: (pred(x) for x in iterable))

Or using decorators:

@Pipe
def stdout(x):
    sys.stdout.write(str(x))

Existing Pipes in this module

Alphabetical list of available pipes; when several names are listed for a given pipe, these are aliases.

chain
    Chain a sequence of iterables:
    >>> list([[1, 2], [3, 4], [5]] | chain)
    [1, 2, 3, 4, 5]

    Warning : chain only unfold iterable containing ONLY iterables:
      [1, 2, [3]] | chain
    Gives a TypeError: chain argument #1 must support iteration
    Consider using traverse.

chain_with()
    Like itertools.chain, yields elements of the given iterable,
    then yields elements of its parameters
    >>> list((1, 2, 3) | chain_with([4, 5], [6]))
    [1, 2, 3, 4, 5, 6]

dedup()
    Deduplicate values, using the given key function if provided (or else
    the identity)

    >>> list([1, 1, 2, 2, 3, 3, 1, 2, 3] | dedup)
    [1, 2, 3]
    >>> list([1, 1, 2, 2, 3, 3, 1, 2, 3] | dedup(key=lambda n:n % 2))
    [1, 2]

filter()
    alias for where(), see where()

groupby()
    Like itertools.groupby(sorted(iterable, key = keyfunc), keyfunc)
    (1, 2, 3, 4, 5, 6, 7, 8, 9) \
            | groupby(lambda x: "Odd" if i % 2 else "Even")
            | select(lambda x: "%s : %s" % (x[0], (x[1] | concat(', '))))
            | concat(' / ')
    'Odd : 1, 3, 5, 7, 9 / Even : 2, 4, 6, 8'

islice()
    Just the itertools.islice
    >>> list((1, 2, 3, 4, 5, 6, 7, 8, 9) | islice(2, 8, 2))
    [3, 5, 7]

izip()
    Just the itertools.izip
    >>> list((1, 2, 3, 4, 5, 6, 7, 8, 9)
    ...  | izip([9, 8, 7, 6, 5, 4, 3, 2, 1]))
    [(1, 9), (2, 8), (3, 7), (4, 6), (5, 5), (6, 4), (7, 3), (8, 2), (9, 1)]

lstrip
    Like Python's lstrip-method for str.
    >>> 'abc   ' | lstrip
    'abc   '
    >>> '.,[abc] ] ' | lstrip('.,[] ')
    'abc] ] '

map(), select()
    Apply a conversion expression given as parameter
    to each element of the given iterable
    >>> list([1, 2, 3] | map(lambda x: x * x))
    [1, 4, 9]

permutations()
    Returns all possible permutations
    >>> list('ABC' | permutations(2))
    [('A', 'B'), ('A', 'C'), ('B', 'A'), ('B', 'C'), ('C', 'A'), ('C', 'B')]

    >>> list(range(3) | permutations)
    [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

reverse
    Like Python's built-in "reversed" primitive.
    >>> list([1, 2, 3] | reverse)
    [3, 2, 1]

rstrip
    Like Python's rstrip-method for str.
    >>> '  abc   ' | rstrip
    '  abc'
    >>> '.,[abc] ] ' | rstrip('.,[] ')
    '.,[abc'

select()
    alias for map(), see map()

skip()
    Skips the given quantity of elements from the given iterable, then yields
    >>> list((1, 2, 3, 4, 5) | skip(2))
    [3, 4, 5]

skip_while()
    Like itertools.dropwhile, skips elements of the given iterable
    while the predicate is true, then yields others:
    >>> list([1, 2, 3, 4] | skip_while(lambda x: x < 3))
    [3, 4]

sort()
    Like Python's built-in "sorted" primitive. Allows cmp (Python 2.x
    only), key, and reverse arguments. By default sorts using the
    identity function as the key.

    >>> ''.join("python" | sort)
    'hnopty'
    >>> list([5, -4, 3, -2, 1] | sort(key=abs))
    [1, -2, 3, -4, 5]

strip
    Like Python's strip-method for str.
    >>> '  abc   ' | strip
    'abc'
    >>> '.,[abc] ] ' | strip('.,[] ')
    'abc'

t
    Like Haskell's operator ":"
    >>> list(0 | t(1) | t(2)) == list(range(3))
    True

tail()
    Yields the given quantity of the last elements of the given iterable.
    >>> list((1, 2, 3, 4, 5) | tail(3))
    [3, 4, 5]

take()
    Yields the given quantity of elements from the given iterable, like head
    in shell script.
    >>> list((1, 2, 3, 4, 5) | take(2))
    [1, 2]

take_while()
    Like itertools.takewhile, yields elements of the
    given iterable while the predicate is true:
    >>> list([1, 2, 3, 4] | take_while(lambda x: x < 3))
    [1, 2]

tee
    tee outputs to the standard output and yield unchanged items, useful for
    debugging
    >>> sum([1, 2, 3, 4, 5] | tee)
    1
    2
    3
    4
    5
    15

transpose()
    Transposes the rows and columns of a matrix
    >>> [[1, 2, 3], [4, 5, 6], [7, 8, 9]] | transpose
    [(1, 4, 7), (2, 5, 8), (3, 6, 9)]

traverse
    Recursively unfold iterables:
    >>> list([[1, 2], [[[3], [[4]]], [5]]] | traverse)
    [1, 2, 3, 4, 5]
    >>> squares = (i * i for i in range(3))
    >>> list([[0, 1, 2], squares] | traverse)
    [0, 1, 2, 0, 1, 4]

uniq()
    Like dedup() but only deduplicate consecutive values, using the given
    key function if provided (or else the identity)

    >>> list([1, 1, 2, 2, 3, 3, 1, 2, 3] | uniq)
    [1, 2, 3, 1, 2, 3]
    >>> list([1, 1, 2, 2, 3, 3, 1, 2, 3] | uniq(key=lambda n:n % 2))
    [1, 2, 3, 2, 3]

where(), filter()
    Only yields the matching items of the given iterable:
    >>> list([1, 2, 3] | where(lambda x: x % 2 == 0))
    [2]

Euler project samples

Find the sum of all the multiples of 3 or 5 below 1000.

euler1 = (
    sum(itertools.count() | select(lambda x: x * 3) | take_while(lambda x: x < 1000))
    + sum(itertools.count() | select(lambda x: x * 5) | take_while(lambda x: x < 1000))
    - sum(itertools.count() | select(lambda x: x * 15) | take_while(lambda x: x < 1000))
)
assert euler1 == 233168

Find the sum of all the even-valued terms in Fibonacci which do not exceed four million.

euler2 = sum(fib() | where(lambda x: x % 2 == 0) | take_while(lambda x: x < 4000000))
assert euler2 == 4613732

Find the difference between the sum of the squares of the first one hundred natural numbers and the square of the sum.

euler6 = sum(itertools.count(1) | take(100)) ** 2 - sum(
    itertools.count(1) | take(100) | select(lambda x: x ** 2)
)
assert euler6 == 25164150

Lazy evaluation

Using this module, you get lazy evaluation at two levels:

  • the object obtained by piping is a generator and will be evaluated only if needed,
  • within a series of pipe commands, only the elements that are actually needed will be evaluated.

To illustrate:

from itertools import count
from pipe import select, where, take


def dummy_func(x):
    print(f"processing at value {x}")
    return x


print("----- test using a generator as input -----")

print(f"we are feeding in a: {type(count(100))}")

res_with_count = (count(100) | select(dummy_func)
                             | where(lambda x: x % 2 == 0)
                             | take(2))

print(f"the resulting object is: {res_with_count}")
print(f"when we force evaluation we get:")
print(f"{list(res_with_count)}")

print("----- test using a list as input -----")

list_to_100 = list(range(100))
print(f"we are feeding in a: {type(list_to_100)} which has length {len(list_to_100)}")

res_with_list = (list_to_100 | select(dummy_func)
                             | where(lambda x: x % 2 == 0)
                             | take(2))

print(f"the resulting object is: {res_with_list}")
print(f"when we force evaluation we get:")
print(f"{list(res_with_list)}")

Which prints:

----- test using a generator as input -----
we are feeding in a: <class 'itertools.count'>
the resulting object is: <generator object take at 0x7fefb5e70c10>
when we force evaluation we get:
processing at value 100
processing at value 101
processing at value 102
processing at value 103
processing at value 104
[100, 102]
----- test using a list as input -----
we are feeding in a: <class 'list'> which has length 100
the resulting object is: <generator object take at 0x7fefb5e70dd0>
when we force evaluation we get:
processing at value 0
processing at value 1
processing at value 2
processing at value 3
processing at value 4
[0, 2]

pipe's People

Contributors

abdur-rahmaanj avatar amper avatar babakness avatar brentp avatar briaoeuidhtns avatar dalexander avatar devrma avatar j450h1 avatar javadba avatar jbvsmo avatar jerabaul29 avatar julienpalard avatar mrjbq7 avatar nykh avatar righthandabacus avatar safwank avatar sergiors avatar sobolevn avatar sugatoray avatar vstoykov avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.