iskandr / parakeet Goto Github PK

Runtime compiler for numerical Python

License: Other

Python 100.00%

parakeet's Introduction

Parakeet

This project is no longer being maintained.

Parakeet was a runtime accelerator for an array-oriented subset of Python. In retrospect, I don't think that whole-function type specialization at the AST level is a scalable approach to speeding up a sufficiently large subset of Python. General-purpose Python code should probably be accelerated using a bytecode JIT, whereas high-performance numerical code should use a DSL with explicit parallel operators.

Example

To accelerate a function, wrap it with Parakeet's @jit decorator:

import numpy as np 
from parakeet import jit 

alpha = 0.5
beta = 0.3
x = np.array([1,2,3])
y = np.tanh(x * alpha) + beta
   
@jit
def fast(x, alpha = 0.5, beta = 0.3):
  return np.tanh(x * alpha) + beta 
   
@jit 
def loopy(x, alpha = 0.5, beta = 0.3):
  y = np.empty_like(x, dtype = float)
  for i in xrange(len(x)):
    y[i] = np.tanh(x[i] * alpha) + beta
  return y
     
@jit
def comprehension(x, alpha = 0.5, beta = 0.3):
  return np.array([np.tanh(xi*alpha) + beta for xi in x])
  
assert np.allclose(fast(x), y)
assert np.allclose(loopy(x), y)
assert np.allclose(comprehension(x), y)

Install

You should be able to install Parakeet from its PyPI package by running:

pip install parakeet

Dependencies

Parakeet is written for Python 2.7 (sorry internet) and depends on:

dsltools
nose for unit tests
NumPy
appdirs

The default backend (which uses OpenMP) requires gcc 4.4+.

Windows: If you have a 32-bit Windows install, your compiler should come from Cygwin or MinGW. Getting Parakeet working on 64-bit Windows is non-trivial and seems to require colossal hacks.

Mac OS X: By default, your machine probably either has only clang or an outdated version of gcc. You can get a more recent version using HomeBrew

If you want to use the CUDA backend, you need to have an NVIDIA graphics card and install both the CUDA Toolkit and PyCUDA.

How does it work?

Your untyped function gets used as a template from which multiple type specializations are generated (for each distinct set of input types). These typed functions are then churned through many optimizations before finally getting translated into native code.

More information

Ask questions on the discussion group
Watch the Parakeet presentation from this year's PyData Boston, look at the HotPar slides from last year
Contact the main developer directly

Supported language features

Parakeet cannot accelerate arbitrary Python code, it only supports a limited subset of the language:

Scalar operations (i.e. x + 3 * y)
Control flow (if-statements, loops, etc...)
Nested functions and lambdas
Tuples
Slices
NumPy array expressions (i.e. x[1:, :] + 2 * y[:-1, ::2])
Some NumPy library functions like np.ones and np.sin (look at the mappings module for a full list)
List literals (interpreted as array construction)
List comprehensions (interpreted as array comprehensions)
Parakeet's higher order array operations like parakeet.imap, parakeet.scan, and parakeet.allpairs

Backends

Parakeet currently supports compilation to sequential C, multi-core C with OpenMP (default), or LLVM (deprecated). To switch between these options change parakeet.config.backend to one of:

"openmp": compiles with gcc, parallel operators run across multiple cores (default)
"c": lowers all parallel operators to loops, compile sequential code with gcc
"cuda": launch parallel operations on the GPU (experimental)
"llvm": older backend, has fallen behind and some programs may not work
"interp" : pure Python intepreter used for debugging optimizations, only try this if you think CPython is about 10,000x too fast for your taste

parakeet's People

Contributors

Stargazers

Watchers

Forkers

lucciano cournape tillsten tlevine waytai psattige schevalier abramovuch py-crash harshattray diegslva timgates42

parakeet's Issues

Unbound Local Variable in negative_index_elim.py

I understand that indexing by boolean mask is not suppoerted but it shouldn't fail that way.

code snippet:

@jit
def hyst_(mag, edge_map, labels, num_labels, high_thresh):
  for i in range(num_labels):
    if np.max(mag[labels == i]) < high_thresh:
        edge_map[labels == i] = False
  return edge_map

Stacktrace:

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/frontend/decorators.pyc in __call__(self, *args, **kwargs)
     25 
     26     typed_fn, linear_args = specialize(self.untyped, args, kwargs)
---> 27     return run_typed_fn(typed_fn, linear_args, backend_name)
     28 
     29 

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/frontend/run_function.pyc in run_typed_fn(fn, args, backend)
     77 
     78   elif backend == 'openmp':
---> 79     return openmp_backend.run(fn, args)
     80 
     81   elif backend == 'cuda':

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/openmp_backend/run_function.pyc in run(fn, args)
     12   args = prepare_args(args, fn.input_types)
     13 
---> 14   fn = after_indexify(fn)
     15   fn = final_loop_optimizations.apply(fn)
     16   if config.value_specialization:

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/transforms/phase.pyc in __call__(self, fn, run_dependencies)
    118 
    119   def __call__(self, fn, run_dependencies = True):
--> 120     return self.apply(fn, run_dependencies)
    121 
    122   def should_skip(self, fn):

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/transforms/phase.pyc in apply(self, fn, run_dependencies)
    149 
    150     if self.depends_on and run_dependencies:
--> 151       fn = apply_transforms(fn, self.depends_on)
    152 
    153     if self.copy:

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/transforms/phase.pyc in apply_transforms(fn, transforms, cleanup, phase_name, transform_history)
     27       continue
     28 
---> 29     fn = t.apply(fn)
     30 
     31     assert fn is not None, "%s transformed fn into None" % T

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/transforms/phase.pyc in apply(self, fn, run_dependencies)
    149 
    150     if self.depends_on and run_dependencies:
--> 151       fn = apply_transforms(fn, self.depends_on)
    152 
    153     if self.copy:

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/transforms/phase.pyc in apply_transforms(fn, transforms, cleanup, phase_name, transform_history)
     27       continue
     28 
---> 29     fn = t.apply(fn)
     30 
     31     assert fn is not None, "%s transformed fn into None" % T

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/transforms/phase.pyc in apply(self, fn, run_dependencies)
    164                            cleanup = self.cleanup,
    165                            phase_name = str(self),
--> 166                            transform_history = fn.transform_history)
    167 
    168       if self.post_apply:

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/transforms/phase.pyc in apply_transforms(fn, transforms, cleanup, phase_name, transform_history)
     27       continue
     28 
---> 29     fn = t.apply(fn)
     30 
     31     assert fn is not None, "%s transformed fn into None" % T

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/transforms/transform.pyc in apply(self, fn)
    500 
    501     # pop the outermost block, which have been written to by
--> 502     new_body = self.transform_block(fn.body)
    503 
    504     if len(pre_block) > 0:

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/transforms/transform.pyc in transform_block(self, stmts)
    455 
    456     for old_stmt in stmts:
--> 457       new_stmt = self.transform_stmt(old_stmt)
    458       if new_stmt is not None:
    459         self.blocks.append_to_current(new_stmt)

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/transforms/transform.pyc in transform_stmt(self, stmt)
    432       return self.transform_Assign(stmt)
    433     elif stmt_class is ForLoop:
--> 434       return self.transform_ForLoop(stmt)
    435     elif stmt_class is While:
    436       return self.transform_While(stmt)

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/transforms/transform.pyc in transform_ForLoop(self, stmt)
    414     stmt.stop = self.transform_expr(stmt.stop)
    415     stmt.step = self.transform_expr(stmt.step)
--> 416     stmt.body = self.transform_block(stmt.body)
    417     stmt.merge = self.transform_merge_after_loop(stmt.merge)
    418     return stmt

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/transforms/transform.pyc in transform_block(self, stmts)
    455 
    456     for old_stmt in stmts:
--> 457       new_stmt = self.transform_stmt(old_stmt)
    458       if new_stmt is not None:
    459         self.blocks.append_to_current(new_stmt)

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/transforms/transform.pyc in transform_stmt(self, stmt)
    430     stmt_class = stmt.__class__
    431     if stmt_class is Assign:
--> 432       return self.transform_Assign(stmt)
    433     elif stmt_class is ForLoop:
    434       return self.transform_ForLoop(stmt)

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/transforms/transform.pyc in transform_Assign(self, stmt)
    382 
    383   def transform_Assign(self, stmt):
--> 384     stmt.rhs = self.transform_expr(stmt.rhs)
    385     stmt.lhs = self.transform_lhs(stmt.lhs)
    386     return stmt

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/transforms/transform.pyc in transform_expr(self, expr)
    278       result = self.transform_TupleProj(expr)
    279     elif expr_class is Index:
--> 280       result = self.transform_Index(expr)
    281     elif expr_class is Slice:
    282       result = self.transform_Slice(expr)

/usr/lib/python2.7/site-packages/parakeet-0.23.2-py2.7.egg/parakeet/transforms/negative_index_elim.pyc in transform_Index(self, expr)
     48       index_elts = given_elts + [slice_none] * (n_expected - n_given)
     49 
---> 50     index_elts = list(index_elts)
     51     index_ranges = [self.get(idx_elt) for idx_elt in index_elts]
     52 

UnboundLocalError: local variable 'index_elts' referenced before assignment

Support clang on OS X 10.9

On OS X 10.9, clang is the only compiler shipped with XCode. Clang doesn't yet support OpenMP, so it might make sense to disable OpenMP backend was disabled when clang was detected. I'm also seeing various warnings or errors about compiler options for some examples, like invalid argument '-bundle' not allowed with '-dynamiclib'.

If I manually switch to the CUDA backend, I see the error ld: unknown option: -Wl, followed by the stack trace:

eriv(Rosenbrock):
Traceback (most recent call last):
  File "rosenbrock.py", line 25, in <module>
    print "Deriv(Rosenbrock):", rosenbrock_derivative(x)
  File "/work/mobi/parakeet/parakeet/frontend/decorators.py", line 20, in __call__
    return run_python_fn(self.f, args, kwargs, backend = backend_name)
  File "/work/mobi/parakeet/parakeet/frontend/run_function.py", line 133, in run_python_fn
    return run_untyped_fn(untyped, args, kwargs, backend)
  File "/work/mobi/parakeet/parakeet/frontend/run_function.py", line 115, in run_untyped_fn
    return run_typed_fn(typed_fn, linear_args, backend)
  File "/work/mobi/parakeet/parakeet/frontend/run_function.py", line 83, in run_typed_fn
    return cuda_backend.run(fn, args)
  File "/work/mobi/parakeet/parakeet/cuda_backend/run_function.py", line 17, in run
    compiled_fn = CudaCompiler().compile_entry(fn)
  File "/work/mobi/parakeet/parakeet/c_backend/pymodule_compiler.py", line 771, in compile_entry
    linker_flag_prefix = self.linker_flag_prefix)
  File "/work/mobi/parakeet/parakeet/c_backend/compile_util.py", line 271, in compile_module_from_source
    if not compiler_is_gnu(compiler):
  File "/work/mobi/parakeet/parakeet/c_backend/compile_util.py", line 209, in compiler_is_gnu
    return (compiler.endswith("gcc") or
AttributeError: 'list' object has no attribute 'endswith'

I am going to try installing gcc 4.8 next to see if that solves these problems.

Website

Heya, just ran across this library, which seems abandoned, but FYI the website for the repo seems unrelated to the repo itself. Might want to update or delete the reference.

Args import problem

There is a problem with args module being imported wrong. I suspect it works when you don't install the package, but I'm trying to install it and it cannot find the file args.

Also, please rename this file, since there is a very common module called args in Python for parsing command line arguments. The two has the exact same name!

In [1]: import parakeet
---------------------------------------------------------------------------
ImportError   Traceback (most recent call last)
<ipython-input-1-c88f960e1507> in <module>()
----> 1 import parakeet

e:\docs\iskandr-parakeet-c6bd092\iskandr-parakeet-c6bd092\parakeet\__init__.py i
n <module>()
  1 import config
  2
----> 3 from frontend import jit, macro
  4 from lib import *
  5 from run_function import run, specialize_and_compile

e:\docs\iskandr-parakeet-c6bd092\iskandr-parakeet-c6bd092\parakeet\frontend\__in
it__.py in <module>()
  1
  2
----> 3 from ast_conversion import translate_function_value, translate_function_
ast
  4 from decorators import jit, macro, staged_macro
  5 import mappings

e:\docs\iskandr-parakeet-c6bd092\iskandr-parakeet-c6bd092\parakeet\frontend\ast_
conversion.py in <module>()
 17 from .. syntax import Assign, If, ForLoop, Var, PrimCall, Map
 18 from .. syntax import none, true, false, one_i64, zero_i64
---> 19 from decorators import macro, jit
 20 from function_registry import already_registered_python_fn
 21 from function_registry import register_python_fn, lookup_python_fn

e:\docs\iskandr-parakeet-c6bd092\iskandr-parakeet-c6bd092\parakeet\frontend\deco
rators.py in <module>()
  5 from .. syntax import const, is_python_constant
  6
----> 7 from run_function import run
  8
  9 class jit:

e:\docs\iskandr-parakeet-c6bd092\iskandr-parakeet-c6bd092\parakeet\frontend\run_
function.py in <module>()
----> 1 from args import FormalArgs, ActualArgs
  2
  3 import config
  4 import names
  5 import syntax

ImportError: No module named args

test_windowed.py fails with version 0.13

    C:\Documents and Settings\reckoner\Desktop\parakeet-0.13\tests
    >python test_windowed.py
    Running test_winmap_zeros...
    Stack dump:
    0.      Running pass 'Basic CallGraph Construction' on function '@winmap_zeros.2'
    0x000A3E3E (0x02094360 0x037EB738 0x02094360 0x00000000) <unknown module>

Win64 and llvm backend

I try to run the following simple example:

import parakeet
parakeet.config.default_backend = 'llvm'
import numpy as np

a = np.random.rand(400, 600)

@parakeet.jit
def add(a, b):
    return a + b

add(a, a)

and i get a

AttributeError: 'Compiler' object has no attribute 'compile_AllocArray'

llvm version is 0.12.0.

Fails to compile simple function

Parakeet gives an Index != IntT error when trying to run this simple script to find a border around an image:


@jit
def find_border(is_background):
  h, w = is_background.shape[:2]

  top, left, right, bottom = (-1, -1, -1, -1)
  # find top
  for i in range(h):
    if is_background[i, :].sum() != w and top == -1:
      top = i

  for i in range(h - 1, 0, -1):
    if is_background[i, :].sum() != w and bottom == -1:
      bottom = i

  for i in range(w):
    if is_background[:, i].sum() != h and left == -1:
      left = i

  for i in range(w - 1, 0, -1):
    if is_background[:, i].sum() != h and right == -1:
      right = i

  return top, left, right, bottom

is_background = np.empty((100, 100), dtype=np.bool)
find_border(is_background)

Numpy accepts it without any errors.

nfs issue

Hi,

As I tried to make it works on a grid, nfs may cause issue with os.rename call.
https://mail.python.org/pipermail/python-list/2005-February/342893.html

regards,
Pierrick

Running convolution.py example results in AttributeError

Successfully installed parakeet on Ubuntu 14.04 LTS 64bit + up to date Anaconda

All parakeet unit tests completed successfully:

fviktor@ml:~/dev/parakeet/test$ nosetests --processes=6 .
.........................................................................................................................................................................................................................................................................................................................................................................................................................
----------------------------------------------------------------------
Ran 409 tests in 37.595s

OK

numpy version: 1.9.2

Tried to run convolution.py example, but got exception:

fviktor@ml:~/dev/parakeet/examples$ python convolution.py 
Input [[  2.58042993e-01   5.62971928e-01   3.18518354e-01 ...,  -4.92677614e-01
   -5.33775979e-01  -1.08248866e+00]
 [  4.90764757e-03   1.17602492e+00  -2.87544067e-03 ...,   7.22071302e-01
    3.81995794e-01  -2.45555615e-01]
 [  1.31822646e+00   1.15793986e+00   1.14658108e+00 ...,  -1.12846011e+00
    1.19572064e+00   3.12640182e+00]
 ..., 
 [ -2.06217431e-01  -2.20358510e-03   1.80575683e+00 ...,   6.71510654e-01
    3.18903622e-01   1.30837962e-01]
 [  4.84092250e-01  -3.69639627e-01  -2.79786297e-01 ...,  -1.35688015e+00
   -1.10111849e+00   2.49257160e-01]
 [ -2.35768967e-02  -1.46622324e+00   5.89974521e-02 ...,  -1.23096420e+00
   -3.50698146e-01   6.66309537e-01]]
Convolved output
Traceback (most recent call last):
  File "convolution.py", line 24, in <module>
    print "Convolved output", conv_3x3_trim(x,w)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/frontend/decorators.py", line 27, in __call__
    return run_typed_fn(typed_fn, linear_args, backend_name)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/frontend/run_function.py", line 77, in run_typed_fn
    return c_backend.run(fn, args)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/c_backend/run_function.py", line 13, in run
    fn = lower_to_loops(fn)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/transforms/phase.py", line 120, in __call__
    return self.apply(fn, run_dependencies)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/transforms/phase.py", line 153, in apply
    fn = apply_transforms(fn, self.depends_on)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/transforms/phase.py", line 29, in apply_transforms
    fn = t.apply(fn)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/transforms/phase.py", line 153, in apply
    fn = apply_transforms(fn, self.depends_on)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/transforms/phase.py", line 29, in apply_transforms
    fn = t.apply(fn)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/transforms/phase.py", line 153, in apply
    fn = apply_transforms(fn, self.depends_on)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/transforms/phase.py", line 29, in apply_transforms
    fn = t.apply(fn)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/transforms/phase.py", line 153, in apply
    fn = apply_transforms(fn, self.depends_on)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/transforms/phase.py", line 29, in apply_transforms
    fn = t.apply(fn)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/transforms/phase.py", line 168, in apply
    transform_history = fn.transform_history)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/transforms/phase.py", line 36, in apply_transforms
    fn = apply_transforms(fn, cleanup, [], phase_name = "cleanup")
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/transforms/phase.py", line 29, in apply_transforms
    fn = t.apply(fn)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/transforms/transform.py", line 492, in apply
    pre_fn = self.pre_apply(self.fn)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/transforms/simplify.py", line 52, in pre_apply
    self.mutable_types = ma.visit_fn(fn)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/analysis/mutability_analysis.py", line 79, in visit_fn
    self.visit_block(fn.body)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/analysis/syntax_visitor.py", line 272, in visit_block
    self.visit_stmt(s)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/analysis/syntax_visitor.py", line 343, in visit_stmt
    getattr(self, self._stmt_method_names[c])(stmt)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/analysis/syntax_visitor.py", line 296, in visit_Return
    self.visit_expr(stmt.value)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/analysis/syntax_visitor.py", line 237, in visit_expr
    return method(expr)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/analysis/syntax_visitor.py", line 129, in visit_OuterMap
    self.visit_expr(arg)
  File "/home/fviktor/anaconda/lib/python2.7/site-packages/parakeet-0.24-py2.7.egg/parakeet/analysis/syntax_visitor.py", line 234, in visit_expr
    method = getattr(self, method_name)
AttributeError: 'TypeBasedMutabilityAnalysis' object has no attribute 'visit_NoneType'

Using distutils.

Using the something along following (copied from the cython-inline function https://github.com/cython/cython/blob/master/Cython/Build/Inline.py), seems to do it:

from distutils.core import Distribution, Extension
from distutils.command.build_ext import build_ext

def _get_build_extension():
    dist = Distribution()
    # Ensure the build respects distutils configuration by parsing
    # the configuration files
    config_files = dist.find_config_files()
    dist.parse_config_files(config_files)
    build_extension = build_ext(dist)
    build_extension.finalize_options()
    return build_extension

build_extension = _get_build_extension()  
ext = Extension(fn_name, [src_filename], include_dirs=[np.get_include()])
build_extension.extensions = [ext]
#build_extension.build_temp = os.path.dirname(pyx_file)
#build_extension.build_lib = lib_dir
build_extension.run()

module = imp.load_dynamic(fn_name, module_path)

Maybe it is worth to try it because it seems to replace most of the comp_util file.

ImportError: cannot import name IntT

    Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import parakeet
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "c:\Python27\lib\site-packages\parakeet-0.13-py2.7.egg\parakeet\__init__.py", line 2, in <module>
        import type_conv_decls
      File "c:\Python27\lib\site-packages\parakeet-0.13-py2.7.egg\parakeet\type_conv_decls.py", line 3, in <module>
        from ndtypes import ArrayT, type_conv, typeof_array
      File "build\bdist.win32\egg\ndtypes\__init__.py", line 3, in <module>
        # $Id$
      File "build\bdist.win32\egg\ndtypes\array_type.py", line 9, in <module>
      File "build\bdist.win32\egg\ndtypes\ptr_type.py", line 3, in <module>
    ImportError: cannot import name IntT

generic ndindexing

Do you have any recommendations for doing generic nd indexing in a parakeet jit'ed function?

I'm having some problems finding something parakeet will take since numpy uses tuples for generic indexing and parakeet doesn't like those.

Trying to do something like so:

def filter(data, output, neighbors):
    for index in np.ndindex(data.shape):
         for neighbor in neighbors_of(index):
              output[index] += data[neighbor]
    return output

Parakeet

Hi,

Just wondering why Parakeet was abandonned, since the performance
look very nice.

I have some questions too.
Thanks, regards

Pure Python numpy-based function faster than parakeet

from parakeet import jit
import numpy as np

@jit
def waldjit(v, a, rands, u_rands, sigma, accum):
    mu = a / v
    lam = a**2 / sigma**2
    y = rands[:, accum]**2
    x = mu + (mu**2. * y) / (2.*lam) - mu / (2.*lam) * np.sqrt(4.*mu*lam*y + mu**2. * y**2.)
    z = u_rands[:, accum]

    return x 

def waldnojit(v, a, rands, u_rands, sigma, accum):
    mu = a / v
    lam = a**2 / sigma**2
    y = rands[:, accum]**2
    x = mu + (mu**2. * y) / (2.*lam) - mu / (2.*lam) * np.sqrt(4.*mu*lam*y + mu**2. * y**2.)
    z = u_rands[:, accum]
    return x

rands = np.random.randn(10000, 1)
urands = np.random.rand(10000, 1)
#warmup
waldjit(1, 2, rands, u_rands, 1, 0)
%timeit waldnojit(1, 2, rands, u_rands, 1, 0)
%timeit waldjit(1, 2, rands, u_rands, 1, 0)

produces:

10000 loops, best of 3: 170 µs per loop [numpy]
100 loops, best of 3: 2.73 ms per loop [parakeet 0.22]

Quite significant, any idea why?

parakeet jit-ed functions fail with IOError when run in the shell

Hi Alex, I met you and heard your talk in Boston, and was trying out parakeet; neat little project!

This may just be an architectural issue with how parakeet is implemented, but I can't get examples that run fine as python scripts to run at the interactive shell. Everything is fine until I try to call a jit-ed function, and then I get this:

Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/parakeet/frontend/decorators.py", line 19, in call
return run_python_fn(self.f, args, kwargs, backend = backend_name)
File "/usr/local/lib/python2.7/dist-packages/parakeet/frontend/run_function.py", line 144, in run_python_fn
untyped = ast_conversion.translate_function_value(fn)
File "/usr/local/lib/python2.7/dist-packages/parakeet/frontend/ast_conversion.py", line 868, in translate_function_value
source = inspect.getsource(fn)
File "/usr/lib/python2.7/inspect.py", line 701, in getsource
lines, lnum = getsourcelines(object)
File "/usr/lib/python2.7/inspect.py", line 690, in getsourcelines
lines, lnum = findsource(object)
File "/usr/lib/python2.7/inspect.py", line 538, in findsource
raise IOError('could not get source code')
IOError: could not get source code

I tried multiple examples that are work fine in ".py" files, but always this same problem in the shell. Also, the issue seems to only occur if the function being jit-ed is defined at the shell; if I get the same function from an import it instead, then the problem goes away.

I'm on Ubuntu 13.04 64-bit, python2.7, llvm 3.2 (freshly compiled), llvmpy trunk (as of 9/5/13), and parakeet from pip (0.14.2).

I'm not too worried about this issue, but just thought I'd put it out there.

Cheers,
-David

python 3 support ?

Hi,

Are you planning to add python 3 support to parakeet ?

NPY_ARRAY_OWNDATA error

I'm trying to jit this function:

@jit
def waldjit(v, a, rands, u_rands, sigma=1, accum=0):
    mu = a / v
    lam = a**2 / sigma**2
    y = rands[:, accum]**2
    out = np.empty_like(y)   
    x = mu + (mu**2. * y) / (2.*lam) - mu / (2.*lam) * np.sqrt(4.*mu*lam*y + mu**2. * y**2.)
    z = u_rands[:, accum]
    return x

But trying to execute produces:

/tmp/parakeet_waldjit_THQUC6.c: In function ‘waldjit’:
/tmp/parakeet_waldjit_THQUC6.c:145:38: error: ‘NPY_ARRAY_OWNDATA’ undeclared (first use in this function)
/tmp/parakeet_waldjit_THQUC6.c:145:38: note: each undeclared identifier is reported only once for each function it appears in

This is with parakeet 0.21.

Also, for some reason the gcc error output wasn't written to console when I called it from python, I just get:

Parakeet encountered error(s) during compilation:

Array closure args get fed into transform

Readme doesn't mention config file.

Had to dig into the source to see how i can change the backend.

nose missing in the requirements.txt

Since nose is missing in the requirements.txt, installation fails

unrecognized command line option 'msse2'

Hi,

I'm trying to use parakeet on my raspberry pi 2, with the code I tested on my laptop.
While the parakeet was successfully installed with pip, I get error same as the title.
The detail I found was following:

File "/usr/local/lib/python2.7/dist-packages/numpy/distutils/ccompiler.py", line 205, in CCompiler_compile
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
File "/usr/local/lib/python2.7/dist-packages/numpy/distutils/unixccompiler.py", line 40, in UnixCCompiler__compile
raise CompileError(msg)
CompileError: Command "gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/lib/python2.7/dist-packages/numpy/core/include -I/usr/include/python2.7 -I/usr/include/python2.7 -c /tmp/parakeet_labeling_YAqlBb.c -o build/temp.linux-armv7l-2.7/tmp/parakeet_labeling_YAqlBb.o -I/usr/local/lib/python2.7/dist-packages/numpy/core/include -I/usr/include/python2.7 -fPIC -O2 -msse2 -ffast-math -fopenmp" failed with exit status 1

So, I guess some how gcc doesn't recognise -msse2 only on raspberry pi? I've tried reinstall the gcc, but it didn't help.

Cannot install on windows

I'm trying to install parakeet, however setup fails both from pip and git.

From pip, here is the error:

Downloading/unpacking parakeet
  Downloading parakeet-0.13.tar.gz (138kB): 138kB downloaded
  Running setup.py egg_info for package parakeet

package init file '.\tests\__init__.py' not found (or not a regular file)
error: package directory '.\data' does not exist
Complete output from command python setup.py egg_info:
running egg_info

creating pip-egg-info\parakeet.egg-info

writing pip-egg-info\parakeet.egg-info\PKG-INFO

writing top-level names to pip-egg-info\parakeet.egg-info\top_level.txt

writing dependency_links to pip-egg-info\parakeet.egg-info\dependency_links.txt

writing manifest file 'pip-egg-info\parakeet.egg-info\SOURCES.txt'

warning: manifest_maker: standard file '-c' not found



package init file '.\tests\__init__.py' not found (or not a regular file)

error: package directory '.\data' does not exist

----------------------------------------
Command python setup.py egg_info failed with error code 1 in c:\users\user\appda
ta\local\temp\pip-build-user\parakeet
Storing complete log in C:\Users\user\pip\pip.log

If I run 'python setup.py install' from github source, here is the error log:

Processing parakeet-0.14-py2.7.egg
creating c:\python27\lib\site-packages\parakeet-0.14-py2.7.egg
Extracting parakeet-0.14-py2.7.egg to c:\python27\lib\site-packages
Sorry: IndentationError: unexpected indent (closure_specializations.py, line 3)
  File "c:\python27\lib\site-packages\parakeet-0.14-py2.7.egg\parakeet\frontend\
type_conv_decls.py", line 4
from loopjit.ndtypes import type_conv, typeof_array,
SyntaxError: trailing comma not allowed without surrounding parentheses

Adding parakeet 0.14 to easy-install.pth file

Installed c:\python27\lib\site-packages\parakeet-0.14-py2.7.egg
Processing dependencies for parakeet==0.14
Finished processing dependencies for parakeet==0.14

Sample code contains errors in README.md

Variables alpha and beta are not defined.

y = np.tanh(x * alpha) + beta
NameError: name 'alpha' is not defined

C backend print options can't be changed at runtime

Mention PyCUDA as required for CUDA support

Although not surprising, the Parakeet documentation doesn't actually say that PyCUDA is required to use the CUDA backend. I had to dig through the cuda backend source code to figure out why my GPU was not being detected.

It would be good to mention that in the README. :)

memory leak

I think there might be a memory leak in parakeet.

import numpy as np
from parakeet import jit

def local_maxima1(a,size):
    maxima = np.ones(a.shape).astype(bool)
    for shifts in neighbors(a,size):
        maxima &= ( a > nd.shift(a,shifts,mode="nearest",order=0) )
    return maxima

@jit
def local_maxima2(a,size):
    rows, cols = a.shape
    maxima = np.zeros(a.shape)  # <-- maybe leaks this?
    for row in range(rows):
        for col in range(cols):
            maxima[row,col] = local_maximum(a,row,col,size)
    return maxima

@jit
def local_maximum(a,row,col,size):
    value = a[row,col]
    rows, cols = a.shape
    rmin, rmax = max(0,row-size), min(rows,row+size+1)
    cmin, cmax = max(0,col-size), min(cols,col+size+1)
    for rs in range(rmin,rmax):
        for cs in range(cmin,cmax):
            if ( not ( rs == row and cs == col ) ) and a[rs,cs] >= value:
                return 0.0
    return value

the call to local_maxima2 seems to leak the created array? Otherwise... this is super impressive. I'm getting a 53X speedup over the numpy version, particularly since it is much easier to abort early when using an explicit loop.

iskandr / parakeet Goto Github PK

parakeet's Introduction

Parakeet

Example

Install

Dependencies

How does it work?

More information

Supported language features

Backends

parakeet's People

Contributors

Stargazers

Watchers

Forkers

parakeet's Issues

Recommend Projects

Recommend Topics

Recommend Org