Giter VIP home page Giter VIP logo

plum's Introduction

DOI CI Coverage Status Latest Docs Code style: black

Everybody likes multiple dispatch, just like everybody likes plums.

The design philosophy of Plum is to provide an implementation of multiple dispatch that is Pythonic, yet close to how Julia does it. See here for a comparison between Plum, multipledispatch, and multimethod.

Note: Plum 2 is now powered by Beartype! If you notice any issues with the new release, please open an issue.

Installation

Plum requires Python 3.8 or higher.

pip install plum-dispatch

See here.

What's This?

Plum brings your type annotations to life:

from numbers import Number

from plum import dispatch


@dispatch
def f(x: str):
    return "This is a string!"


@dispatch
def f(x: int):
    return "This is an integer!"


@dispatch
def f(x: Number):
    return "This is a general number, but I don't know which type."
>>> f("1")
'This is a string!'

>>> f(1)
'This is an integer!'

>>> f(1.0)
'This is a number, but I don't know which type.'

>>> f(object())
NotFoundLookupError: `f(<object object at 0x7fd3b01cd330>)` could not be resolved.

Closest candidates are the following:
    f(x: str)
        <function f at 0x7fd400644ee0> @ /<ipython-input-2-c9f6cdbea9f3>:6
    f(x: int)
        <function f at 0x7fd3a0235ca0> @ /<ipython-input-2-c9f6cdbea9f3>:11
    f(x: numbers.Number)
        <function f at 0x7fd3a0235d30> @ /<ipython-input-2-c9f6cdbea9f3>:16

Important

Dispatch, as implemented by Plum, is based on the positional arguments to a function. Keyword arguments are not used in the decision making for which method to call. In particular, this means that positional arguments without a default value must always be given as positional arguments!

Example:

from plum import dispatch

@dispatch
def f(x: int):
   return x

>>> f(1)        # OK
1

>> try: f(x=1)  # Not OK
... except Exception as e: print(f"{type(e).__name__}: {e}")
NotFoundLookupError: `f()` could not be resolved...

This also works for multiple arguments, enabling some neat design patterns:

from numbers import Number, Real, Rational

from plum import dispatch


@dispatch
def multiply(x: Number, y: Number):
    return "Performing fallback implementation of multiplication..."


@dispatch
def multiply(x: Real, y: Real):
    return "Performing specialised implementation for reals..."


@dispatch
def multiply(x: Rational, y: Rational):
    return "Performing specialised implementation for rationals..."
>>> multiply(1, 1)
'Performing specialised implementation for rationals...'

>>> multiply(1.0, 1.0)
'Performing specialised implementation for reals...'

>>> multiply(1j, 1j)
'Performing fallback implementation of multiplication...'

>>> multiply(1, 1.0)  # For mixed types, it automatically chooses the right optimisation!
'Performing specialised implementation for reals...'

Projects Using Plum

The following projects are using Plum to do multiple dispatch! Would you like to add your project here? Please feel free to open a PR to add it to the list!

  • Coordinax implements coordinates in JAX.
  • GPAR is an implementation of the Gaussian Process Autoregressive Model.
  • GPCM is an implementation of various Gaussian Process Convolution Models.
  • Galax does galactic and gravitational dynamics.
  • Geometric Kernels implements kernels on non-Euclidean spaces, such as Riemannian manifolds, graphs, and meshes.
  • LAB uses Plum to provide backend-agnostic linear algebra (something that works with PyTorch/TF/JAX/etc).
  • MLKernels implements standard kernels.
  • MMEval is a unified evaluation library for multiple machine learning libraries.
  • Matrix extends LAB and implements structured matrix types, such as low-rank matrices and Kronecker products.
  • NetKet, a library for machine learning with JAX/Flax targeted at quantum physics, uses Plum extensively to pick the right, efficient implementation for a large combination of objects that interact.
  • NeuralProcesses is a framework for composing Neural Processes.
  • OILMM is an implementation of the Orthogonal Linear Mixing Model.
  • PySAGES is a suite for advanced general ensemble simulations.
  • Quax implements multiple dispatch over abstract array types in JAX.
  • Unxt implements unitful quantities in JAX.
  • Varz uses Plum to provide backend-agnostic tools for non-linear optimisation.

See the docs for a comparison of Plum to other implementations of multiple dispatch.

plum's People

Contributors

francesco-ballarin avatar gabrieldemarmiesse avatar hodgespodge avatar invenia-blog avatar nstarman avatar pablolion avatar philipvinc avatar rtbs-dev avatar ruancomelli avatar wesselb avatar yukinarit avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

plum's Issues

[FR] Add a way to define a function (whitout defining a method)

AKA, I'd like the equivalent of Julia's

"""
docstring
"""
function test end

so that I can more easily organize my code.

Also i am trying to make my code more functional, but for backward compatibility sometimes I have patterns like this

class A:
   def kron(self, other):
        kron(self, other)
...

@dispatch
def kron(a: A, b:A):
   ..

@dispatch
def kron(a:A, b:B)
  ..

in which case, again, being able to define a function with no methods would be great.

dispatch decorator in __init_subclass__

Hi!
I want to apply the @dispatch decorator to method in child class.
I can't edit the child class, so I try to apply the decorator in the __init_subclass__ method
When I try to run the test, it throws an error plum.resolvable.ResolutionError: Promise 'Promise()' was not kept.
Here's the test:

class A:
    def __init_subclass__(cls, **kwargs):
        cls.f = dispatch(cls.f)

    def f(self, x: str):
        return x


class B(A):
    def f(self, x: int):
        return x

    @dispatch
    def _(self):
        pass


class MethodOverloadingTests(unittest.TestCase):
    def test_call_overloaded_method(self):
        result = B().f(1)
        self.assertEqual(1, result)

Hack some way to dispatch on number of dimensions of numpy/jax arrays

It would be nice (even though admittedly hacky) if we could dispatch on the number of dimensions of a jax/numpy array, which can be probed with .ndim on an instance.

This is admittedly not part of the type information in python, but maybe we can hack it in?

I was thinking of creating a custom parametric type for signatures, but then the problem is resolving the call signature and bringing this information from the value domain to the type domain.

This happens in parametric.py if I am not mistaken, and would require changing this function to allow hacking in some custom types...
Do you have any idea on what would be the best way to implement this?

def type_of(obj):
    """Get the Plum type of an object.

    Args:
        obj (object): Object to get type of.

    Returns
        ptype: Plum type of `obj`.
    """
    if isinstance(obj, list):
        return List(_types_of_iterable(obj))

    if isinstance(obj, tuple):
        return Tuple(*(type_of(x) for x in obj))

    return ptype(type(obj))

FR: Make plum work with ipython's autoreload

The two don't work well together.
I think it's because autoreload changes the classes of objects, but for some reason forgets to update the Types in plum's dispatch tables.

Should be investigated...

Transition to pure PyProject.toml

Discussion about transitioning to pure Pyproject.toml using either setuptools or poetry.

@PhilipVinc Probably a question for another issue, but any chance you guys might consider using a python-poetry based toolchain to manage pyproject.toml? Makes life a bit easier re: dev dependencies, environments, and deployment.

I have a branch to transit plum to a pure setuptools + pyproject.toml (so not poetry) but was waiting for pip to get together and support editable installs. It seems they fixed it a few months ago so might as well open the PR.

This is now much easier to do since plum does no longer use cython.

If you think poetry > setuptools I'd be happy to know what's the advantage for packaging @tbsexton ?

operator.attrgetter/itemgetter is not supported by signature

Hi there ๐Ÿ‘‹ I'm back with another issue ๐Ÿ˜„ Here's a reproducible example:

from operator import attrgetter
from plum import Function
f = Function(lambda x: x)
pf = Function(f)
g = attrgetter('a')  # same holds for operator.itemgetter
g = pf.dispatch(g)
g(0)
ValueError: callable operator.attrgetter('a') is not supported by signature

It might be possible for us to implement a workaround on our side but it would be great if it were handled by plum. My thinking, which isn't very elegant, is to wrap it in another callable that passes through and to set that callable's __signature__, which inspect.signature uses as a sort of cache. I haven't tested that yet.

cc @wesselb


For completeness, the following also fails with a different error, but isn't a use-case for us atm so less crucial:

from operator import attrgetter
from plum import dispatch
f = attrgetter('a')
pf = dispatch(f)
AttributeError: 'operator.attrgetter' object has no attribute '__qualname__'

How should inheritance ambiguities be handled?

Hello,

First of all thank you for the good effort, this library is very useful at addressing one of the limitations of Python.

We all know that Julia doesn't support inheritance (for a good reason). However, in Python you can quite easily mix the dispatch with inheritance, and at one point I realised that I don't know what the correct behaviour should be, when you have the following setup:

from plum import dispatch

class DataA():
    pass

class DataB(DataA):
    pass

class FuncA():
    @dispatch
    def to_string(self, data: DataA):
        print('FuncA DataA')

    @dispatch
    def to_string(self, data: DataB):
        print('FuncA DataB')

class FuncB(FuncA):
    @dispatch
    def to_string(self, data: DataA):
        print('FuncB DataA')
        super().to_string(data)

data = DataB()
func = FuncB()
func.to_string(data)

It is not clear to me whether we should call FuncA.to_string(data: DataB) or FuncB.to_string(data: DataA). This is a classic ambiguity problem. In fact, if you define the same architecture as above but abstract the methods away from the classes as follows:

from plum import dispatch

class DataA():
    pass

class DataB(DataA):
    pass

class FuncA():
    pass

class FuncB(FuncA):
    pass

@dispatch
def to_string(self:FuncA, data: DataA):
    print('FuncA DataA')

@dispatch
def to_string(self:FuncA, data: DataB):
    print('FuncA DataB')

@dispatch
def to_string(self:FuncB, data: DataA):
    print('FuncB DataA')

data = DataB()
func = FuncB()
to_string(func, data)

you get the following error (which I think should be the correct behaviour):

plum.function.AmbiguousLookupError: For function "to_string", signature Signature(__main__.FuncB, __main__.DataB) is ambiguous among the following:
  Signature(__main__.FuncA, __main__.DataB) (precedence: 0)
  Signature(__main__.FuncB, __main__.DataA) (precedence: 0)

However, the first example runs without any ambiguity errors, and produces the following output:

FuncB DataA
FuncA DataB

Should this behaviour be changed to an ambiguity error or should the self variable be treated differently when resolving the ambiguities? That's one question I don't know the answer to but I would like to hear your thoughts on that.

Many thanks.

Support for pandas.core.series.Series type

I am trying to dispatch on class pandas.core.series.Series with code like,

import pandas as pd
@dispatch
def foo(x: Union[list, tuple, pd.core.series.Series]) -> str:
    ...

and I got error:
plum.function.NotFoundLookupError: For function "_quote", signature Signature(pandas.core.series.Series) could not be resolved.

It seems panas.core.series.Series is not considered as a class/type to be dispatch on. Can it be implemented? Or, it is there, but my code was wrong?

Improve resolution error messages

It would be great if we could offer some ' closest methods matches' when hitting an error, similarly to what Julia does.

If @wesselb you pointed me in the right direction, maybe by giving me a starting point of how to guesstimate the closest matches, I can try to finish a PR after that. I will have to take a 6 hours train with little internet on Wednesday so might put myself on that.

Lookup error on __init__ method

I have several different constructors of a class based on its types, so I decided to dispatch them. For example I have the following method:

class BlockArray:
    @dispatch
    def __init__(self, blocks: List[Array], grid: Sequence[int]):
        pass

data = [np.zeros(1)]
grid = (1,)
arr = BlockArray(data, grid)

But this gives the following error:

...
.venv/lib/python3.8/site-packages/plum/parametric.py:65: in __call__
    type_parameter = cls.__infer_type_parameter__(*args, **kw_args)
plum/function.py:226: in plum.function.ClassFunction.__call__
    ???
plum/function.py:536: in plum.function.Function.__call__
    ???
plum/function.py:498: in plum.function.Function.resolve_method
    ???
plum/function.py:464: in plum.function.Function.resolve_method
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   plum.function.NotFoundLookupError: For function "__infer_type_parameter__" of rosnet.array.block.BlockArray, signature Signature(plum.parametric.CovariantMeta, List[numpy.ndarray], Tuple[builtins.int, builtins.int, builtins.int]) could not be resolved.

plum/function.py:435: NotFoundLookupError

I've tried changing the type annotations but nothing. Problem seems to be in the self argument. If I otherwise cal __init__ directly like this, plum correctly dispatches the method.

class mock: pass
arr = mock()
arr.data = None
BlockArray.__init__(arr, data, grid)

No way to dispatch based on return type

I found it very annoying that I couldn't get the value with the right type from my own property node implementation. Example:

class Node:
  def __init__(self, value):
    self.value = value
  โ€ฆ
  @dispatch
  def getValue(self) -> int:
    return int(self.value)
  
  @dispatch
  def getValue(self) -> bool:
    if str(self.value).lower() == "true":
      return True
    elif str(self.value).lower() == "false":
      return False
    else:
      return bool(self.value)

n = Node("false")
v: int = n.getValue()
print(type(v))

which prints bool instead of the expected int. The type hint of variables is encoded in the namespaces __annotations__ - any chance of this getting implemented ? :) Thanks in advance !

How to handle doc strings

How do doc strings work with several methods? It would be nice to put different doc strings in different methods. I don't think this works.

Is there a recommended way then ? Maybe put one big doc string in the first method?

Plum 2 and np.typing.NDArray

Hi @wesselb
I am testing out #73 due to the improved numpy support. In the very first example there, you show how to dispatch based on shape and type. In my use cases, I am only interested in dispatching based on type. Since that is actually supported by np.typing.NDArray, I thought I could just use that, but that doesn't seem to be the case.

I report a minimal example below, in case you want to have a look before the plum 2 release.

from plum import dispatch, parametric
from typing import Any, Optional, Tuple, Union

import numpy as np
import numpy.typing

class NDArrayMeta(type):
    def __instancecheck__(self, x):
        if self.concrete:
            shape, dtype = self.type_parameter
        else:
            shape, dtype = None, None
        return (
            isinstance(x, np.ndarray)
            and (shape is None or x.shape == shape)
            and (dtype is None or x.dtype == dtype)
        )


@parametric
class NDArray(np.ndarray, metaclass=NDArrayMeta):
    @classmethod
    @dispatch
    def __init_type_parameter__(
        cls,
        shape: Optional[Tuple[int, ...]],
        dtype: Optional[Any],
    ):
        """Validate the type parameter."""
        return shape, dtype

    @classmethod
    @dispatch
    def __le_type_parameter__(
        cls,
        left: Tuple[Optional[Tuple[int, ...]], Optional[Any]],
        right: Tuple[Optional[Tuple[int, ...]], Optional[Any]],
    ):
        """Define an order on type parameters. That is, check whether
        `left <= right` or not."""
        shape_left, dtype_left = left
        shape_right, dtype_right = right
        return (
            (shape_right is None or shape_left == shape_right)
            and (dtype_right is None or dtype_left == dtype_right)
        )


@dispatch
def f(x: NDArray[None, np.int32]):
    print("An int array!")

@dispatch
def f(x: NDArray[None, np.float64]):
    print("A float array!")

print("BEGIN f")
f(np.ones((3, 3), np.int32))
f(np.ones((2, 2), np.float64))
print("END f")

@dispatch
def g(x: np.typing.NDArray[np.int32]):
    print("An int array!")


@dispatch
def g(x: np.typing.NDArray[np.float64]):
    print("A float array!")

print("BEGIN g")
g(np.ones((3, 3), np.int32))
g(np.ones((2, 2), np.float64))
print("END g")

and the corresponding output

BEGIN f
An int array!
A float array!
END f
BEGIN g
/usr/local/lib/python3.11/dist-packages/plum/signature.py:203: UserWarning: Could not resolve the type hint of `numpy.ndarray[typing.Any, numpy.dtype[numpy.int32]]`. I have ended the resolution here to not make your code break, but some types might not be working correctly. Please open an issue at https://github.com/wesselb/plum.
  annotation = resolve_type_hint(p.annotation)
/usr/local/lib/python3.11/dist-packages/plum/type.py:261: UserWarning: Could not resolve the type hint of `numpy.ndarray[typing.Any, numpy.dtype[numpy.int32]]`. I have ended the resolution here to not make your code break, but some types might not be working correctly. Please open an issue at https://github.com/wesselb/plum.
  return _is_faithful(resolve_type_hint(x))
/usr/local/lib/python3.11/dist-packages/plum/type.py:261: UserWarning: Could not determine whether `numpy.ndarray[typing.Any, numpy.dtype[numpy.int32]]` is faithful or not. I have concluded that the type is not faithful, so your code might run with subpar performance. Please open an issue at https://github.com/wesselb/plum.
  return _is_faithful(resolve_type_hint(x))
/usr/local/lib/python3.11/dist-packages/plum/signature.py:203: UserWarning: Could not resolve the type hint of `numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]`. I have ended the resolution here to not make your code break, but some types might not be working correctly. Please open an issue at https://github.com/wesselb/plum.
  annotation = resolve_type_hint(p.annotation)
/usr/local/lib/python3.11/dist-packages/plum/type.py:261: UserWarning: Could not resolve the type hint of `numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]`. I have ended the resolution here to not make your code break, but some types might not be working correctly. Please open an issue at https://github.com/wesselb/plum.
  return _is_faithful(resolve_type_hint(x))
/usr/local/lib/python3.11/dist-packages/plum/type.py:261: UserWarning: Could not determine whether `numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]` is faithful or not. I have concluded that the type is not faithful, so your code might run with subpar performance. Please open an issue at https://github.com/wesselb/plum.
  return _is_faithful(resolve_type_hint(x))
Traceback (most recent call last):
  File "/tmp/p.py", line 72, in <module>
    g(np.ones((3, 3), np.int32))
  File "/usr/local/lib/python3.11/dist-packages/plum/function.py", line 342, in __call__
    self._resolve_pending_registrations()
  File "/usr/local/lib/python3.11/dist-packages/plum/function.py", line 237, in _resolve_pending_registrations
    self._resolver.register(subsignature)
  File "/usr/local/lib/python3.11/dist-packages/plum/resolver.py", line 58, in register
    existing = [s == signature for s in self.signatures]
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/plum/resolver.py", line 58, in <listcomp>
    existing = [s == signature for s in self.signatures]
                ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/plum/util.py", line 132, in __eq__
    return self <= other <= self
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/plum/signature.py", line 132, in __le__
    [TypeHint(x) <= TypeHint(y) for x, y in zip(self_types, other_types)]
  File "/usr/local/lib/python3.11/dist-packages/plum/signature.py", line 132, in <listcomp>
    [TypeHint(x) <= TypeHint(y) for x, y in zip(self_types, other_types)]
     ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/beartype/door/_doormeta.py", line 148, in __call__
    _HINT_KEY_TO_WRAPPER.cache_or_get_cached_func_return_passed_arg(
  File "/usr/local/lib/python3.11/dist-packages/beartype/_util/cache/map/utilmapbig.py", line 231, in cache_or_get_cached_func_return_passed_arg
    value = value_factory(arg)
            ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/beartype/door/_doormeta.py", line 220, in _make_wrapper
    wrapper_subclass = get_typehint_subclass(hint)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/beartype/door/_doordata.py", line 108, in get_typehint_subclass
    raise BeartypeDoorNonpepException(
beartype.roar.BeartypeDoorNonpepException: Type hint numpy.ndarray[typing.Any, numpy.dtype[numpy.int32]] invalid (i.e., either PEP-noncompliant or PEP-compliant but currently unsupported by "beartype.door.TypeHint").

Thanks!

Can't use @dispatch on methods with types inheriting from Generic

Simple example:

from typing import TypeVar, Generic

from plum import dispatch

T = TypeVar('T')


class FooBar:
    pass


class Foo(Generic[T]):
    pass


class Bar:
    def __init__(self):
        pass

    @dispatch
    def some_method(self, foo: Foo[FooBar]):
        pass


Bar().some_method(Foo())

This will fail with an error as follows:

line 420, in ptype
    raise RuntimeError(f'Could not convert "{obj}" to a type.')
RuntimeError: Could not convert "__main__.Foo[__main__.FooBar]" to a type.

Is this functionality missing or do I need to add any more assistance in for plum to get this working?

Plum 2

What is This?

This issue proposes possible major improvements for a version two of the package. The goal is to start a discussion around these proposals. If you agree or disagree with some or all of changes, or think that other improvements are missing, then please do share your thoughts by commenting below. :) Tagging @PhilipVinc, @tbsexton, @leycec, and @seeM here since they have been involved in recent relevant discussions.

None of the proposed changes should change current behaviour, thus maintaining backwards compatibility. I'll be continuously updating and adding to this issue over time.

The list of proposed improvements is as follows:

Instance-Check-Based Dispatch

Instead of performing dispatch based on the types of the arguments, perform dispatch entirely using isinstance, hence fully eliminating the need to use type_of. See this proposal by @tbsexton. The massive benefit of doing things this way is it becomes much easier to support types like Literals (#66), Protocols (#28), numpy arrays with a specific number of dimensions (#10), and PyTrees (Stheno #21).

The big downside of this change is that performance will take a hit, since instance checks will need to happen every time dispatch happens. However, by using @beartype (@leycec), instance checks will be blazingly fast, so the performance hit should be minimal. Moreover, caching is still possible for functions with a so-called faithful dispatch tree (see here), so we should get the best of both worlds!

Although dispatch will not rely on type_of anymore, other functionality of the package still does, such as promotion. Hence, a type_of function is still necessary.

Postponed Evaluation of Types

@beartype now includes beartype.peps.resolve_pep538(), which should handle PEP538-style postponed types for us. A massive thanks to @leycec for this!! This allows us to get rid of all use of inspect, which I believe currently is causing performance issues (Stheno #25). Deferring the responsibility of handling postponed evaluation of types to @beartype should significantly simplify the internals of the package.

Wrapping of Types

Internally Plum currently wraps types in its own types. We should get rid of those entirely and use types from typing. This does pose a challenge for some more advanced custom types, such as PromisedType.

Compatibility with Modern Tooling

Whatever Plum does should be compatible with modern tooling. Think mypy, pylance (#51), and possible others.

How to use @dispatch with @staticmethod inside a class?

Hey,

consider the following code snippet:

   @dispatch
    def encode(
        data: MyDataType,
        autodetect: bool=False,
        encodings:Optional[Dict[str, List[str]]] =None,
    ) -> Optional[MyDataType]:
        """Encode a MyDataType object"""
        ... do soemthing here

and

    @staticmethod
    def encode(
        data: MyOtherDataType,
        autodetect: Optional[Dict] = None,
        encodings: Optional[Dict[str, Dict[str, List[str]]]] = None,
    ) -> None:
 .... do something here

So I have two staticmethods inside a class and I want to use dispatch on them.
When I try to run the code above it always fails with:

/my/home/dir/plum/function.py:226 in plum.function.ClassFunction.__call__    โ”‚
โ”‚                                                                                           โ”‚
โ”‚ [Errno 2] No such file or directory: 'my/home/dir/plum/function.py'  (What is this even trying to do here??)
...
ResolutionError: Promise `Promise()` was not kept.

I read in your docs, that the dispatch decorator should be the outermost decorator. But doing so with staticmethod, will also result in an error indicating that the dispatch operator does not get a callable object (but a staticmethod object) passed.

Any ideas whats going on here and how to resolve this?
Many thanks ;)

Disambiguation with unions fails

I get the following (on master 7e9b865)

from plum import dispatch

@dispatch
def f(x: {object, int}, y: int):
    return x + y

@dispatch
def f(x: int, y: object):
    return x * y

print(f(3, 3.0)) # 9.0
print(f(3.0, 3)) # 6.0
print(f(3, 3)) # Should be 6. Error, fails to disambiguate

The multipledispatch library does correctly dispatch in this case. I don't know if this is a bug here, or it's not intended to be supported.

Keyword args not supported

First, I really like this. I was getting pretty confused when I kept getting "plum.function.NotFoundLookupError" and couldn't find issue with the simple class I'd made. Randomly, I forgot to provide a keyword for the arg and it worked. Am I correct that keyword args are not supported?

NotFoundLookupError when using dispatch.multi in class method

master 7e9b865

from plum import dispatch

@dispatch.multi((int, str), (int, int))
def f(x: int, y: {int, str}):
    print(f'{x}, {y}')

class C():
    @dispatch
    def g(self, x: int, y: int):
        print(f'{x}, {y}')

    @dispatch
    def g(self, x: int, y: str):
        print(f'{x}, {y}')

    @dispatch.multi((int, str), (int, int))
    def h(self, x: int, y: {int, str}):
        print(f'{x}, {y}')

f(3, 4) # OK

c = C()
c.g(3, 4) # OK
c.g(3, 'cat') # OK

c.h(3, 4) # NotFoundLookupError
c.h(3, 'cat') # NotFoundLookupError

I did not yet try this on the latest released branch.

Async support

Any possible support for applying plum dispatch on async methods?

One-liner register

I usually find myself doing something like this:

from somepackage import somefun

@dispatch
def f(x: float):
    return somefun(x)

is there a one liner way to do this that does not require defining f? Something like:

dispatch("f", (float, ), somefun)

As far as I can tell there is no way to do it because you allways take the fuction name from the function, right?

Does not work with future annotations

A resolution error is raised when you make a promise using annotations.
It works if you stop importing the annotations package and use quotation marks.

The following code breaks.
If you comment out the first line, the code works.

from __future__ import annotations
from plum import dispatch


class Displacement:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    @dispatch
    def __add__(self, other: "Displacement") -> "Displacement":
        return Displacement(self.x + other.x, self.y + other.y)

    @dispatch
    def __add__(self, other: int) -> "Displacement":
        return Displacement(self.x + other, self.y + other)


if __name__ == "__main__":
    d1 = Displacement(1, 2)
    d2 = Displacement(3, 4)
    print(d1 + d2)    

Traceback (most recent call last):
  File "C:\Users\gaspa\dev\python\py-chess\src\geometry_copy.py", line 22, in <module>
    print(d1 + d2)
  File "plum\function.py", line 585, in plum.function._BoundFunction.__call__
  File "plum\function.py", line 509, in plum.function.Function.__call__
  File "plum\function.py", line 373, in plum.function.Function._resolve_pending_registrations
  File "C:\Users\gaspa\.virtualenvs\py-chess-YZvcAOQM\lib\site-packages\plum\type.py", line 436, in is_object
    return t.get_types() == (object,)
  File "C:\Users\gaspa\.virtualenvs\py-chess-YZvcAOQM\lib\site-packages\plum\type.py", line 259, in get_types
    return ptype(self.resolve()).get_types()
  File "C:\Users\gaspa\.virtualenvs\py-chess-YZvcAOQM\lib\site-packages\plum\resolvable.py", line 41, in resolve
    raise ResolutionError(f"Promise `{self!r}` was not kept.")
plum.resolvable.ResolutionError: Promise `ForwardReferencedType(name="'Displacement'")` was not kept.

Process finished with exit code 1

Parametric classes usage

Hi,
I would like to use parametric classes.
I'd like to implement something akin to (Julia)

struct DoubledSpace{T} where T
   space::T
end

double_int = DoubledSpace(1)
double_float = DoubledSpace(1.0)

# and dispatch on the parameter
size(a::DoubledSpace{Int}) = ...
size(a::DoubledSpace{Float64}) = ...

(The parameter class will be a custom class, but anyhow...)
I don't care about dispatch cost as all this is inside jax jitted function.

Anyhow, I tried to implement this by doing

from plum impor dispatch, parametric

@parametric
class doubledspace:
   def __init__(self, space):
      print("initializing doubledspace")
      self.space = space

a = doubledspace(1.0)

but the init never gets called.

I also tried to dive into your code in plum/parametric.py and instrument it a bit:

def parametric(Class):
    """A decorator for parametric classes."""
    subclasses = {}

    def __new__(cls, *ps):
        print(f"executing {cls.__name__}.__new__(*{ps})")
        # Only create new subclass if it doesn't exist already.
        print(f"  SubClasses are:")
        for p in subclasses:
            print(f"  \t{p}")
        print("  ---------------")
        if ps not in subclasses:
            print(f"  it is not in sublcasses")

            def __new__(cls, *args, **kw_args):
                print(f"    Calling the original {Class.__name__}.__new__(*{args}, **{kwargs}) but dropping them")
                return Class.__new__(cls)

            # Create subclass.
            name = Class.__name__ + "[" + ",".join(str(p) for p in ps) + "]"
            SubClass = type.__new__(
                CovariantMeta,
                name,
                (ParametricClass,),
                {"__new__": __new__, "_is_parametric": True},
            )
            SubClass._type_parameter = ps[0] if len(ps) == 1 else ps
            SubClass.__module__ = Class.__module__

            print(f"  Creating a new subclass called {name}")
            # Attempt to correct docstring.
            try:
                SubClass.__doc__ = Class.__doc__
            except AttributeError:  # pragma: no cover
                pass

            subclasses[ps] = SubClass

        print(f"  returning the subclass {subclasses[ps]}")
        return subclasses[ps]

and the output of the snippet before is

โžœ python -i test.py                    
now constructing
executing doubledspace.__new__(*(1,))
  SubClasses are:
  ---------------
  it is not in sublcasses
  Creating a new subclass called doubledspace[1]
  returning the subclass <class '__main__.doubledspace[1]'>

So it seems that even the new method

            def __new__(cls, *args, **kw_args):
                print(f"    Calling the original {Class.__name__}.__new__(*{args}, **{kwargs}) but dropping them")
                return Class.__new__(cls)

is never called. As well as my custom init.

Can you give me a few pointers on what i should do?

Support for typing.Sequence[int]

@dispatch
async def get_updates(version: Sequence[int]) -> List[Update]:

For the code above I got
NotImplementedError: There is currently no support for "typing.Sequence[int]". Please open an issue at https://github.com/wesselb/plum/issues

Pylance confused with @dispatch functions

If I use pylance to type check functions in Python, all multiple-dispatched functions are shown as

Function declaration "name_of_function" is obscured by a declaration of the same name

I wonder if this can be fixed ๐Ÿค”

Issue when the output is the a Generator

It seems that the plum has issue in dispatch function when the output is Generator

I have the sample code as

from typing import Generator, List
from plum import dispatch
class Convert():
    @dispatch
    def run(self, input: List[int, ]) -> Generator[int, None, None]:
        for item in input:
            yield int(item)

    @dispatch
    def run(self, input: List[float, ]) -> Generator[float, None, None]:
        for item in input:
            yield float(item)

I got

Traceback (most recent call last):
  File "/Users/zwu/miniconda3_x86/envs/exs-orionbssfloes/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-26-2a8843a2d5ae>", line 1, in <module>
    class Convert():
  File "<ipython-input-26-2a8843a2d5ae>", line 3, in Convert
    def run(self, input: List[int, ]) -> Generator[int, None, None]:
  File "/Users/zwu/miniconda3_x86/envs/exs-orionbssfloes/lib/python3.7/site-packages/plum/dispatcher.py", line 43, in __call__
    signature, return_type = extract_signature(method)
  File "plum/function.py", line 120, in plum.function.extract_signature
  File "/Users/zwu/miniconda3_x86/envs/exs-orionbssfloes/lib/python3.7/site-packages/plum/type.py", line 409, in ptype
    f'There is currently no support for "{obj}". '
NotImplementedError: There is currently no support for "typing.Generator[int, NoneType, NoneType]". Please open an issue at https://github.com/wesselb/plum/issues

[BUG] Default arguments are not supported with Dispatch (regression from 0.2.5)

This was working in 0.2.5,

import plum
from plum import dispatch

@dispatch
def test(a:int, b: int = 1):
	print('int, int')

@dispatch
def test(a:float, b: int = 1):
	print('float, int')

In [33]: 

In [33]: test(1,1)
---------------------------------------------------------------------------
NotFoundLookupError                       Traceback (most recent call last)
<ipython-input-33-9b86683066a3> in <module>
----> 1 test(1,1)

~/Documents/pythonenvs/netket_env/lib64/python3.8/site-packages/plum/function.cpython-38-x86_64-linux-gnu.so in plum.function.Function.__call__()

~/Documents/pythonenvs/netket_env/lib64/python3.8/site-packages/plum/function.cpython-38-x86_64-linux-gnu.so in plum.function.Function.resolve_method()

~/Documents/pythonenvs/netket_env/lib64/python3.8/site-packages/plum/function.cpython-38-x86_64-linux-gnu.so in plum.function.Function.resolve_signature()

NotFoundLookupError: For function "test", signature Signature(builtins.int, builtins.int) could not be resolved.

In [34]: test(1)
int, int

Installation of plum-dispath as a requirement via setup.py

We are having trouble to use plum-dispatch as a dependency in my own project.
We include plum-dispatch as a requirement in our setup.py, but unfortunately the automatic installation fails with

Searching for plum-dispatch
Reading https://pypi.org/simple/plum-dispatch/
Downloading https://files.pythonhosted.org/packages/aa/ce/3467306a9bd3e3e7fe4450f87f283e9f997f0646a410d716296db9bfa3ad/plum-dispatch-1.0.0.tar.gz#sha256=766da66a93d0f00911777175dc7a43235ee3f18ead34c73977e710f732a918fd
Best match: plum-dispatch 1.0.0
Processing plum-dispatch-1.0.0.tar.gz
Writing /tmp/easy_install-s90fi7cu/plum-dispatch-1.0.0/setup.cfg
Running plum-dispatch-1.0.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-s90fi7cu/plum-dispatch-1.0.0/egg-dist-tmp-9fh6ov4j
warning: the 'license_file' option is deprecated, use 'license_files' instead
error: Setup script exited with error: unknown file type '.py' (from 'plum/function.py')
Error: Process completed with exit code 1.

I am no expert with setuptools, but it seems to me as if license_file instead of license_files in the setup.cfg is causing issues.

Installing plum-dispatch before hand via pip pip install plum-dispatch is a work around without problems though.

Conda recipe

Hi again!
Thanks for the release!

As we decided to rely on plum inside of NetKet, and we provide a conda recipe, we would like plum to be available in conda.

I already set up the recipe in this PR to conda-forge conda-forge/staged-recipes#14653 .
If you want to be listed as a maintainer for the codna recipe (typicaly you have nothing to do, as new releases on PyPi will be picked up automatically, unless you change the dependencies in a sensible way), let me know (you should post a comment on the PR saying that 'I aknowledge I will be listed as a maintainer of this recipe').

I'm struggling with one error due to the way you package the cython extension...
What you are doing is that you are compiling function.py as a cython module, right?

List of "projects using plum"?

It would be useful to have a list of projects that use plum, or even other Python MD libraries.
Best would be OSS so we can see exactly how it is used.

(Maybe there is a way to do a pypi reverse-dependency search?)

Some things that such a list might help with:

  • My main motivation: Say I want to introduce MD in a garden-variety existing large project. The response of the other
    devs will be "no". It would be useful to point to code bases using MD that didn't cause the end of the world, or even helped.
  • Even if I think plum is the best MD library for a particular project, but the best large example uses, say multipledispatch,
    it's worth knowing. If I can convince others (and most importantly myself!) that this is was a good choice, it is a relatively small jump
    to say plum is better in this case. The harder question is whether MD is useful at all.
  • What are the design problems and choices peculiar to Python? For example, integration with Python's OO model.

Add support for "|" operator in union types

PEP 604 allowed for the writing of union types as x | y
but this break in plum.dispatch:

@plum.dispatch
def foo(x: int | float):
    return x + 1
>>> foo(3.14)
RuntimeError: Could not convert "int | float" to a type.

Accept `None` as type annotation

Hi! Thank you so much for this library, I've been looking for something like this for a long time!

I would like to suggest accepting None as a valid type in Plum. For instance, this snippet does not work:

from plum import dispatch

def foo(x: None) -> None:
    return x

print(foo(None)) # raises: RuntimeError: Could not convert "None" to a type.

whereas this one works as expected:

from plum import dispatch

NoneType = type(None)

@dispatch
def foo(x: NoneType) -> NoneType:
    return x

print(foo(None)) # prints: None

Since type-annotating None's type as None is the standard way to do it, I think this simple enhancement would make dispatching a lot more intuitive in this case.

I would also love to submit a PR if you agree, I'm just not familiar with Plum's implementation. It seems to me that adding

if obj is None:
    return Type(type(None))

just before the RuntimeError should fix this, am I correct?

[FR] Support 'typing.Any' as a fallback

Support a default method fallback with signature 'Any' or when no signature is defined.

This is usefull for defining functions like

@dispatch
def is_fancy_object(x)
   return False

@dispatch
def is_fancy_object(x: MyFancyClass)
  return True

which is a common pattern in Julia

Support for multiple dispatch with constructors

Thanks for putting this package together. I noticed that multiple dispatch does not work on constructors. Here is a MWE:

from plum import dispatch

class MyClass:
    
    @dispatch
    def __init__(self, x: int, y: int):
        self.x = x
        self.y = y
        
MyClass(x=1, y=2)

[QUESTION] How to see all methods?

If I define a few dispatched methods:

@dispatch
def foo(x: str):
    print("String")


@dispatch
def foo(x: int):
    print("Int")


@dispatch
def foo(x: float):
    print("Float")


foo  # <function <function foo at 0x7f897111ca60> with 3 method(s)>

It's clearly telling me that there's three methods for foo, but not what types it can dispatch on. What I think would be nice is like in Julia's help system ?, we can search for the method name and get a list of the concrete implementations. Is something like that possible for plum too?

Non-supported typings as ReturnType.

There seems to be an issue with (yet) non-supported typings, even if the type in question is only a return type.

With the latest version (via pip) and python 3.8.5, this example code

from plum import dispatch
from typing import Dict

@dispatch
def foo(x: int, y: str) -> Dict[int, str]:
    return {x: y}

print(foo(1, "1"))

yields the following error:

Traceback (most recent call last):
  File "main.py", line 5, in <module>
    def foo(x: int, y: str) -> Dict[int, str]:
  File "/home/bob/dev_env/lib/python3.8/site-packages/plum/dispatcher.py", line 38, in __call__
    signature, return_type = extract_signature(method)
  File "plum/function.py", line 81, in plum.function.extract_signature
  File "/home/bob/dev_env/lib/python3.8/site-packages/plum/type.py", line 353, in ptype
    f'There is currently no support for "typing.{obj.__name__}". '
  File "/home/bob/anaconda3/lib/python3.8/typing.py", line 760, in __getattr__
    raise AttributeError(attr)
AttributeError: __name__

Is this expected behavior or is there any way to resolve this?

Support for `Protocol`s

Protocols were introduced in Python 3.8 by PEP 544, and it would be awesome if Plum could handle them.

A simple use case would look like this:

from typing import Any, Protocol
from plum import dispatch

class Named(Protocol):
    name: str

class Person:
    def __init__(self, name: str, age: int) -> None:
        self.name: str = name
        self.age: int = age

@dispatch
def greet(anyone: Any) -> None:
    print('Hi!')

@greet.dispatch
def _greet_named(named: Named) -> None:
    print(f'Hello, {named.name}')

me = Person('Ruan', 24)
greet(me)
# should print "Hello, Ruan"
# currently throws the following error:
# TypeError: Instance and class checks can only be used with @runtime_checkable protocols

Another use case is what I am actually interested in: using dataclasses. Dataclasses can be identified by a special attribute: __dataclass_fields__ (see python/mypy#8578), so that I could write

class Dataclass(Protocol):
    __dataclass_fields__: Dict[str, Any]

@dispatch
def do_something(dataclass_instance: Dataclass):
    ...

I have no idea how to proceed to implement this, but I'm interested in seeing this in Plum and would love to help in any way I can.
Just for the record, Dry Python Classes has this feature, we could take some inspiration from them.

redefinition error in type checkers

from plum import dispatch

@dispatch
def add_nums(num1: int, num2: int) -> int:
    return num1 + num2

@dispatch
def add_nums(num1: str, num2: str) -> int:
    return add_nums(int(num1), int(num2))

add_nums(10, 15)
add_nums('30', '25')

flake8 reports redefinition of unused add_nums from line 4 flake8(F811). The code works fine though. flake8 error

Another issue (I was using VS Code), editor always shows the last defined function in the hint. If there are multiple overloads. VS Code hints

Do not trigger type_of on return type hint

I am working with nested NumPy arrays (arrays of arrays) and I want to dispatch based on its nesting level.

from plum import dispatch

def something(a: NestedArray[0]):
    pass

def something(a: NestedArray[1]):
    pass

I'm using the following hook for that:

from plum import parametric, type_of

@parametric(runtime_type_of=True)
class NestedArray(np.ndarray):
    """A type for recursive numpy arrays (array of arrays) where the type parameter specifies the nesting level."""
    pass


@type_of.dispatch
def type_of(x: np.ndarray):
    level = 0
    while isinstance(x.flat[0], np.ndarray):
        level += 1
        x = x.flat[0]

    return NestedArray[level]

It works like a charm. But I run into a really annoying side-effect when I have a dispatched function with numpy.ndarray as a return type hint:

@dispatch
def to_numpy(a) -> numpy.ndarray:
    ...
    return numpy.zeros(4) # returns a ndarray, zeros for example

This code crashes because it runs type_of on the returning object, which returns a Type. Then compares it against the type hint of to_numpy, which fails.

...
  File "plum/function.py", line 537, in plum.function.Function.__call__
  File "plum/function.py", line 173, in plum.function._convert
  File "plum/function.py", line 537, in plum.function.Function.__call__
  File "/home/mofeing/Develop/rosnet/.venv/lib/python3.8/site-packages/plum/promotion.py", line 32, in convert
    return _convert.invoke(type_of(obj), type_to)(obj, type_to)
  File "plum/function.py", line 552, in plum.function.Function.invoke.wrapped_method
  File "/home/mofeing/Develop/rosnet/.venv/lib/python3.8/site-packages/plum/promotion.py", line 43, in _convert
    if type_from <= type_to:
  File "/home/mofeing/Develop/rosnet/.venv/lib/python3.8/site-packages/plum/util.py", line 43, in __ge__
    return other.__le__(self)
TypeError: descriptor '__le__' requires a 'numpy.ndarray' object but received a 'Type'

If I remove the return type hint, it works.

I guess this is a bug because there is no way (yet) to dispatch based on return type. Also because the function should be returning the object, not the hooked type.

Publish `plum-dispatch-purepython` without C extensions to PyPI

If my benchmarks are accurate (using benchmark.py), cythonising plum.function is at best 2x faster. For our use-case โ€” a high-level framework on top of lower-level libraries like numpy and torch โ€” this isn't worth the loss of stack trace and inspect support.

@wesselb would you consider publishing a build of plum without C extensions to PyPI?

One way to go about this is for setup.py to only add to ext_modules if an environment variable is set (e.g. USE_CYTHON), and for the GitHub Action to do an additional build and publish with that variable set. If you're on board, I'm happy to help where I can!

Tag a new release on PyPi?

Hi!
I recently discovered your package and really like it.

I've seen you have recently added support for inheriting the dispatch types from the type annotations. That's very nice!
As I'd like to use it in one of my packages, could you tag a new release on PyPi?

Thanks!

Naming of args

Hi - this looks like a cool package so just installed to play around with it.

I noticed that the following (dumb / trivial) example does not work for the 2nd pair of print statements (where I supply names for the args). Is there a design reason for this? I like always supplying these so others can understand what my code is doing (i.e. in the example which number is the numerator and which is the denominator, without having to click thru to the function def).

Thanks!

image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.