coady / multimethod Goto Github PK

1. If a method is decorated with staticmethod, classmethod or abstractmethod, the program will not execute correctly.
2. The argument (1, 1) can not match the generic type Collection[Real], but [2, 3] and np.array([5, 5]) can. [2, '3'] can also match but np.array([5, '5']) and ['2', '3'] can't. (Real from numbers, Collection from typing )

Issue for Pull #35. The dispatch resolution may match types even though it's clear the parameter parity is mismatched.

@multimethod
def temp(x: int, y: float):
    return "int, float"

@multimethod
def temp(x: bool):
    return "bool"

assert temp(True, 1.0) == "int, float"  # DispatchError

If an @overloaded function is decorated with the return annotation, then the following error is raised:

File "multimethod/multimethod/__init__.py", line 260, in <genexpr>
if all(predicate(arguments[name]) for name, predicate in func.__annotations__.items()):
KeyError: 'return'

Currently multimethod cannot recognize type contaning ellipsis such as tuple[int, ...]
However it is a standard way to declare variable-length tuple
ref: https://docs.python.org/3/library/typing.html#typing.Tuple

Maybe add another check in __init__.py:subtype.__subclasscheck__ would help

Hello,

As I was using this library for multi dispatching for functions, I was wondering if the same functionality can be added to methods as well. It would look some like this:

class Test:

    @multidispatchmethod
    def meth(self, *args, **kwargs):
        ...

    @meth.register(int, float)
    def _(self, x: int, y: float):
        ...

    @meth.register(str, str)
    def _(self, x: str, y: str):
        ...

I would be open to work on this eventual feature 😄 .

Hi,
I noticed that removing __init__.py from the tests folder causes

========================================================== FAILURES ==========================================================
__________________________________________________ test_dispatch_exception ___________________________________________________

    def test_dispatch_exception():
        @multimethod
        def temp(x: int, y):
            return "int"
    
        @multimethod
        def temp(x: int, y: float):
            return "int, float"
    
        @multimethod
        def temp(x: bool):
            return "bool"
    
        @multimethod
        def temp(x: int, y: object):
            return "int, object"
    
        with pytest.raises(DispatchError, match="test_methods.py"):
            # invalid number of args, check source file is part of the exception args
            temp(1)
>       assert temp(1, y=1.0) == "int"
E       AssertionError: assert 'int, object' == 'int'
E         - int
E         + int, object

test_methods.py:293: AssertionError
================================================== short test summary info ===================================================
FAILED test_methods.py::test_dispatch_exception - AssertionError: assert 'int, object' == 'int'
================================================ 1 failed, 10 passed in 0.09s ================================================

My python version is 3.10, and I am on the latest pypi release with a manual application of 857e4e9

The @multimethod decorator, at least, seems to call the __iter__ method, if the class has one. It only seems to happen if one of the arguments is iterable (like list or a tuple), but TBH I'm not sure precisely what's going on. Here's a semi-minimal repro (tested Python 3.9, multimethod 1.5):

from multimethod import multimethod

class Foo(object):
	@multimethod
	def __init__( self, arg:int ):
		print("Constructor 1")
	@multimethod
	def __init__( self, arg:list[int] ):
		print("Constructor 2")

	def __iter__(self):
		print("Why is this called?")
		#Returns `None`, causing crash inside multimethod . . .

foo = Foo(0)

Neither constructor runs. Instead, the __iter__ runs, and that method probably can't even do anything sensible since the class hasn't even been constructed yet.

I'm also interested in workarounds.

Hello! I am trying to use it with pydantic.BaseModel and I am facing the issue:

class Mapper:
    @multimethod
    def map(self, source, target_type):
        ...

    @map.register
    def _(self, source: ProfileDTO, target_type: typing.Type[UserDTO]) -> UserDTO:
        # logic
        ...

Where ProfileDTO and UserDTO are pydantic models.

Then I am trying to use this class:

mapper = Mapper()
mapper.map(profile, UserDTO)

But the library doesn't use the registered method, because it recognizes typing.Type[UserDTO] construction like pydantic.main.ModelMetaclass.

Is there some solution for this?

Hi,
in my application I would like to dispatch on a sequence of callable.

To this end, I slightly patched your test_callable test

diff --git a/tests/test_subscripts.py b/tests/test_subscripts.py
index 7a293e1..cf83f89 100644
--- a/tests/test_subscripts.py
+++ b/tests/test_subscripts.py
@@ -1,6 +1,6 @@
 import sys
 import pytest
-from typing import Callable, Generic, List, TypeVar
+from typing import Callable, Generic, List, Sequence, TypeVar
 from multimethod import multimethod, subtype, DispatchError
 
 
@@ -74,10 +74,15 @@ def test_callable():
     def _(arg: int):
         return 'int'
 
+    @func.register
+    def _(arg: Sequence[Callable[[bool], bool]]):
+        return arg[0].__name__ + "0"
+
     tp = subtype(func.__annotations__['arg'])
     assert not issubclass(tp.get_type(f), tp.get_type(g))
     assert issubclass(tp.get_type(g), tp.get_type(f))
     with pytest.raises(DispatchError):
         func(f)
     assert func(g) == 'g'
+    assert func([g]) == 'g0'
     assert func(h) is ...

Such new test fails with

_______________________________________________________ test_callable ________________________________________________________

    def test_callable():
        def f(arg: bool) -> int:
            ...
    
        def g(arg: int) -> bool:
            ...
    
        def h(arg) -> bool:
            ...
    
        @multimethod
        def func(arg: Callable[[bool], bool]):
            return arg.__name__
    
        @func.register
        def _(arg: Callable[..., bool]):
            return ...
    
        @func.register
        def _(arg: int):
            return 'int'
    
        @func.register
        def _(arg: Sequence[Callable[[bool], bool]]):
            return arg[0].__name__ + "0"
    
        tp = subtype(func.__annotations__['arg'])
        assert not issubclass(tp.get_type(f), tp.get_type(g))
        assert issubclass(tp.get_type(g), tp.get_type(f))
        with pytest.raises(DispatchError):
            func(f)
>       assert func(g) == 'g'

test_subscripts.py:86: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/lib/python3.10/dist-packages/multimethod/__init__.py:310: in __call__
    func = self[tuple(func(arg) for func, arg in zip(self.type_checkers, args))]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = {(<class 'multimethod.typing.Callable[[bool], bool]'>,): <function test_callable.<locals>.func at 0x7fcd6081d3f0>, (<c...'multimethod.typing.Sequence[typing.Callable[[bool], bool]]'>,): <function test_callable.<locals>._ at 0x7fcd6081d630>}
types = (<class 'function'>,)

    def __missing__(self, types: tuple) -> Callable:
        """Find and cache the next applicable method of given types."""
        self.evaluate()
        if types in self:
            return self[types]
        groups = collections.defaultdict(list)
        for key in self.parents(types):
            if key.callable(*types):
                groups[types - key].append(key)
        keys = groups[min(groups)] if groups else []
        funcs = {self[key] for key in keys}
        if len(funcs) == 1:
            return self.setdefault(types, *funcs)
        msg = f"{self.__name__}: {len(keys)} methods found"  # type: ignore
>       raise DispatchError(msg, types, keys)
E       multimethod.DispatchError: ('func: 0 methods found', (<class 'function'>,), [])

/usr/local/lib/python3.10/dist-packages/multimethod/__init__.py:304: DispatchError
================================================== short test summary info ===================================================
FAILED test_subscripts.py::test_callable - multimethod.DispatchError: ('func: 0 methods found', (<class 'function'>,), [])
================================================ 1 failed, 3 passed in 0.08s =================================================

Notice that the failing line is the call func(g) and not my newly added call func([g]).

My python version is 3.10, and I am on the latest pypi release with a manual application of 857e4e9

Thanks @coady for this package. It has been quite useful to me! Have you ever releasing it on conda-forge so that other packages can depend on it?

Also, I've been trying to decide which package to use between this one and multipledispatch (https://github.com/mrocklin/multipledispatch). I gather that the latter does not supprt keyword arguments which multimethod seems to understand. Appart from that is there a reason to choose one or another?

I'm trying to use @multimethod in combination with Protocol. But at runtime I get this error

TypeError: Protocols with non-method members don't support issubclass()

@runtime_checkable
class SenderContext(Protocol):
    @property
    def cancellation_token(self) -> bool:
        ...

    def send(self, target: PID, message: Any) -> None:
        raise NotImplementedError("Should Implement this method")


@multimethod
def subscribe(self, msg_type: type,
              context: SenderContext) -> None:

How can I fix this error? If I remove the property cancellation_token, then everything works fine. But I need a property cancellation_token.

Hi! I just found this library and really like it so far, but I miss having the docstring available in the tooltip when calling a multimethod in a Jupyter Notebook.

Any clue how difficult it would be to fix the mypy reported return type in the case of TypeVar annotations? See this link for more details (or the example below):
CadQuery/cadquery#379 (comment)

from typing import TypeVar
from multimethod import multimethod

T0 = TypeVar('T0', bound='Shape')
T1 = TypeVar('T1', bound='MMShape')


class Shape:
    def set_scale(self: T0, scale: float) -> T0:
        self.scale = scale
        return self


class MMShape:
    @multimethod
    def set_scale(self: T1, scale: float) -> T1:
        self.scale = scale
        return self


s0 = Shape().set_scale(1.0)
reveal_type(s0)

s1 = MMShape().set_scale(1.0)
reveal_type(s1)

> mypy chain_test.py
chain_test.py:22: note: Revealed type is 'chain_test.Shape'
chain_test.py:25: note: Revealed type is 'Any'

Hi,

The following example produces an error:
`
class A:
@multimethod
def m(self, c: int, ii: Optional[int] = None):
print('1', c, ii)

@multimethod
def m(self, ff: float):
    print('2', ff)

obj = A()
obj.m(20)
`

when calling obj.m(20):
`
File "/usr/local/lib/python3.8/dist-packages/multimethod/init.py", line 184, in call
return self[tuple(map(self.get_type, args))](*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/multimethod/init.py", line 180, in missing
raise DispatchError(msg, types, keys)
multimethod.DispatchError: ('a: 0 methods found', (<class 'main.A'>, <class 'int'>), [])

Process finished with exit code 1
`

Is there a way around it?

Python 3.9, when I define such a custom type

PositiveInt = NewType('PositiveInt', int)

and use it to annotate methods, the code breaks:

Traceback (most recent call last):
  File "C:\Users\jared\Desktop\source_code\test_sdm.py", line 19, in <module>
    Hello(1, 2, 3)
  File "C:\Users\jared\AppData\Local\Programs\Python\Python39\lib\site-packages\multimethod\__init__.py", line 298, in __call__
    func = self[tuple(func(arg) for func, arg in zip(self.type_checkers, args))]
  File "C:\Users\jared\AppData\Local\Programs\Python\Python39\lib\site-packages\multimethod\__init__.py", line 284, in __missing__
    for key in self.parents(types):
  File "C:\Users\jared\AppData\Local\Programs\Python\Python39\lib\site-packages\multimethod\__init__.py", line 233, in parents
    parents = {key for key in self if isinstance(key, signature) and key < types}
  File "C:\Users\jared\AppData\Local\Programs\Python\Python39\lib\site-packages\multimethod\__init__.py", line 233, in <setcomp>
    parents = {key for key in self if isinstance(key, signature) and key < types}
  File "C:\Users\jared\AppData\Local\Programs\Python\Python39\lib\site-packages\multimethod\__init__.py", line 169, in __lt__
    return self != other and self <= other
  File "C:\Users\jared\AppData\Local\Programs\Python\Python39\lib\site-packages\multimethod\__init__.py", line 166, in __le__
    return len(self) <= len(other) and all(map(issubclass, other, self))
TypeError: issubclass() arg 2 must be a class or tuple of classes

Should we consider interpreting PositiveInt as PositiveInt.__supertype__ (which is int)?

I would like to dispatch the __init__ method of a superclass. But it does not work with multimethod & multimeta in the way I expected. Moreover, manual registration is impossible as I cannot access the parent __init__ method inside the class body. I have described more here.

Regarding multimethod.multimethod w/ arity of 1 consider this:

from multimethod import multimethod

def clone_of_default(x):
    raise NotImplementedError("no implementation for {} found".format(type(x))) 

clone_of = multimethod(clone_of_default)
clone_of[(int,)] = lambda x: int(x)

Adding a method like, which I tried first:

clone_of[int] = lambda x: int(x)

yields TypeError: 'type' object is not iterable.

What is the desired behaviour for arity 1?

multimethod         1.5
Python              3.9.13
flake8              6.0.0
flake8-annotations  2.9.1
Flake8-pyproject    1.2.2

Hi,

I am trying to use multimethod in this way:

from dataclasses import dataclass

@dataclass
class FileSystemObjectInterface:
    fs: 'FileSystemInterface'
    path: str

    @multimethod
    def __init__(self, fs: 'FileSystemInterface', path: str) -> None:
        self.fs = fs
        self.path = path.rstrip(self.fs.path_separator)

    @multimethod
    def __init__(self, fs: 'FileSystemInterface', path: 'FileSystemObjectInterface') -> None:
        self.fs = fs
        self.path = path.__fspath__()

    @multimethod
    def __init__(self, path: 'FileSystemObjectInterface') -> None:
        self.fs = path.fs
        self.path = path.path

However, when running this through flake8 (with flake8-annotations installed), I'm getting the following error:

flake8 file_system.py --dispatch-decorators=multimethod

file_system.py:38:5: F811 redefinition of unused '__init__' from line 33
file_system.py:43:5: F811 redefinition of unused '__init__' from line 38

Is this something that should be resolved in this library, or is this something I should report to flake8-annotations? Not very sure... So I'm posting it here because it looks like flake8-annotations already has support for overloaded operators.

FYI, changing it into .register makes the fault go away:

    @multimethod
    def __init__(self, fs: 'FileSystemInterface', path: str) -> None:
        ...

    @__init__.register
    def _(self, fs: 'FileSystemInterface', path: 'FileSystemObjectInterface') -> None:
        ...

    @__init__.register
    def _(self, path: 'FileSystemObjectInterface') -> None:
        ...

While upgrading to 1.8, I discovered that dispatching on the number of object arguments no longer works. It looks like this behavior was introduced in #23 and became part of v1.5.

Here is an example contrasting the handling of int and object type annotations:

In [1]: from multimethod import multimethod

In [2]: @multimethod
   ...: def foo(bar: int):
   ...:     return "one arg"
   ...: 

In [3]: @multimethod
   ...: def foo(bar: int, baz: int):
   ...:     return "two args"
   ...: 

In [4]: assert foo(1) == "one arg"

In [5]: assert foo(1, 2) == "two args"

In [6]: @multimethod
   ...: def foo_object(bar: object):
   ...:     return "one object arg"
   ...: 

In [7]: @multimethod
   ...: def foo_object(bar: object, baz: object):
   ...:     return "two object args"
   ...: 

In [8]: assert foo_object(1.0) == "one object arg"
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
File ~/.local/lib/python3.10/site-packages/multimethod/__init__.py:312, in multimethod.__call__(self, *args, **kwargs)
    311 try:
--> 312     return func(*args, **kwargs)
    313 except TypeError as ex:

TypeError: foo_object() missing 1 required positional argument: 'baz'

The above exception was the direct cause of the following exception:

DispatchError                             Traceback (most recent call last)
Input In [8], in <cell line: 1>()
----> 1 assert foo_object(1.0) == "one object arg"

File ~/.local/lib/python3.10/site-packages/multimethod/__init__.py:314, in multimethod.__call__(self, *args, **kwargs)
    312     return func(*args, **kwargs)
    313 except TypeError as ex:
--> 314     raise DispatchError(f"Function {func.__code__}") from ex

DispatchError: Function <code object foo_object at 0x7f3c34df64a0, file "<ipython-input-7-0fe682abbfe2>", line 1>

In the object case, get_types returns an empty tuple for both functions, so the second function replaces the first.

My suggestion is to update

One alternative would be to drop trailing objects only if they were introduced implicitly. This would allow users to add : object where it would be needed to differentiate from another function. Something like this:

    missing = object()  # instance to denote an omitted type hint
    annotations = [
        type_hints.get(param.name, missing)
        for param in inspect.signature(func).parameters.values()
        if param.default is param.empty and param.kind in positionals
    ]
    itr = itertools.dropwhile(lambda cls: cls is missing, reversed(annotations))
    return tuple((c if c is not missing else object for c in itr))[::-1]

There are probably other considerations. I'd appreciate hearing about any workarounds—especially if this is desired behavior and unlikely to change.

I'd like use the latest developments in a project (especially the partial literal support). Are there any plans to release 1.6?

This had me frustrated for ages until I thought to look at closed issues on the GitHub and realized what I was doing wrong. When I read "last" in the documentation, I thought last per function (because decorators read inside out, ie "last" being the first decorator listed). Not sure if anyone else would be as dumb as me, but maybe couldn't hurt to put a classmethod/staticmethod example in the documentation with multiple methods.

class Foo:
    @classmethod # <- don't do that
    @multimethod
    def bar(cls, x: str):
        print(x)

    @classmethod # <- or that
    @bar.register(str, str)
    def bar(cls, x: str, y: str):
        pass

    @classmethod # <- only put this @classmethod here on the final definition
    @bar.register(int, int)
    def bar(cls, x: int, y: int):
        print(x+1)

The following does result in a DispatchError:

from multimethod import multimethod
from typing import List

class Dummy:
    
    @multimethod
    def dummy(self, a: float,b : List[float]):
        
        pass

Dummy().dummy(1.0, [1.0])
Dummy().dummy(1.0, [])

Is it intended?

As stated in the title, I get an error on the first invocation. Subsequent invocations do work. I'm using your master branch and py3.8. Any pointers will be welcome.

Here is a MWE :

from typing import Optional, Tuple
from multimethod import multimethod

Point = Tuple[float, float]

class A(object):
    
    @multimethod
    def segment(self, p1: Point, p2: Point) -> "A":

        return self
    
a = A()
try:
    a.segment((0.,1.),(0.,1.))
except Exception as e:
    print(e) #why do I even get to this point?

a.segment((0.,1.),(0.,1.)) #2nd call works as expected

Hi! I tried to use multimethod.multimethod to define several class methods, it works for one argument, however it doesn't work for more.
For example:

from multimethod import multimethod


class Summary:
    @multimethod # line A
    def __init__(self, df: float, head: int, tail: int):
        self.df = df
        self.head = head
        self.tail = tail

    # block B
    @multimethod
    def __init__(self, df: dict, head: int, tail: int):
        self.df = df
        self.head = head
        self.tail = tail
    # end of block B

    @multimethod
    def __summary2(self, df: float):
        print(df * self.head * self.tail)

    @multimethod
    def __summary2(self, df: dict):
        for key in df:
            print(f'Name = {key}')
            print(df[key][self.head])
            print(df[key][self.tail])

    def summary(self):
        self.__summary2(self.df)


if __name__ == '__main__':
    print(f'Case No:1, Dictionary')
    df1 = [1, 2, 3, 10, 11, 12]
    df2 = [4, 5, 6, 13, 14, 15]
    df3 = [7, 8, 9, 16, 17, 18]
    dfs_dct = {'df1': df1, 'df2': df2, 'df3': df3}
    case_s1 = Summary(df=dfs_dct, head=0, tail=-1)
    print('case_s1')
    case_s1.summary()
    print(f'\n\n')


    print(f'Case No:2, List')
    df = 10.34
    case_s2 = Summary(df=df, head=0, tail=-1)
    print('case_s2')
    case_s2.summary()
    print(f'\n\n')

My environment: Python 3.7.4, Multimethod was installed from source, macOS 10.14.6.

Case No:1, Dictionary
Traceback (most recent call last):
  File "example03_3_args_built_in_types.py", line 40, in <module>
    case_s1 = Summary(df=dfs_dct, head=0, tail=-1)
  File "/path/to/multimethod/multimethod.py", line 108, in __call__
    return self[tuple(map(type, args))](*args, **kwargs)
  File "//path/to/multimethod/multimethod.py", line 104, in __missing__
    raise DispatchError("{}{}: {} methods found".format(self.__name__, types, len(keys)))
multimethod.DispatchError: __init__(<class '__main__.Summary'>,): 0 methods found

However, when I commented line A and block B, Case 1 is works.

Case No:1, Dictionary
case_s1
Name = df1
1
12
Name = df2
4
15
Name = df3
7
18



Case No:2, List
case_s2
Traceback (most recent call last):
  File "example03_3_args_built_in_types.py", line 50, in <module>
    case_s2.summary()
  File "example03_3_args_built_in_types.py", line 31, in summary
    self.__summary2(self.df)
  File "/path/to/multimethod/multimethod.py", line 108, in __call__
    return self[tuple(map(type, args))](*args, **kwargs)
  File "/path/to/multimethod/multimethod.py", line 104, in __missing__
    raise DispatchError("{}{}: {} methods found".format(self.__name__, types, len(keys)))
multimethod.DispatchError: __summary2(<class '__main__.Summary'>, <class 'float'>): 0 methods found

I also tried to use multimethod.multimeta and the func.register syntax, but the result was the same.

I tried to find a minimal example (tested with Python 3.10 and 3.11):

from multimethod import multimethod

@multimethod
def f(x: int | float): ...

@multimethod
def f(x: tuple | list): ...

│ ~/miniforge3/envs/py311/lib/python3.11/site-packages/multimethod/__init__.py:110 in                  │
│ __subclasscheck__                                                                                                    │
│                                                                                                                      │
│   107 │   │   │   return issubclass(origin, self.__origin__)                                                         │
│   108 │   │   return (  # check args first to avoid a recursion error in ABCMeta                                     │
│   109 │   │   │   len(args) == nargs                                                                                 │
│ ❱ 110 │   │   │   and issubclass(origin, self.__origin__)                                                            │
│   111 │   │   │   and all(map(issubclass, args, self.__args__))                                                      │
│   112 │   │   )                                                                                                      │
│   113                                                                                                                │
│                                                                                                                      │
│ ╭─────────────────── locals ────────────────────╮                                                                    │
│ │     args = (<class 'tuple'>, <class 'list'>)  │                                                                    │
│ │    nargs = 2                                  │                                                                    │
│ │   origin = tuple | list                       │                                                                    │
│ │     self = <class 'multimethod.int | float'>  │                                                                    │
│ │ subclass = <class 'multimethod.tuple | list'> │                                                                    │
│ ╰───────────────────────────────────────────────╯                                                                    │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: issubclass() arg 1 must be a class

Worth noting that this works fine:

@multimethod
def f(x: Union[int, float]): ...

@multimethod
def f(x: tuple | list): ...

code:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from multimethod import multimethod

@multimethod
def f(a: dict[int, int]):
    pass

f({1:2})
f({})

run and get output

Traceback (most recent call last):
  File "/mnt/d/Home/Downloads/./main.py", line 11, in <module>
    f({})
  File "/mnt/d/Home/Downloads/multimethod/multimethod/__init__.py", line 188, in __call__
    return self[tuple(map(self.get_type, args))](*args, **kwargs)
  File "/mnt/d/Home/Downloads/multimethod/multimethod/__init__.py", line 184, in __missing__
    raise DispatchError(msg, types, keys)
multimethod.DispatchError: ('f: 0 methods found', (<class 'dict'>,), [])

similarly, I cannot pass empty list [] into list[int]

I have two methods in my Gradient class that I would like to overload as scalar_product:

def scalar_gradient_product(self, gradient: Gradient) -> Density:
    ...

and

def scalar_density_product(self, density: Density) -> ndarray:
    ...

I decorated both with multimethod as follows:

@multimethod
def scalar_product(self, gradient: Gradient) -> Density:
    ...

and

@multimethod
def scalar_product(self, density: Density) -> ndarray:
    ...

This works, i.e. calling scalar_product with Gradient or Density arguments yields the expected respective results. However, I seem to have lost my type-hinting benefits. My editor (VS code) now thinks scalar_product is a property with a return type of multimethod or MethodType.

Am I missing something? Is there some way of gaining the overloading functionality without losing type-hinting?

Thanks in advance 🙂

Currently, the following does not run:

from typing import Callable
from multimethod import multimethod

plusOne : Callable[[int],int] = lambda n : n + 1
addFrog : Callable[[str],str] = lambda s : s + ' and frogs!'

@multimethod
def nameMyFunc(f: Callable[[int], int]):
    print('My argument is a function that takes an int and returns an int')
    return

@nameMyFunc.register
def _(f: Callable[[str], str]):
    print('My argument is a function that takes a str and returns a str')
    return

nameMyFunc(plusOne)
nameMyFunc(addFrog)

The expected output is:

My argument is a function that takes an int and returns an int
My argument is a function that takes a str and returns a str

The actual output is:

Traceback (most recent call last):
  File "/home/lux/repos/cats/test.py", line 17, in <module>
    nameMyFunc(plusOne)
  File "/home/lux/miniconda3/lib/python3.9/site-packages/multimethod/__init__.py", line 298, in __call__
    func = self[tuple(func(arg) for func, arg in zip(self.type_checkers, args))]
  File "/home/lux/miniconda3/lib/python3.9/site-packages/multimethod/__init__.py", line 292, in __missing__
    raise DispatchError(msg, types, keys)
multimethod.DispatchError: ('nameMyFunc: 0 methods found', (<class 'function'>,), [])

Is supporting function arguments with specific argument and return types like this even possible?

If one of the registered functions of a multimethod targets a certain type A and another function targets the Union of several subclasses of A, e.g., B and C, the former will always be preferred (and the latter ignored), even if calling the multimethod with instances of B or C. I would have expected the latter to take priority in this case, since I think of the union function as several functions, each targeting one type in the union, but all with the same code (so I use Union to avoid repeating myself).

Example

from typing import Union

class A: pass
class B(A): pass
class C(A): pass

@multimethod
def f(x: A): print("A")

@f.register
def f(x: Union[B, C]): print("B, C")

Expected

>>> f(A())
A

>>> f(B())
B, C

>>> f(C())
B, C

Actual

>>> f(A())
A

>>> f(B())
A

>>> f(C())
A

Cause

From what I understood from the docs, a Union is checked directly with isinstance, and a tie such as in the last two examples above is resolved using __mro__, but since the __mro__ of the Union will be completely different this won't work as expected.

Workarounds

• Keep the implementation in a function that targets a single type in the union, and make a separate function for each remaining type that explicitly calls the former:

  from typing import Union
  
  class A: pass
  class B(A): pass
  class C(A): pass
  
  @multimethod
  def f(x: A): print("A")
  
  @f.register
  def f(x: B): print("B, C")
  
  @f.register
  def f(x: C): f[B](x)

• Define a common base class for all the types of the union (this requires being able to modify those subclasses):

  from typing import Union
  
  class A: pass
  class Intermediate: pass
  class B(Intermediate): pass
  class C(Intermediate): pass
  
  @multimethod
  def f(x: A): print("A")
  
  @f.register
  def f(x: Intermediate): print("B, C")

You may need to type annotate the package itself. Or to make it work with @overload in some way.

Hi! Please make a new release on PyPI to enable typing backport compatibility. Thanks!

python --version Python 3.8.0
pip freeze multimethod==1.2
Test case:

from collections import namedtuple
from typing import List

from multimethod import multimethod

TestNamedTuple = namedtuple(
    'TestNamedTuple', ['a', 'b', 'c']
)


class TestClass(object):
    @multimethod
    def accept(self, items: List[TestNamedTuple]):
        pass


if __name__ == '__main__':
    a = TestClass()
    a.accept([TestNamedTuple("1", "1", "1"), TestNamedTuple("2", "2", "2")])

This outputs:
RecursionError: maximum recursion depth exceeded while calling a Python object

Using this code

import typing
from multimethod import multidispatch

@multidispatch
def f(s: int): print("int", s)

@multidispatch
def f(s: typing.List): print("list", s)

f(1)
f([1, 2])

the library generates the error

TypeError: issubclass() arg 1 must be a class

The error is in the usage of "issubclass" in

class signature(tuple):
def __le__(self, other):
return len(self) <= len(other) and all(map(**issubclass**, other, self))
...

because other contains the tuple "(typing.List,)".

Can be a good idea to extend the support to all types that it is possible to use with the "type annotation".
For example, if it is used "typing.List[int]" it is possible to check ONLY the FIRST element of the list.
If the list is empty, ANY method that support "typing.List" is good.

Mypy complains about some problems with typing when using this package.

• First, keyword arguments (which are not used in order to select the function to dispatch) cannot be typed in the current version, or the dispatch function hangs.
• Second, Mypy complains when using the register decorator, with "Untyped decorator makes function untyped", because the types for this decorator are not specified.

Using type (or any other subclass of type, e.g. a custom metaclass) as the type hint for one of the parameters of a function of a multimethod makes the dispatch process raise a TypeError when calling type.mro() (which expects the type object as an argument).

Example

@multimethod
def f(x: type):
    print("success")

Expected

>>> f(int)
success

Actual

>>> f(int)
TypeError: unbound method type.mro() needs an argument

Traceback:

Traceback (most recent call last):
  File "C:\Users\Paolo\Code\JB-PycharmProjects\LoveLetter\venv\lib\site-packages\IPython\core\interactiveshell.py", line 3418, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-34-0374cd2152b2>", line 1, in <module>
    f(int)
  File "C:\Users\Paolo\Code\JB-PycharmProjects\LoveLetter\venv\lib\site-packages\multimethod\__init__.py", line 184, in __call__
    return self[tuple(map(self.get_type, args))](*args, **kwargs)
  File "C:\Users\Paolo\Code\JB-PycharmProjects\LoveLetter\venv\lib\site-packages\multimethod\__init__.py", line 184, in __call__
    return self[tuple(map(self.get_type, args))](*args, **kwargs)
  File "C:\Users\Paolo\Code\JB-PycharmProjects\LoveLetter\venv\lib\site-packages\multimethod\__init__.py", line 174, in __missing__
    groups = groupby(signature(types).__sub__, self.parents(types))
  File "C:\Users\Paolo\Code\JB-PycharmProjects\LoveLetter\venv\lib\site-packages\multimethod\__init__.py", line 17, in groupby
    groups[func(value)].append(value)
  File "C:\Users\Paolo\Code\JB-PycharmProjects\LoveLetter\venv\lib\site-packages\multimethod\__init__.py", line 104, in __sub__
    return tuple(mro.index(cls if cls in mro else object) for mro, cls in zip(mros, other))
  File "C:\Users\Paolo\Code\JB-PycharmProjects\LoveLetter\venv\lib\site-packages\multimethod\__init__.py", line 104, in <genexpr>
    return tuple(mro.index(cls if cls in mro else object) for mro, cls in zip(mros, other))
  File "C:\Users\Paolo\Code\JB-PycharmProjects\LoveLetter\venv\lib\site-packages\multimethod\__init__.py", line 103, in <genexpr>
    mros = (subclass.mro() for subclass in self)
TypeError: unbound method type.mro() needs an argument

Cause

Methods of metaclasses like type, e.g. .mro() expect an explicit first argument which is the actual type object being used.

Right now this results in a dispatch error, because dispatch is based only on unnamed arguments:

@multimethod
def f(foo: str):
    return foo

f(foo="bar")
# => DispatchError: ('f: 0 methods found', (), [])

Not a huge deal, but some people are rather particular about using named arguments whenever possible. Would this be possible to support?

This would complicate the dispatch algorithm, admittedly. You'd have to keep track of argument names for each overload and, accordingly, rearrange *args / **kwargs in multimethod.__call__(). This could get a bit gross when accounting for argument names that are repeated or arranged in different orders for different overloads ...

Would it be possible to implement multimethods dispatching on typing.Literal?

This may be working as intended, but I was a bit surprised that multidispatch.signature was always set to the last registered function (rather than the original function defining the "spec"). This works ok for methods with consistent parameters, but breaks in weird (but understandable) ways with different parameters:

from typing import Any

from multimethod import multidispatch


@multidispatch
def add(a: Any, b: Any) -> Any:
    ...


print(add.signature)  # (a: Any, b: Any) -> Any


@add.register
def add_str(a: str, b: str) -> str:
    return a + b


print(add.signature)  # (a: str, b: str) -> str
print(add("a", "b"))  # ab
print(add(a="a", b="b"))  # ab


@add.register
def add_extra(a: str, b: str, c: str) -> str:
    return a + b + c


print(add.signature)  # (a: str, b: str, c: str) -> str
print(add("a", "b"))  # ab
try:
    # This should route to add_str
    print(add(a="a", b="b"))
except TypeError as e:
    print(e)  # missing a required argument: 'c'
print(add(a="a", b="b", c="c"))  # abc


@add.register
def add_str(a: str, b: str) -> str:
    return a + b


print(add.signature)  # (a: str, b: str) -> str
print(add("a", "b"))  # ab
print(add(a="a", b="b"))  # ab
try:
    # This should route to add_extra
    print(add(a="a", b="b", c="c"))
except TypeError as e:
    print(e)  # got an unexpected keyword argument 'c'

This is caused by the signature being set in multidispatch.__init__, but overload.register (via multimethod.register) calling __init__ for each registered item. Support for dispatching on dynamic kwargs would be nice, but short of that, maybe just a note in the documentation?

I actually encountered this when trying to add a few extra checks in multidispatch.register in a subclass, but noticed that the .signature (and other .__init__ properties I added) changed every time.

First, let me thank you for this small but incredibly useful package.

I have a problem when the register method tries to infer the parameters from the typing annotations. If I register a function as

@inner_product.register
def inner_product_fdatabasis(arg1: Union[FDataBasis, Basis],
                             arg2: Union[FDataBasis, Basis]):

then, upon calling the multimethod it will fail with

  ...
  File "E:\Programas\Utilidades\Lenguajes\Miniconda\envs\fda\lib\site-packages\multimethod.py", line 171, in __call__
    return self[tuple(map(self.get_type, args))](*args, **kwargs)
  File "E:\Programas\Utilidades\Lenguajes\Miniconda\envs\fda\lib\site-packages\multimethod.py", line 209, in get_type
    return subtype(type(arg), *map(get_type, itertools.islice(arg, 1)))
  File "E:\Programas\Utilidades\Lenguajes\Miniconda\envs\fda\lib\site-packages\multimethod.py", line 171, in __call__
    return self[tuple(map(self.get_type, args))](*args, **kwargs)
  File "E:\Programas\Utilidades\Lenguajes\Miniconda\envs\fda\lib\site-packages\multimethod.py", line 209, in get_type
    return subtype(type(arg), *map(get_type, itertools.islice(arg, 1)))
  File "C:\Users\Carlos\git\scikit-fda\skfda\representation\_functional_data.py", line 661, in __iter__
    yield self[i]
  File "C:\Users\Carlos\git\scikit-fda\skfda\representation\basis\_fdatabasis.py", line 756, in __getitem__
    return self.copy(coefficients=self.coefficients[key:key + 1])
  File "C:\Users\Carlos\git\scikit-fda\skfda\representation\basis\_fdatabasis.py", line 518, in copy
    basis = copy.deepcopy(self.basis)
  File "E:\Programas\Utilidades\Lenguajes\Miniconda\envs\fda\lib\copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "E:\Programas\Utilidades\Lenguajes\Miniconda\envs\fda\lib\copy.py", line 275, in _reconstruct
    y = func(*args)
  File "E:\Programas\Utilidades\Lenguajes\Miniconda\envs\fda\lib\copy.py", line 274, in <genexpr>
    args = (deepcopy(arg, memo) for arg in args)
  File "E:\Programas\Utilidades\Lenguajes\Miniconda\envs\fda\lib\copy.py", line 157, in deepcopy
    y = _deepcopy_atomic(x, memo)
  File "E:\Programas\Utilidades\Lenguajes\Miniconda\envs\fda\lib\copy.py", line 190, in _deepcopy_atomic
    def _deepcopy_atomic(x, memo):
RecursionError: maximum recursion depth exceeded while calling a Python object

I see from the call stack that __iter__ is being called in my objects without reason, so maybe the problem is related with iterable objects.

Registering the types manually, however, works well:

@inner_product.register(FDataBasis, FDataBasis)
@inner_product.register(FDataBasis, Basis)
@inner_product.register(Basis, FDataBasis)
@inner_product.register(Basis, Basis)
def inner_product_fdatabasis(arg1: Union[FDataBasis, Basis],
                             arg2: Union[FDataBasis, Basis]):

Thanks in advance.

Consider the following script.

from typing import Sequence, Union

from multimethod import multimethod


@multimethod
def func(x: Union[int, Sequence[int]]) -> Sequence[int]:
    raise NotImplementedError


@func.register
def _from_int(x: int) -> Sequence[int]:
    return [x, x]


@func.register
def _from_sequence(x: Sequence[int]) -> Sequence[int]:
    return x


func(1)
func([1, 2])

When executing python script.py I noticed weird behavior.

• func(1) call works fine.
• func([1, 2]) either executes fine, or fails with the following error

Traceback (most recent call last):
  File "script.py", line 22, in <module>
    func([1, 2])
  File "C:\venv\lib\site-packages\multimethod\__init__.py",
line 301, in __call__
    func = self[tuple(func(arg) for func, arg in zip(self.type_checkers, args))]

  File "C:\venv\lib\site-packages\multimethod\__init__.py",
line 295, in __missing__
    raise DispatchError(msg, types, keys)
multimethod.DispatchError: ('func: 0 methods found', (<class 'list'>,), [])

I would like to emphasize that this behavior is flaky. Sometimes this error is raised, but sometimes the code executes just fine.

$ pip list
Package     Version
----------- -------
multimethod 1.6
pip         19.2.3
setuptools  41.2.0

$ python --version
Python 3.8.3

In the previous version of multimethod there was no such issue.

Hello. I am trying to use Callable in my code. But I get the error multimethod.DispatchError: ('from_producer: 0 methods found', (<class 'function'>,), []) .

What am I doing wrong?

from typing import Callable

from multimethod import multimethod


class AAA():
    @staticmethod
    @multimethod
    def from_producer(producer: Callable[..., int]) -> None:
        q = 1

    @staticmethod
    @multimethod
    def from_producer(producer: int) -> None:
        q = 1


def qwerty() -> int:
    return 1


def test_1():
    AAA.from_producer(qwerty)

Hi!

I am occasionally having a race condition while using multimethod-decorated functions in different threads:

  File "/path/to/env/multimethod/__init__.py", line 313, in __call__
    func = self[tuple(func(arg) for func, arg in zip(self.type_checkers, args))]
  File "/path/to/env/multimethod/__init__.py", line 299, in __missing__
    for key in self.parents(types):
  File "/path/to/env/multimethod/__init__.py", line 248, in parents
    parents = {key for key in self if isinstance(key, signature) and key < types}
  File "/path/to/env/multimethod/__init__.py", line 248, in <setcomp>
    parents = {key for key in self if isinstance(key, signature) and key < types}
RuntimeError: dictionary changed size during iteration

I didn't manage to figure out the conditions in which it occurs, but in my code it happens in around 0.5% of runs.
Maybe you could protect the types -> function mapping with a RLock?

from multimethod import multimethod
class A:
    @multimethod
    def p(self, v: int):
        print("i")
class B(A):
    @multimethod
    def p(self, v: float):
        print("f")
B().p(233)

Currently @multimethod in derived class cannot access function defined in base class

A easy thought is directly add a copy in derived class like this:

from multimethod import multimethod
class A:
    @multimethod
    def p(self, v: int):
        print("i")
class B(A):
    p = A.p
    @multimethod
    def p(self, v: float):
        print("f")
B().p(233)

now the code work, but it also change A.p's behavior since I can now call A().p(2.333)

so is there any easy copy function for multimethod.multimethod?

Hello and thank you for this nice package.
As I was hit by the threading problem that was fixed in commit be8564a, could you please make a new release with this fix?

Consider this example:

from typing import TypeVar, Generic
from multimethod import multimethod

T = TypeVar('T')

class A(Generic[T]):
    pass

class B:
    pass

@multimethod
def func(a: A[B]):
    pass

func(A[B]())

It crashes with multimethod.DispatchError: ('func: 0 methods found', (<class '__main__.A'>,), []).

Whereas without the annotation everything works fine.

With a test.py containing:

from multimethod import multimethod


class Test:

    @multimethod
    def __init__(self) -> None:
        pass

    @__init__.register
    def _(self, a: int) -> None:
        pass

> mypy test.py
test.py:6: error: Unsupported decorated constructor type
Found 1 error in 1 file (checked 1 source file)

Can I multimethod an __init__ (and pass mypy)? Sorry if I'm missing anything obvious, I'm just getting started with multiple dispatch.

Postponed evaluation will be added into python3.10, see this, and we can already use it since python 3.7

But multimethod does not support it very well. For example, I have three file in same directory:

a.py:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import annotations
from multimethod import multimethod
import b

class A:
    @multimethod
    def ping(self, v: b.B):
        print(self, "ping", v)

b.py:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import annotations
from multimethod import multimethod
import a

class B:
    @multimethod
    def ping(self, v: a.A):
        print(self, "ping", v)

and main.py:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import annotations

import a
import b

a.A().ping(b.B())

run python main.py and the program crash

the output is

Traceback (most recent call last):
  File "/mnt/d/Home/Downloads/main.py", line 6, in <module>
    import a
  File "/mnt/d/Home/Downloads/a.py", line 6, in <module>
    import b
  File "/mnt/d/Home/Downloads/b.py", line 8, in <module>
    class B:
  File "/mnt/d/Home/Downloads/b.py", line 10, in B
    def ping(self, v: a.A):
  File "/usr/lib/python3.9/site-packages/multimethod/__init__.py", line 121, in __init__
    self[get_types(func)] = func
  File "/usr/lib/python3.9/site-packages/multimethod/__init__.py", line 25, in get_types
    annotations = dict(typing.get_type_hints(func))
  File "/usr/lib/python3.9/typing.py", line 1386, in get_type_hints
    value = _eval_type(value, globalns, localns)
  File "/usr/lib/python3.9/typing.py", line 254, in _eval_type
    return t._evaluate(globalns, localns, recursive_guard)
  File "/usr/lib/python3.9/typing.py", line 493, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
AttributeError: partially initialized module 'a' has no attribute 'A' (most likely due to a circular import)

Would it be in scope of multimethod to fallback to one of the methods (possibly explicitly marked) in case of a dispatch error?

It would help to maintain backward compatibility for our codebase for people specifying positional arguments with keywords.

Python 3.8 saw the introduction of functools.singledispatchmethod. I would love to see support for the same interface in a multipledispatchmethod.

If one tries to specialize on an argument with a Literal[_] type, they get TypeError: typing.Literal cannot be used with issubclass(). It'd be nice to have support for this; I suspect patching issubclass to add the following rules would work:

• Literal[x] <: Literal[x]
• Literal[x] <: type(x)