jspahrsummers / adt Goto Github PK

The README says:

It's conventional to declare your fields with ALL_UPPERCASE names, but the only true restriction is that they cannot be lowercase.

If I understand this correctly, it means that the only kind of names that are invalid are names that are fully in lowercase. That is, FOO_BAR, FooBar, and foo_Bar are valid names, but foobar and foo_bar aren't.

However, match method does not seem to work nicely with names that are not fully in uppercase. The following code is supposed to work:

from adt import adt, Case

@adt
class Foo:
    Bar: Case
    Baz: Case

value = Foo.Bar()
print(value.match(
    bar=lambda: 'bar',
    baz=lambda: 'baz',
))

But it fails with the error: ValueError: Unrecognized case BAR in pattern match against <<class '__main__.Foo'>.Bar: None> (expected one of dict_keys(['Bar', 'Baz'])).

The code of match seems to convert the names of variants to upper case, which may not be correct when original names contain lowercase letters (provided that such names are allowed).

I see two possible solutions here.

1. Forbidding variant names not in the format ALL_UPPERCASE, stating this in the README and perhaps enforcing this rule in the decorator in order to prevent accidental mistakes. This may be backwards-incompatible, but is probably the easiest solution. As a compromise, it is possible to deprecate names containing lowercase characters and possibly to warn about them in the decorator.
2. Modifying the code to accept non-fully uppercase names. This is tricky because we either have to continue accepting case-insensitive variant names in match and somehow convert them to their original case, or accept only names in their original case (that is, one would only be able to write something like value.match(Bar=..., Baz=...), not value.match(bar=..., baz=...)). The former may be tricky to implement, and the latter would probably totally break backward compatibility.

Out of these two, I personally prefer the first solution, but it's just my opinion on this problem.

P.S. I used the term variant as a synonym of case here to avoid confusion between lower/upper case and ADT cases.

First, about #13,
FYI, Pampy is not a real pattern matching(hereafter PM) library. It is atmost an implememtation of an interpreter that supports very limited and inextensible PM. Further, there has already been thousands of "plausive patterm matching" implemented for Python, and IMHO MacroPy could be a more correct one. Pampy made use of the information asymmetry to be attractive(even the 10-year-ago similar libraries didn't boast that way), and finally successfully become a cheater.

Your notations is a bit similar to Pampy, and a.match(case=action_under_case) seems to be so much evil, unmaintainable.

More,
I suggest you to focus on ADTs instead of getting burdened from the mess of "pattern matching for PY". We can use method dispatch to achieve a part of PM's functionality, as you know we have typing.overload which works well.

Besides, there could be some feasible ways to achieve the true PM for Python without requiring any new syntax constructs:

Core idea: convert code objects(sorry you cannot get ASTs in runtime, except you choose the evil and unreliable inspect.getsource), to change semantics for specific structures; simultaneously add type checking support. E.g., you can use the notation like

with val_to_match:
    if Case(p1, p2, ...):
        ...
    if ....

to denote

case val_to_match of
    Case(p1, p2, ...) -> ...
     ...

HOW TO

• use the library uncompyler to convert code objects into ASTs/analyze code objects to recognize the semantics.
• transform ASTs/your own semantics representations, and rewrite specific structures to have PM semantics.
• compile, get your new code objects and replace your original functions with them.

You might also google CASE TREE or TREE PATTERN MATCHING for the canonical algorithms of PM.

There is some discussion on typing-sig about sealed classes here:
https://mail.python.org/archives/list/[email protected]/thread/QSCT2N4RFPRQN2U7NIX6VCNVNDHGO22U/

Perhaps you can chime in with your thoughts as the author of this library.

@sealed
class Tree:
    EMPTY: None
    LEAF: Leaf
    NODE: Node
  
@dataclass  
class Leaf:
   data: int
   
@dataclass
class Node:
   left: Tree
   right: Tree  

would be nice. Other python libraries with similar functionality linked from here:

https://www.reddit.com/r/Python/comments/ipqqca/sum_types_discriminated_unions_for_python/

is pretty nice syntactically.

Looking at the published files in pypi we can see that there is no source file published. Compare that with, for example the attr package on pypi.

Having the ability to download the source from pypi is useful for down-stream systems that consume dependencies from pypi but want to build packages themselves.

https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies

This would also allow us to put version constraints on it, for dependent packages.

Running the plugin with mypy==0.711 on the following code leads to an error:

from adt import adt, Case

@adt
class Expression:
    LITERAL: Case[float]

result: None = Expression.LITERAL(0.1).match(literal=lambda n: None)

error: Cannot infer type argument 1 of "match" of "Expression"

This does not happen when using something other than None as return values:

result: int = Expression.LITERAL(0.1).match(literal=lambda n: 1)

My Code:

@adt
class Result(Generic[S, E]):
    Success: Case[S]
    Error: Case[E]

    async def map(self, func: Callable[[S], Awaitable[Union[S, E]]]) -> "Result[S, E]":
        return self.match(success=lambda s: self.Success(func(s)), error=lambda e: e)

    async def flat_map(self, func: Callable[[S], Awaitable["Result[S, E]"]]) -> "Result[S, E]":
        return self.match(success=lambda s: (await func(s)), error=lambda e: e)

Pycharm said "Case object is not callable" for this line.

self.Success(func(s))

also intellisense doesn't work for completing self.match

I understand that this is architectural issue. to make pycharm intellisense work, I think we have to use base class, not use decorator.

this is just a suggestion. your lib works perfectly on runtime :)

#10 demonstrates that it is not currently. Should it be?

One issue I have with the match function is that it relies on either using lambdas (which can only contain one line) or if you need multiple lines, first defining the handler function and then passing it to the match function.

An option that I think is worth considering is using context managers for pattern matching. The main drawback is that it would not allow returning a value like with the match function, but the syntax is nicer when the case handlers contain more than one statement. As a proof of concept I have implemented this new interface while keeping the library backwards compatible.

Here is some example code of the new proposed syntax.

@adt
class ContextMatching:
    EMPTY: Case
    INTEGER: Case[int]
    STRINGS: Case[str, str]

foo = ContextMatching.INTEGER(1)

with foo.empty:
    print("Is empty")

with foo.integer as value:
    print("Is integer:", value)

with foo.strings as (string_1, string_2):
    print("Is strings:", string_1, string_2)

This example will end up printing out Is integer: 1

This opens up the possibility for matching values as well. Although not implemented yet I believe code such as this is possible to implement while keeping the current API intact.

@adt
class ContextMatching:
    NONE: Case
    OK: Case[int, float]

with foo:
    with foo.ok[:4, :] as (val_1, val_2):
        print("val_1 less than 4 and val_2 anything")
    
    with foo.ok[4:, 1.3:9.9] as (val_1, val_2):
        print("val_1 4 or higher and val_2 between 1.3 and 9.9")
    
    with foo.ok as (val_1, val_2):
        print("Unhandled ok cases")

(From the README)

Lets say I have an ADT defined as:

@adt
class Foo:
        BAR = Case[str]
        BAZ = Case[str]

CAsting a list of adts as a set:

set([BAR("1"), BAR("1"), BAZ("1")])

will return:

set(BAR("1"), BAR("1"), BAZ("1"))

instead of:

set([BAR("1"), BAZ("1")])

It used to work with mypy==0.761 but broke when when upgraded to 0.812

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "mypy/semanal.py", line 4835, in accept
  File "mypy/nodes.py", line 950, in accept
  File "mypy/semanal.py", line 1048, in visit_class_def
  File "mypy/semanal.py", line 1125, in analyze_class
  File "mypy/semanal.py", line 1134, in analyze_class_body_common
  File "mypy/semanal.py", line 1180, in apply_class_plugin_hooks
  File "/usr/local/lib/python3.8/site-packages/adt/mypy_plugin.py", line 187, in _transform_class
    _add_accessor_for_case(context, case)
  File "/usr/local/lib/python3.8/site-packages/adt/mypy_plugin.py", line 269, in _add_accessor_for_case
    return_type=case.accessor_return())
  File "/usr/local/lib/python3.8/site-packages/adt/mypy_plugin.py", line 145, in accessor_return
    return mypy.types.TupleType(
  File "mypy/types.py", line 1373, in __init__
TypeError: list object expected; got tuple[mypy.types.Instance, mypy.types.Instance]
/builds/ad53de79/0/path/to/my/file.py:30: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.rtfd.io/en/latest/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.812

Using python 3.8.5

I couldn't get a _MatchResult type variable to work, but maybe each callable could return its own TypeVar which are then all Unioned together to obtain the return type of match().

ListADT.CONS(("a", ListADT.CONS(("b", ListADT.NIL)))) is ugly as hell (with the nested parens)

Actions are a bit nicer (and certainly better-integrated), so we can dump CircleCI.

Can we, e.g., derive useful comparison operators? NamedTuple does this.

Hi! Nice work, thanks for publishing, absolutely appreciated.

Let's take a tiny example:

from adt import adt, Case

@adt
class Cmd:
    READKEY:   Case[str, str]
    WRITEKEY:  Case[str, str, str]
    DELETEKEY: Case[str, str]

This works just fine on the REPL, but the mypy plugin thing isn't giving it to me:

command.py:6: error: "Case" expects no type arguments, but 2 given
command.py:7: error: "Case" expects no type arguments, but 3 given
command.py:8: error: "Case" expects no type arguments, but 2 given
Found 3 errors in 1 file (checked 1 source file)

Of course I followed the README, and had put the lines in the project's setup.cfg:

[mypy]
plugins = adt.mypy_plugin

I'm abolutely sure that the plugin gets loaded: mypy spews errors as expected whenever I garble the config. But the complaints are still there. Python 3.6; mypy 0.730 — the freshest at pypi right now.

Do you see anything I'm missing? Or if you could perhaps retest with recent mypy, perhaps they broke the plugin...

Found out about pampy after starting this project. It seems like it will provide a lot of similar utility vis-a-vis pattern matching, though it doesn't particularly make the construction/definition of algebraic data types easier.

My instinct is that we should integrate with it really nicely (which might also clean up our own pattern matching abstraction), but otherwise this library still offers additional value—but open to other feedback here.

Putting aside arguments about failing fast vs. failing safe, the ListADT example is a bit weird with the current behavior of returning None.

The following adt with an extra method bind is failing to typecheck. Mypy is returning

main.py:15: error: Argument "ok" to "match" of "Result" has incompatible type "Callable[[A], Result[C, B]]"; expected "Callable[[Result[C, B]], Result[C, B]]"

from adt import adt, Case
from typing import Generic, TypeVar, Callable

A = TypeVar("A")
B = TypeVar("B")
C = TypeVar("C")

@adt
class Result(Generic[A, B]):
    OK:  Case[A]
    ERR: Case[B]

    # bind :: Result a b -> (a -> Result c b) -> Result c b
    def bind(self, fun : Callable[[A], Result[C, B]]) -> Result[C, B]:
        return self.match(ok=fun, err=Result.ERR)

If I reveal the type of the match function, mypy says:

main.py:15: note: Revealed type is 'def [_MatchResult] (*, ok: def [_MatchResult] (A`1) -> _MatchResult`1, err: def [_MatchResult] (B`2) -> _MatchResult`1) -> _MatchResult`1'

and revealing the ok value:

main.py:15: note: Revealed type is 'def (A`1) -> main.Result[C`-1, B`2]'

So, the revealed types seems to be correct, but mypy still reports an error.

(where we want to place a constructor)

First, let just thank you so much for your work on this. Python is inescapable for many people working in data/machine learning, but any modern language should have sum types and pattern matching! Somehow Javascript sorted it out with Typescript. This really should be a standard feature with support for standard | sum type syntax 😖

Ok. Rant complete.

Are you considering adding a default option for pattern matching? I know that there are arguments against it, but otherwise you end up writing things like:

e.match(
  VAR1=lambda x: f(x)
  VAR2=lambda _: g()
  VAR3=lambda _: g()
  VAR4=lambda _: g()
  # you get the idea...
)

According to the Python documentation:

The only required property is that objects which compare equal have the same hash value

At the moment, this isn't the case:

from adt import adt, Case


a1 = "abc"

a2 = "ab"
a2 += "c"

assert a1 == a2
assert hash(a1) == hash(a2)

@adt
class OptionStr:
    SOME: Case[str]
    NONE: Case

b1 = OptionStr.SOME(a1)
b2 = OptionStr.SOME(a2)

assert b1 == b2
assert hash(b1) == hash(b2)  # Fails

It looks like the fix is to add an additional function in adt/decorator.py – are there any subtleties that complicate that?

@adt
class ExampleADT:
    EMPTY: Case
    INTEGER: Case[int]
    STRING_PAIR: Case[str, str]

First, I love what you are doing with this library. There is just one thing holding me back from using it. I would like to name the fields, rather than access them with positional indexing. Something like what I am showing below.

@adt
class ExampleADT:
    EMPTY: Case
    INTEGER: Case[int]
    STRING_PAIR: Case[foo: str, bar: str]

The accessor method could return a NamedTuple instead of just a Tuple.

def string_pair(self) -> Tuple[str, str]:
        # unpacks strings and returns them in a tuple

When running mypy==0.711 against the following code taken from the README, mypy throws an error:

@adt
class ExampleADT:
    EMPTY: Case
    INTEGER: Case[int]
    STRING_PAIR: Case[str, str]

    @property
    def safe_integer(self) -> Optional[int]:
        return self.match(empty=lambda: None,
                          integer=lambda n: n,
                          string_pair=lambda a, b: None)

error: Incompatible return value type (got "None", expected "int")
error: Incompatible return value type (got "None", expected "int")

Hi, Justin.

After several months, my project providing the very high performance pattern matching finally got released.

I wonder if we could mutually refer the projects in each README, then people can have a good instruction about how to work with both pattern matching and adt in Python.

Besides, my project does not work with static checker like Mypy due to my limited knowledge about writing mypy plugins. I'd ask if you could help me out of this, any stuff would be appreciated.