Giter VIP home page Giter VIP logo

python-lenses's Introduction

Lenses

Lenses is a python library that helps you to manipulate large data-structures without mutating them. It is inspired by the lenses in Haskell, although it's much less principled and the api is more suitable for python.

Installation

You can install the latest version from pypi using pip like so:

pip install lenses

You can uninstall similarly:

pip uninstall lenses

Documentation

The lenses library makes liberal use of docstrings, which you can access as normal with the pydoc shell command, the help function in the repl, or by reading the source yourself.

Most users will only need the docs from lenses.UnboundLens. If you want to add hooks to allow parts of the library to work with custom objects then you should check out the lenses.hooks module. Most of the fancy lens code is in the lenses.optics module for those who are curious how everything works.

Some examples are given in the examples folder and the documentation is available on ReadTheDocs.

Example

>>> from pprint import pprint
>>> from lenses import lens
>>>
>>> data = [{'name': 'Jane', 'scores': ['a', 'a', 'b', 'a']},
...         {'name': 'Richard', 'scores': ['c', None, 'd', 'c']},
...         {'name': 'Zoe', 'scores': ['f', 'f', None, 'f']}]
...
>>> format_scores = lens.Each()['scores'].Each().Instance(str).call_upper()
>>> cheat = lens[2]['scores'].Each().set('a')
>>>
>>> corrected = format_scores(data)
>>> pprint(corrected)
[{'name': 'Jane', 'scores': ['A', 'A', 'B', 'A']},
 {'name': 'Richard', 'scores': ['C', None, 'D', 'C']},
 {'name': 'Zoe', 'scores': ['F', 'F', None, 'F']}]
>>>
>>> cheated = format_scores(cheat(data))
>>> pprint(cheated)
[{'name': 'Jane', 'scores': ['A', 'A', 'B', 'A']},
 {'name': 'Richard', 'scores': ['C', None, 'D', 'C']},
 {'name': 'Zoe', 'scores': ['A', 'A', 'A', 'A']}]

The definition of format_scores means "for each item in the data take the value with the key of 'scores' and then for each item in that list that is an instance of str, call its upper method on it". That one line is the equivalent of this code:

def format_scores(data):
    results = []
    for entry in data:
        result = {}
        for key, value in entry.items():
            if key == 'scores':
                new_value = []
                for letter in value:
                    if isinstance(letter, str):
                        new_value.append(letter.upper())
                    else:
                        new_value.append(letter)
                result[key] = new_value
            else:
                result[key] = value
        results.append(result)
    return results

Now, this code can be simplified using comprehensions. But comprehensions only work with lists, dictionaries, and sets, whereas the lenses library can work with arbitrary python objects.

Here's an example that shows off the full power of this library:

>>> from lenses import lens
>>> state = (("foo", "bar"), "!", 2, ())
>>> lens.Recur(str).Each().Filter(lambda c: c <= 'm').Parts().call_mut_reverse()(state)
(('!oo', 'abr'), 'f', 2, ())

This is an example from the Putting Lenses to Work talk about the haskell lenses library by John Wiegley. We extract all the strings inside of state, extract the characters, filter out any characters that come after 'm' in the alphabet, treat these characters as if they were a list, reverse that list, before finally placing these characters back into the state in their new positions.

This example is obviously very contrived, but I can't even begin to imagine how you would do this in python code without lenses.

License

python-lenses is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.

python-lenses's People

Contributors

cage433 avatar cebamps avatar daenyth avatar gdmcbain avatar gurkenglas avatar ingolemo avatar snoopj avatar sobolevn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

python-lenses's Issues

Is it time to release a new version?

The recent changes dropping old python support (and with them dependency on typing) is a welcomed change, and we would love to pull these from pypi. How about a 1.1.0 ?

nested _each does not work as expected

The following example does not appear to work as expected:

In [17]: lens([[1,3], 4]).tuple_(lens()[0].each_(), lens()[1]).each_().get_all()
Out[17]: [4, 4]

Perhaps I'm misunderstanding, but shouldn't the result be [1,3,4] or something?
It looks like it's implicitly using the + monoid

Make a release?

The documentation on here doesn't match the latest version any more, and it seems like the api has changed quite a bit (for the better). A new release would be nice

[Help Wanted] - Memoizing when modifying similar subparts.

Hi,
The lenses library was invaluable to helping write the initial prototype of my sequential circuit library
pyAiger. I mainly used the lens api to perform very recursive updates, for example, to turn latches into inputs:

circ = bind(self).Recur(LatchIn) \
                 .Filter(lambda x: x.name in latches) \
                 .modify(lambda x: Input(name))

Unfortunately, due to scaling issues I've had to rewrite a number of methods. The key problem is that the same sub-circuit appears multiple times in the data structure. Thus, the Recur lens goes down the same path multiple times. My new hard-coded traversals are similarly recursive, but memoize to avoid going down the same path. How could I tell python-lenses to do this? For reference, this datastructure is essentially a forest with many of the trees overlapping. Is the best way to do this to memoize the setattr hook?

Fails on import with python3.5

Python 3.5.2, ubuntu 16.04

I get what looks like a namespace collision when I try to import lenses

$ python -c "import lenses"

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/user/.virtualenvs/abc/lib/python3.5/site-packages/lenses/__init__.py", line 23, in <module>
    from . import optics
  File "/home/user/.virtualenvs/abc/lib/python3.5/site-packages/lenses/optics/__init__.py", line 1, in <module>
    from .base import *
  File "/home/user/.virtualenvs/abc/lib/python3.5/site-packages/lenses/optics/base.py", line 7, in <module>
    from ..maybe import Just, Nothing
  File "/home/user/.virtualenvs/abc/lib/python3.5/site-packages/lenses/maybe.py", line 71, in <module>
    class Nothing(Just[A]):
  File "/usr/lib/python3.5/typing.py", line 1033, in __getitem__
    extra=self.__extra__)
  File "/usr/lib/python3.5/typing.py", line 909, in __new__
    self = super().__new__(cls, name, bases, namespace, _root=True)
  File "/usr/lib/python3.5/typing.py", line 107, in __new__
    return super().__new__(cls, name, bases, namespace)
  File "/home/user/.virtualenvs/abc/lib/python3.5/abc.py", line 133, in __new__
    cls = super().__new__(mcls, name, bases, namespace)
ValueError: 'item' in __slots__ conflicts with class variable

Steps to reproduce

Blank python3.5 virtualenv. pip install lenses==0.4.0

0.3.0 works fine though.

Compose producing unexpected values

The following two snippets produce different results:

The following appears to me to be correct:

lens.Each().Parts().F(sum).get()([1,2,3]) # returns 6

Whereas I think this is a bug:

(lens.Each() & lens.Parts().F(sum)).get()([1,2,3]) # returns 1

I'm using python-lenses 1.1.0, running in python 3.9.13.

Suggestion for notification when data backing a lens changes

I think it would be nice to get notified when the data backing a lens changes.
I suppose the simplest method is to be able to attach a callback function to the lens.
For example clojure atoms have the methods
(add-watch atom key fn)
(remove-watch atom key)
where the callback function is
fn [key atom old-value new-value]

Ultimately, I think anytime you are dealing with notifications things can get complicated.
It might be best to have a means of delegating the notification task.
There are considerations of pushing vs polling.
There are concepts of having formulas based upon reactive inputs and then the formulas need to be reactive also.
Then you get problems like batching changes and notifying on an animation frame.
There can be consistency issues with the data.

The following is a story for the motivation of what I want.
I have a GUI built with Python/Qt. I'm trying to figure out a better design for it. I have spent some time looking at React.js, Redux.js, Knockout.js, Reagent/Reframe.cljs, Trellis(py), Elm. I desire to borrow ideas from these tools, but it is taking me too long to figure out these tools, especially how they are implemented.

I like the idea of storing all the application state in a global atom.
I like the broad idea of thinking about changes to this global atom as a fold() function.
fold(reducer, app-state, events)
I like the idea of the app-state data structure being a persistent data structure and there
being a single mutable atom holding the app-state.
I like the idea of one way data flow, where the GUI sends messages to the data store, and it is these messages that are queued up for the fold operation above.

I want GUI elements that can be composed into hierarchies and moved around. By GUI element, I mean a collection of widgets with some functionality.

I want the GUI to automatically and efficiently re-render when state that the GUI depends upon changes.

The thing I don't like about Reagent/Knockout/Trellis is that in order to get change notification I need to wrap every single piece of data into some kind of object. A ratom/observable/cell.
I don't want my app-state composed of tons of little mutable objects so that I can get notified when their data changes. I want a single persistent app-state.

I want my GUI elements to depend on lenses/paths into the app-state.
I want a GUI element to re-render when that data behind the lens/path changes.

I'm not certain, but with Elm, I think one needs to pass pieces of the global app-state down the tree hierarchy of GUI elements. That is app-state is refined further and further as it is passed down the tree.
Leaf nodes see just what they need, but parent nodes have extra state.

I think I like the idea of each GUI element having read-only access to the entire global app-state through a set of paths or a lens.

I think I can get everything else working with some sort of lens changed notification.

& vs. StateFunction

state_dict &= (lens.Keys()&regex("^g.model")).set("0") works, state_dict &= lens.Keys()&regex("^g.model").set("0") doesn't. & should lift StateFunctions.

Using setters with frozen dataclasses

One of the projects I work on makes extensive use of frozen dataclasses, and generally uses the dataclasses.replace function to construct new instances of dataclasses with specific fields updated. This seems like it would be a natural fit for the setters available in lenses, or an extension thereof. Is this something that has been done before, or do you have any advice about how to accomplish this?

Add type annotations

Having the library fully typed and verified by mypy would make it a lot more usable, both from a documentation point of view as well as letting people use the library more easily in their own typed code

Using lens to set an iterative value

Hi, ingolemo. Thank you very much for this awesome library.

I wonder if lens could be used to set a “yielding value”. Let me give you an example:

data = {'utt_1': {'words': [], 'index': None},
'utt_2': {'words': [], 'index': None},
'utt_3': {'words': [], 'index': None},
'utt_4': {'words': [], 'index': None},
'utt_5': {'words': [], 'index': None},
'utt_6': {'words': [], 'index': None},
'utt_7': {'words': [], 'index': None},
'utt_8': {'words': [], 'index': None}}

index_generator = (i for i in range(1, 8))

format = lens.Each()[1]['index'].set(next(index_generator))

format(data)

which returns:

{'utt_1': {'words': [], 'index': 1}, 'utt_2': {'words': [], 'index': 1}, 'utt_3': {'words': [], 'index': 1}, 'utt_4': {'words': [], 'index': 1}, 'utt_5': {'words': [], 'index': 1}, 'utt_6': {'words': [], 'index': 1}, 'utt_7': {'words': [], 'index': 1}, 'utt_8': {'words': [], 'index': 1}}

instead of expected:

{'utt_1': {'words': [], 'index': 1}, 'utt_2': {'words': [], 'index': 2}, 'utt_3': {'words': [], 'index': 3}, 'utt_4': {'words': [], 'index': 4}, 'utt_5': {'words': [], 'index': 5}, 'utt_6': {'words': [], 'index': 6}, 'utt_7': {'words': [], 'index': 7}, 'utt_8': {'words': [], 'index': 8}}

Would there be any way to achieve the expected behavior?

Debugging code involving lenses is hard

The call stacks end up super deep and the order of operations strange. Have you considered a refactor where lens.....get() is compiled into a function block where .F(getter) corresponds to string2 = getter(int5), .Recur(Foo) corresponds to for foo in recur(Foo, bar):, etc.? If the only reason against is months of tedious refactoring, say so - they might be the kind that AI tools these days solve.

Suggestion: Lens for a union of two properties?

I have a usecase where I'm trying to rewrite a particular kind of object in a deeply nested structure - docker image refs in Kubernetes specifications.

A simplified version is: given a dict like

containers:
    - image: image/foo
      serviceAccountName: sa
      args: ["bar", "image/foo", "baz"]
    - image: image/bar
      serviceAccountName: sa
      args: ["bar", "image/bar", "baz"]

I'd like to rewrite both occurrences of the "image/foo" and "image/bar" string alone.The same function can rewrite both, so have something like this in my Python code, to pipe the results of one modify through another:

containers_lens = lens['containers']
image_lens = lens['image']
args_lens = lens['args'].Each()
all_containers_lens = containers_lens.Each()

args_done = (all_containers_lens & args_lens).modify(rewrite_image)(doc)
return (all_containers_lens & image_lens).modify(rewrite_image)(args_done)

That works a treat, but I could avoid that second line if I could say something like (all_containers_lens & (args_lens | image_lens)) to focus on both the image key and the elements of the args key at the same time.

That's assuming I'm not missing a more elegant way to do it that's already there!

Thanks, love the library!

Help request: Turn Each() back into a whole-sequence lens?

Hi

I'm trying to get my head around how to compose lenses / optics. Here's what I'm stuck on:


from lenses import lens

class Parcel(NamedTuple):
    sku: str
    size: int

class Line:
    def __init__(self, parcels):
        self.parcels = parcels

    skus = lens.parcels.Each().sku

    def print_first_sku(self):
        # This prints what I want ("abc")
        print(self.skus.collect()[0])
        # I would like to write something like this instead (but skus is not
        # defined correctly for this to work):
        print(self.skus[0].get())

def test():
    line = Line([Parcel("abc", 1), Parcel("bcd", 2)])
    line.print_first_sku()

if __name__ == "__main__":
    test()

I see why the second print does print "a" -- the [0] effectively indexes into the list items, not the list, and .get() is defined to return the first item in the resulting list of characters -- but I don't see how to define "skus" so that I can use it like self.skus[0].get() and get "abc" back.

Any clues? Am I approaching this the wrong way?

License under LGPL?

Hi there. I'd like to use this in a project that is MIT licensed, however the GPL prohibits that. Anything that imports GPL'd code must be GPL'd. Would you consider re-licensing this under the LGPL (which does not have this restriction), or multi-licensing under GPL + a less restrictive license?

I understand if not.

Thanks!

Allow unbound lenses to modify, set, etc

Proposed api along with a motivating example:

from lenses import lens
class Foo(NamedTuple):
    x: int
    y: int

x_plus1 = lens().x.modify(lambda x: x + 1)
y_0 = lens().y.set(0)
both = y_0.add_lens(x_plus1)
fs = [Foo(2, 2), Foo(3, 3)]
new_fs = map(both.run, fs)

Edit: actually the proposed api doesn't make a ton of sense, but I'd still like to be able to compose multiple setters together

Setter -> Traversal

Python's nonlocal state side effects give every setter traversal powers. You should unify them or have a utility to convert them.
I say this because using re.sub (which allows a function argument) for a traversal is turning out much harder than the lambda regex: Setter(lambda f s: re.sub(regex, f, s)) it should be.

Bug: Composing Fork Not Possible

Code

lens.Fork()._compose_optic(lens).modify(lambda x: x)

Traceback

Traceback (most recent call last):
File "", line 1, in
File "/home/viktor/miniconda3/envs/functorch/lib/python3.9/site-packages/lenses/ui/init.py", line 156, in _compose_optic
return UnboundLens(self._optic.compose(optic))
File "/home/viktor/miniconda3/envs/functorch/lib/python3.9/site-packages/lenses/optics/base.py", line 235, in compose
return ComposedLens([self]).compose(other)
File "/home/viktor/miniconda3/envs/functorch/lib/python3.9/site-packages/lenses/optics/base.py", line 617, in compose
if result.kind() is None:
File "/home/viktor/miniconda3/envs/functorch/lib/python3.9/site-packages/lenses/optics/base.py", line 254, in kind
if self._is_kind(optic):
File "/home/viktor/miniconda3/envs/functorch/lib/python3.9/site-packages/lenses/optics/base.py", line 625, in _is_kind
return all(lens._is_kind(cls) for lens in self.lenses)
File "/home/viktor/miniconda3/envs/functorch/lib/python3.9/site-packages/lenses/optics/base.py", line 625, in
return all(lens._is_kind(cls) for lens in self.lenses)
TypeError: 'UnboundLens' object is not callable

Problem

base.py:625

return all(lens._is_kind(cls) for lens in self.lenses)

lens._is_kind is UnboundLens(GetZoomAttrTraversal('_is_kind')) because of this function

API too magic?

The hardest part of writing a library like this is presenting a good pythonic api. While I think that the current api is decent, there are some things that could be better.

I dislike the underscore attributes and how they're not used consistently; it's lens().each_(), but lens().get(). This represents an underlying difference in the implementation, but it might not be a good thing to expose to the user. I may add the underscores to every method of Lens, or remove them. Perhaps I can find a way to separate the building of a lens from the using of one. Part of the problem is that I'm using attribute lookup on the lens to do attribute focusing, which is super convenient, but also sloppy.

I especially dislike the _l attributes. These come from a desire to make functionality easily accessible, but it's not clear to me how useful ZoomAttrLens really is. I may just remove them.

Which is a better api?

from lenses import lens
lens([[1, 2j], 3]).tuple_(lens()[0], lens()[1].wraplist_()).each_().each_().real.get_all()

from lenses import l
l.tuple_(l[0], l[1].wraplist_()).each_().each_().real.get_all(state=[[1, 2j], 3])

The latter just needs l = lens() and is more convenient in many situations but makes bound lenses more difficult to make. How useful are bound lenses? Could do the equivalent of lens = lens() if l is too short.

A few other variations for comparison:

from lenses import lens
lens([[1, 2j], 3]).tuple(lens()[0], lens()[1].wraplist()).each().each().real.get_all()

from lenses import lens, shard
lens([[1, 2j], 3]).tuple_(shard[0], shard[1].wraplist_()).each_().each_().real.get_all_()

import lenses as l
(l.Tuple(l.GetItem(0), l.GetItem(1) & l.WrapList()) & l.Each() & l.Each() & l.GetAttr('real').get_all(state=[[1, 2j], 3])

from lenses import *
(Tuple(lens[0], lens[1] & WrapList()) & Each() & Each() & lens.real).get_all(state=[[1, 2j], 3])

I'm not sure anyone will read this, but it does good to get my thoughts out in the open. Suggestions welcome.

lens.Tuple but for Getters

Hey, it would be useful to have a version of lens.Tuple that accepts two getters.

(Implementation in pseudo-code:) Could make a class GetterTuple(getters) which has type Getter and whose getter returns (getters[0].get(), ...). And then add a GetterTuple to method which creates such a getter and composes it with the current lens.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.