magicstack / immutables Goto Github PK
View Code? Open in Web Editor NEWA high-performance immutable mapping type for Python.
License: Other
immutables's People
Contributors
Stargazers
Watchers
Forkers
ptzagk tednology aweffr rubbish822 nicoddemus dalavancloud 4thel00z pombredanne antonagestam u2386 xmonader applio akhomchenko hakib scsctjgz hugoren jruere tigirardi hukkin marginalhours unicorn-blubber patrickb42 corona10 dmgolembiowski 00mjk marmeladze ofek fantix bryanforbes dish59742 sauravshah31 generalcommission carlwgeorge alvistack cyberflamego beyonddream-productions avipaghadar1729 cnheider sanyam07 mvaled frodejac molaxx tobereplaced ajunlonglive vishalsingh17 mattp- eltoder arpitjain799 arkamar vlifeng romanlakhreev jamestiotio mgorny musicinmybrainimmutables's Issues
Iteration bug
Hi,
I've stumbled upon a bug where I can't access all the entries in the immutables.Map via iteration.
The following bash script should reproduce it.
I tested it on:
Python 3.9.5 (default, Nov 23 2021, 15:27:38)
[GCC 9.3.0]
Python 3.9.9 (main, Nov 21 2021, 03:23:42)
[Clang 13.0.0 (clang-1300.0.29.3)]
PYTHONHASHSEED=0 python <<EOF
import itertools
import immutables
import random
seed=b'b\xe6\xe2\x82\xe5\xc1e|'
r = random.Random(seed)
a = immutables.Map(
zip(
(r.randrange(0, 10000000000) for i in range(820000)),
itertools.repeat(None, 820000),
)
)
len1 = len(a)
len2 = len(tuple(a))
if len1 != len2:
print(f"BADDDD seed:{seed} len(a)={len1} len(tuple(a))={len2}")
EOF
I'll be happy to get help debugging this.
Thanks you
Eli
Data from benchmark and plot do not match
Why is reguar dict faster than HAMT?
Plot does not seem to match with times that can be found alongside the banchmark code: https://gist.github.com/1st1/292e3f0bbe43bd65ff3256f80aa2637d
Help on KeyError being raised
Hi, first of all thanks for this package!
I've found very useful the ability to hash immutable mappings.
Recently I came across a very strange issue and I can not tell why it's happening. I was able to reproduce it with this snippet:
Python 3.8.2 (default, Mar 26 2020, 15:53:00)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from immutables import Map
>>> keys = range(27)
>>> new_entries = dict.fromkeys(keys, True)
>>> m = Map(new_entries)
>>> print(f"{17 in m=}")
17 in m=True
>>>
>>> with m.mutate() as mm:
... for i in keys:
... print(f"Deleting {i=}")
... try:
... del mm[i]
... except KeyError as exc:
... print(f"Did not find: {i}")
... print(f"{i in m=}")
... print(f"{i in mm=}")
... print(type(i))
... raise
... mm.update(new_entries)
... m2 = mm.finish()
... print("ok")
...
Deleting i=0
Deleting i=1
Deleting i=2
Deleting i=3
Deleting i=4
Deleting i=5
Deleting i=6
Deleting i=7
Deleting i=8
Deleting i=9
Deleting i=10
Deleting i=11
Deleting i=12
Deleting i=13
Deleting i=14
Deleting i=15
Deleting i=16
Deleting i=17
Did not find: 17
i in m=True
i in mm=False
<class 'int'>
Traceback (most recent call last):
File "<stdin>", line 5, in <module>
KeyError: 17
>>> m.delete(17)
<immutables.Map({0: True, 1: True, 2: True, 3: True, 4: True, 5: True, 6: True, 7: True, 8: True, 9: True, 10: True, 11: True, 12: True, 13: True
, 14: True, 15: True, 16: True, 18: True, 19: True, 20: True, 21: True, 22: True, 23: True, 24: True, 25: True, 26: True}) at 0x7ff586fb57c0>
>>>Every key after 17 is not able to be deleted on the mutate context manager, but they are in the mapping. I would expect mutate to be able to delete everything.
Is that expectation correct?
Tested on windows:
Python 3.7.3 (default, Apr 24 2019, 15:29:51) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32
>>> import immutables
>>> immutables.__version__
'0.11'
ubuntu WSL
Python 3.8.2 (default, Mar 26 2020, 15:53:00)
[GCC 7.3.0] :: Anaconda, Inc. on linux
>>> import immutables
>>> immutables.__version__
'0.11'
pythoncapi_compat.h is 0BSD not MIT
The LICENSE file says the pythoncapi_compat.h file is licensed under MIT, but actually the license of the file is 0BSD.
Add support to release aarch64 wheels
Problem
On aarch64, pip install immutables builds the wheels from source code and then install it. It requires user to have development environment installed on his system. also, it take some time to build the wheels than downloading and extracting the wheels from pypi.
Resolution
On aarch64, pip install immutables should download the wheels from pypi
@fantix, please let me know your interest on releasing aarch64 wheels. I can help in this.
Some validations are always disabled
Support merge and update operators
Python 3.9 will add the merge (|) and update (|=) operator support for the built-in dict (PEP 584). We should consider adding the same support for immutables.Map.
Adding support for shared_memory allocator for multiprocessing.
How feasible is it modifying static MapObject * map_alloc(void) (and the ...dealloc functions) to use a custom allocator, if buffer parameter is passed to constructor, so a multiprocessing.shared_memory.SharedMemory().buf can be used, similarly to Numpy?
[Security] Potential Secret Leak
It has been noticed that while using edgedb/action-release/merge@master your gpg_key_id 5C468778062D87BF! is present in plaintext. Please ensure that secrets are encrypted or not passed as plain text in github workflows.
Please add a changelog
Looks like the new release might have some great improvements, but it’s harder to tell what they are without a changelog (or any description associated with the release on GitHub, for those of us watching releases).
Thanks for your work on Immutables!
Immutables Class
Maybe i'm missing something in the docs, but is there an immutables class that can be used/inherited from, or is this just a Map?
e.g. I'm hoping for an interface like this
class CustomClass(Immutables.Class):
def __init__(self):
self.x = 'x'
def foo(self):
self.x = 'y'
pass
obj = CustomClass()
with obj.mutate() as obj:
obj.foo()
obj2 = obj.finish()
obj.x # returns x
obj2.x # returns y
Any plans for a list/vector?
I like the trend of increasing functional programming data types for Python. Any plans for a list/vector?
Unable to install from sdist: pyproject.toml [project] requires a name.
This started to happen since yesterday in some CI environments using python:3.10-bullseye image. However, the CI jobs using python:3.8-bullseye and python:3.9-bullseye don't fail.
I've seen this in two different project: one uses tox to manage the virtualenv, the other one uses poetry. I don't really know why they are preferring the source dist over the binary wheels.
Downloading immutables-0.16.tar.gz (84 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.5/84.5 KB 12.3 MB/s eta 0:00:00
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'error'
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [1009 lines of output]
/tmp/pip-build-env-7qjk9iks/overlay/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py:100: _ExperimentalProjectMetadata: Support for project metadata in `pyproject.toml` is still experimental and may be removed (or change) in future releases.
warnings.warn(msg, _ExperimentalProjectMetadata)
configuration error: `project` must contain ['name'] properties
DESCRIPTION:
Data structure for the **project** table inside ``pyproject.toml`` (as
initially defined in :pep:`621`)
GIVEN VALUE:
{
"requires-python": ">=3.6"
}
OFFENDING RULE: 'required'
How to efficiently track and store deltas between two HAMTs
Ideally, I would like to be able to do
x = Map({'a': 2, 'c': 1})
y = x.update({'b': 3: 'c': 2)
z = y - x # magic
z == Map({'b': 3, 'c': 2})
Is there any particularly efficient way to do this in terms of memory and computational time? Ideally, I'd like z to share its data with y in the same way y shares its data with x. One way that comes to mind is
def diff(y, x):
z = y
for k, v in y.items():
if k in x and x[k] == v:
z = z.delete('k')
return z
But this is O(N log N) (for log N get/set). Is there a more efficient way to go about this?
PyMapNoneTest.test_none_collisions fails on 32-bit x86 systems
When running the package's tests on 32-bit x86, I get the following failure:
$ pytest
=============================================================== test session starts ===============================================================
platform linux -- Python 3.7.9, pytest-5.4.2, py-1.8.0, pluggy-0.13.1
rootdir: /home/mgorny/immutables, inifile: pytest.ini, testpaths: tests
plugins: hypothesis-5.18.1, services-2.0.1, backports.unittest-mock-1.5
collected 152 items
tests/test_issue24.py .....sssssss
tests/test_map.py .............................................................sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
tests/test_none_keys.py ......F..sssssssss
==================================================================== FAILURES =====================================================================
_______________________________________________________ PyMapNoneTest.test_none_collisions ________________________________________________________
Traceback (most recent call last):
File "/home/mgorny/immutables/tests/test_none_keys.py", line 47, in test_none_collisions
self.assertEqual(map_mask(c_hash, j*5), idx)
File "/usr/lib/python3.7/unittest/case.py", line 852, in assertEqual
assertion_func(first, second, msg=msg)
File "/usr/lib/python3.7/unittest/case.py", line 845, in _baseAssertEqual
raise self.failureException(msg)
AssertionError: 10 != 9
============================================================= short test summary info =============================================================
FAILED tests/test_none_keys.py::PyMapNoneTest::test_none_collisions - AssertionError: 10 != 9
==================================================== 1 failed, 74 passed, 77 skipped in 17.31s ====================================================
I've been able to bisect this to:
commit 913572c2ef8a4c948bb8b67ff2064d6920e313e7
Author: TIGirardi <[email protected]>
Date: Mon May 18 00:58:10 2020 -0300
Accept None as a key in pure python module (#42)
immutables/map.py | 31 +--
tests/test_none_keys.py | 511 ++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 530 insertions(+), 12 deletions(-)
create mode 100644 tests/test_none_keys.py
Original bug report: https://bugs.gentoo.org/739578
Tests test_none_collisions in CMapNoneTest and PyMapNoneTest fail on armv7l
When building the package for OpenSUSE on armv7l architecture, I get:
[ 304s] ======================================================================
[ 304s] FAIL: test_none_collisions (test_none_keys.CMapNoneTest)
[ 304s] ----------------------------------------------------------------------
[ 304s] Traceback (most recent call last):
[ 304s] File "/home/abuild/rpmbuild/BUILD/immutables-0.14/tests/test_none_keys.py", line 47, in test_none_collisions
[ 304s] self.assertEqual(map_mask(c_hash, j*5), idx)
[ 304s] AssertionError: 13 != 12
[ 304s]
[ 304s] ======================================================================
[ 304s] FAIL: test_none_collisions (test_none_keys.PyMapNoneTest)
[ 304s] ----------------------------------------------------------------------
[ 304s] Traceback (most recent call last):
[ 304s] File "/home/abuild/rpmbuild/BUILD/immutables-0.14/tests/test_none_keys.py", line 47, in test_none_collisions
[ 304s] self.assertEqual(map_mask(c_hash, j*5), idx)
[ 304s] AssertionError: 13 != 12
[ 304s]
[ 304s] ----------------------------------------------------------------------
[ 304s] Ran 152 tests in 98.443s
[ 304s]
[ 304s] FAILED (failures=2)
Add performance comparison vs dict
Hi!
It would be very useful to have a comparison of the performance of the library vs dict for each operation when using small and large datasets.
`immutables.Map` is not JSON Serializable
import json
import immutables
print(json.dumps(immutables.Map({'hello': 'world'})))Traceback (most recent call last):
File "test.py", line 3, in <module>
print(json.dumps(immutables.Map({'hello': 'world'})))
File "/opt/lyft/brew/Cellar/python36/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/__init__.py", line 231, in dumps
return _default_encoder.encode(obj)
File "/opt/lyft/brew/Cellar/python36/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/opt/lyft/brew/Cellar/python36/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/opt/lyft/brew/Cellar/python36/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/encoder.py", line 180, in default
o.__class__.__name__)
TypeError: Object of type 'Map' is not JSON serializable
Do you want my frozenset implemenation using this lib?
if you do here it is:
from typing import Iterable
import immutables
# Design choices:
# - Using a class to allow for typing
# - The class has no methods to ensure all logic is in the functions below.
# - The wrapped map is kept private.
# - To prevent the user from making subtle mistake, we override `__eq__` to raise an error.
# Corollaries:
# - Will not work with operators ootb, e.g. `in`, `==` or `len`.
class ImmutableSet:
def __init__(self, inner):
self._inner = inner
def __eq__(self, _):
raise NotImplementedError(
"Use the functions in this module instead of operators.",
)
def create(iterable: Iterable) -> ImmutableSet:
return ImmutableSet(immutables.Map(map(lambda x: (x, None), iterable)))
EMPTY: ImmutableSet = create([])
def equals(s1: ImmutableSet, s2: ImmutableSet) -> bool:
return s1._inner == s2._inner # noqa: SF01
def length(set: ImmutableSet) -> int:
return len(set._inner) # noqa: SF01
def add(set: ImmutableSet, element) -> ImmutableSet:
return ImmutableSet(set._inner.set(element, None)) # noqa: SF01
def remove(set: ImmutableSet, element) -> ImmutableSet:
return ImmutableSet(set._inner.delete(element)) # noqa: SF01
def contains(set: ImmutableSet, element) -> bool:
return element in set._inner # noqa: SF01
def union(set1: ImmutableSet, set2: ImmutableSet) -> ImmutableSet:
smaller, larger = sorted([set1, set2], key=length)
return ImmutableSet(larger._inner.update(smaller._inner)) # noqa: SF01
def intersection(set1: ImmutableSet, set2: ImmutableSet) -> ImmutableSet:
smaller, larger = sorted([set1, set2], key=length)
for element in smaller._inner: # noqa: SF01
if not contains(larger, element):
smaller = remove(smaller, element)
return smallerand tests:
import time
def test_add():
assert immutable_set.equals(
immutable_set.add(
immutable_set.create([1, 2, 3]),
4,
),
immutable_set.create(
[1, 2, 3, 4],
),
)
def test_remove():
assert immutable_set.equals(
immutable_set.remove(
immutable_set.create([1, 2, 3]),
2,
),
immutable_set.create([1, 3]),
)
def test_contains():
assert immutable_set.contains(immutable_set.create([1, 2, 3]), 3)
def test_not_contains():
assert not immutable_set.contains(immutable_set.create([1, 2, 3]), 4)
def test_union():
assert immutable_set.equals(
immutable_set.union(
immutable_set.create([1, 2, 3, 4]),
immutable_set.create([1, 2, 3]),
),
immutable_set.create([1, 2, 3, 4]),
)
def _is_o_of_1(f, arg1, arg2):
start = time.perf_counter()
f(arg1, arg2)
return time.perf_counter() - start < 0.0001
_large_number = 9999
def test_intersection():
assert immutable_set.equals(
immutable_set.intersection(
immutable_set.create([1, 2]),
immutable_set.create([2]),
),
immutable_set.create([2]),
)
def test_performance_sanity():
assert not _is_o_of_1(
immutable_set.union,
immutable_set.create(range(_large_number)),
immutable_set.create(range(_large_number)),
)
def test_union_performance():
assert _is_o_of_1(
immutable_set.union,
immutable_set.create(range(_large_number)),
immutable_set.create(range(_large_number // 64, _large_number // 32)),
)
def test_intersection_performance():
assert _is_o_of_1(
immutable_set.intersection,
immutable_set.create(range(_large_number)),
immutable_set.create(range(1)),
)Maintain Map insertion order like dict on Python 3.7+
Is that possible?
3.12 support
Howdy,
I know 3.12 isn't out yet. However, other projects (like my cattrs) might want to start testing early to catch regressions. And these projects might use immutables as a test dependency. Hence my plea ;)
Apparent segfault in immutables.Map.__init__
We're getting an intermittent segfault in the contextvars library, on our MacOS CI: https://ci.cryptography.io/blue/rest/organizations/jenkins/pipelines/python-trio/pipelines/trio/branches/PR-575/runs/2/nodes/6/steps/33/log/?start=0
The crash-handler traceback shows it as happening on line 27 of contextvars/__init__.py, which seems to be:
self._data = immutables.Map()So our current hypothesis is that immutables.Map() sometimes segfaults? Does that ring any bells?
We don't know how to reproduce it locally, but I believe that's the python.org build of 3.6.1.
Previous discussion: python-trio/trio#200 (comment)
better document divergence from dict, misc. suggestions
First off, I just discovered this library – very cool! (I'm the author of https://github.com/jab/bidict so am particularly interested in other Mapping implementations. [1])
I noticed the README says
The Map object implements collections.abc.Mapping ABC so working with it is very similar to working with Python dicts.
The only exception are its Map.set() and Map.delete() methods which return a new instance of Map
But right off the bat I noticed some additional discrepancies that seem worth documenting:
- If you pass items via
*argsand/or**kwargsinto theMap(...)initializer they're ignored?
In [37]: Map([(1, 2), (3, 4)])
Out[37]: <immutables.Map({}) at 0x107788ea0>
In [38]: Map({1: 2, 3: 4})
Out[38]: <immutables.Map({}) at 0x107788ea0>
In [39]: Map(one=1, two=2)
Out[39]: <immutables.Map({}) at 0x107788ea0>map.get('missing')does not raiseKeyErrorasdict.get('missing')does, but rather behaves likedict.get('missing', default=None)
Are these bugs or are they intentional? If intentional, I think these are significantly divergent enough to warrant documenting.
Other suggestions
- add an
update()method that returns a newMapbased on the current one with the provided items set - add an
immutables.__version__attribute
[1] I was immediately curious to see what it would take to implement an alternate version of frozenbidict that used immutables.Map for the backing forward and inverse maps, but the divergence from dict.__init__() and dict.get() made me hesitate. I still might put together a recipe for https://bidict.readthedocs.io/extending.html though if I come up with something useful after playing with it some more.
Build broken on Python 3.10
immutables/_map.c: In function ‘map_node_bitmap_new’:
immutables/_map.c:574:19: error: lvalue required as left operand of assignment
574 | Py_SIZE(node) = size;
| ^
immutables/_map.c: In function ‘map_node_collision_new’:
immutables/_map.c:1359:19: error: lvalue required as left operand of assignment
1359 | Py_SIZE(node) = size;
| ^
error: command '/usr/bin/gcc' failed with exit code 1
@vstinner If the motivation is to make PyObject* opaque in the limited ABI, then why did you also make this change in the non-limited ABI?
0.16: pytest is failing
I'm trying to package your module as rpm packag. So I'm using typical in such case build, install and test cycle used on building package from non-root account:
- "setup.py build"
- "setup.py install --root </install/prefix>"
- "pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
May I ask for help because few units are failing:
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-immutables-0.16-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-immutables-0.16-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.11, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/tkloczko/rpmbuild/BUILD/immutables-0.16, configfile: pytest.ini, testpaths: tests
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, case-1.5.3, isort-1.3.0, aspectlib-1.5.2, toolbox-0.5, mock-3.6.1, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, pyfakefs-4.5.0, flaky-3.7.0, benchmark-3.4.1, xdist-2.3.0, pylama-7.7.1, datadir-1.3.1, regressions-2.2.0, cases-3.6.3, hypothesis-6.14.4, xprocess-0.18.1, black-0.3.12, checkdocs-2.7.1, anyio-3.3.0, Faker-8.11.0, asyncio-0.15.1, trio-0.7.0, httpbin-1.0.0, subtests-0.5.0
collected 153 items
tests/test_issue24.py .....sssssss
tests/test_map.py .............................................................sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
tests/test_mypy.py Expected:
...
test.py:30: error: Incompatible types in assignment (expression has type "M...
test.py:31: error: Incompatible types in assignment (expression has type "M...
test.py:33: error: Incompatible types in assignment (expression has type "M...
test.py:34: error: Incompatible types in assignment (expression has type "M...
test.py:44: error: Unexpected keyword argument "three" for "update" of "Map" (diff)
test.py:45: error: Argument 1 to "update" of "Map" has incompatible type "Dict[int, int]"; expected "Union[IterableItems[int, str], Iterable[Tuple[int, str]]]" (diff)
test.py:49: error: Argument 1 to "update" of "Map" has incompatible type "Dict[str, int]"; expected "Union[IterableItems[str, str], Iterable[Tuple[str, str]]]" (diff)
test.py:53: error: Argument 1 to "update" of "Map" has incompatible type "Dict[str, int]"; expected "Union[IterableItems[Union[int, str], str], Iterable[Tuple[Union[int, str], str]]]" (diff)
test.py:57: error: Argument 1 to "update" of "Map" has incompatible type "Dict[int, int]"; expected "Union[IterableItems[str, Union[int, str]], Iterable[Tuple[str, Union[int, str]]]]" (diff)
test.py:63: error: Invalid index type "int" for "MapMutation[str, str]"; expected type "str" (diff)
test.py:64: error: Incompatible types in assignment (expression has type "int", target has type "str") (diff)
test.py:70: note: Revealed type is "immutables._map.Map[builtins.str*, builtins.str*]" (diff)
Actual:
...
test.py:30: error: Incompatible types in assignment (expression has type "M...
test.py:31: error: Incompatible types in assignment (expression has type "M...
test.py:33: error: Incompatible types in assignment (expression has type "M...
test.py:34: error: Incompatible types in assignment (expression has type "M...
test.py:35: error: Incompatible types in assignment (expression has type "Map[str, str]", variable has type "Map[str, Union[int, str]]") (diff)
test.py:45: error: Argument 1 to "update" of "Map" has incompatible type "Dict[int, int]"; expected "Union[Mapping[int, str], Iterable[Tuple[int, str]]]" (diff)
test.py:49: error: Argument 1 to "update" of "Map" has incompatible type "Dict[str, int]"; expected "Union[Mapping[str, str], Iterable[Tuple[str, str]]]" (diff)
test.py:52: error: Argument 1 to "update" of "Map" has incompatible type "Dict[int, str]"; expected "Union[Mapping[Union[int, str], str], Iterable[Tuple[Union[int, str], str]]]" (diff)
test.py:53: error: Argument 1 to "update" of "Map" has incompatible type "Dict[str, int]"; expected "Union[Mapping[Union[int, str], str], Iterable[Tuple[Union[int, str], str]]]" (diff)
test.py:57: error: Argument 1 to "update" of "Map" has incompatible type "Dict[int, int]"; expected "Union[Mapping[str, Union[int, str]], Iterable[Tuple[str, Union[int, str]]]]" (diff)
test.py:63: error: Invalid index type "int" for "MapMutation[str, str]"; expected type "str" (diff)
test.py:64: error: Incompatible types in assignment (expression has type "int", target has type "str") (diff)
test.py:70: note: Revealed type is "immutables._map.Map[builtins.str*, builtins.str*]" (diff)
Alignment of first line difference:
E: test.py:44: error: Unexpected keyword argument "three" for "update" of "...
A: test.py:35: error: Incompatible types in assignment (expression has type...
^
F
tests/test_none_keys.py .........sssssssss
================================================================================= FAILURES =================================================================================
_______________________________________________________________________________ testMypyImmu _______________________________________________________________________________
data: /home/tkloczko/rpmbuild/BUILD/immutables-0.16/tests/test-data/check-immu.test:1:
/usr/lib/python3.8/site-packages/pluggy/hooks.py:286: in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
/usr/lib/python3.8/site-packages/pluggy/manager.py:93: in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
/usr/lib/python3.8/site-packages/pluggy/manager.py:84: in <lambda>
self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
/usr/lib/python3.8/site-packages/_pytest/runner.py:170: in pytest_runtest_call
raise e
/usr/lib/python3.8/site-packages/_pytest/runner.py:162: in pytest_runtest_call
item.runtest()
/usr/lib/python3.8/site-packages/mypy/test/data.py:248: in runtest
suite.run_case(self)
/usr/lib/python3.8/site-packages/mypy/test/testcmdline.py:39: in run_case
test_python_cmdline(testcase, step)
/usr/lib/python3.8/site-packages/mypy/test/testcmdline.py:101: in test_python_cmdline
assert_string_arrays_equal(expected_out, out,
/usr/lib/python3.8/site-packages/mypy/test/helpers.py:117: in assert_string_arrays_equal
raise AssertionError(msg)
E AssertionError: Invalid output (/home/tkloczko/rpmbuild/BUILD/immutables-0.16/tests/test-data/check-immu.test, line 1)
============================================================================= warnings summary =============================================================================
../../../../../usr/lib/python3.8/site-packages/_pytest/config/__init__.py:1183
/usr/lib/python3.8/site-packages/_pytest/config/__init__.py:1183: PytestDeprecationWarning: The --strict option is deprecated, use --strict-markers instead.
self.issue_config_time_warning(
-- Docs: https://docs.pytest.org/en/stable/warnings.html
========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/test_issue24.py:137: C Map is not available
SKIPPED [1] tests/test_issue24.py:126: C Map is not available
SKIPPED [1] tests/test_issue24.py:59: C Map is not available
SKIPPED [1] tests/test_issue24.py:47: C Map is not available
SKIPPED [1] tests/test_issue24.py:88: C Map is not available
SKIPPED [1] tests/test_issue24.py:74: C Map is not available
SKIPPED [1] tests/test_issue24.py:14: C Map is not available
SKIPPED [1] tests/test_map.py:913: C Map is not available
SKIPPED [1] tests/test_map.py:886: C Map is not available
SKIPPED [1] tests/test_map.py:899: C Map is not available
SKIPPED [1] tests/test_map.py:22: C Map is not available
SKIPPED [1] tests/test_map.py:1359: C Map is not available
SKIPPED [1] tests/test_map.py:36: C Map is not available
SKIPPED [1] tests/test_map.py:40: C Map is not available
SKIPPED [1] tests/test_map.py:70: C Map is not available
SKIPPED [1] tests/test_map.py:77: C Map is not available
SKIPPED [1] tests/test_map.py:86: C Map is not available
SKIPPED [1] tests/test_map.py:123: C Map is not available
SKIPPED [1] tests/test_map.py:329: C Map is not available
SKIPPED [1] tests/test_map.py:376: C Map is not available
SKIPPED [1] tests/test_map.py:438: C Map is not available
SKIPPED [1] tests/test_map.py:495: C Map is not available
SKIPPED [1] tests/test_map.py:537: C Map is not available
SKIPPED [1] tests/test_map.py:589: C Map is not available
SKIPPED [1] tests/test_map.py:698: C Map is not available
SKIPPED [1] tests/test_map.py:745: C Map is not available
SKIPPED [1] tests/test_map.py:761: C Map is not available
SKIPPED [1] tests/test_map.py:764: C Map is not available
SKIPPED [1] tests/test_map.py:787: C Map is not available
SKIPPED [1] tests/test_map.py:826: C Map is not available
SKIPPED [1] tests/test_map.py:806: C Map is not available
SKIPPED [1] tests/test_map.py:1353: C Map is not available
SKIPPED [1] tests/test_map.py:596: C Map is not available
SKIPPED [1] tests/test_map.py:617: C Map is not available
SKIPPED [1] tests/test_map.py:638: C Map is not available
SKIPPED [1] tests/test_map.py:643: C Map is not available
SKIPPED [1] tests/test_map.py:649: C Map is not available
SKIPPED [1] tests/test_map.py:668: C Map is not available
SKIPPED [1] tests/test_map.py:916: C Map is not available
SKIPPED [1] tests/test_map.py:1091: C Map is not available
SKIPPED [1] tests/test_map.py:1111: C Map is not available
SKIPPED [1] tests/test_map.py:1127: C Map is not available
SKIPPED [1] tests/test_map.py:1148: C Map is not available
SKIPPED [1] tests/test_map.py:1169: C Map is not available
SKIPPED [1] tests/test_map.py:1178: C Map is not available
SKIPPED [1] tests/test_map.py:1190: C Map is not available
SKIPPED [1] tests/test_map.py:1204: C Map is not available
SKIPPED [1] tests/test_map.py:1211: C Map is not available
SKIPPED [1] tests/test_map.py:1226: C Map is not available
SKIPPED [1] tests/test_map.py:950: C Map is not available
SKIPPED [1] tests/test_map.py:1231: C Map is not available
SKIPPED [1] tests/test_map.py:1260: C Map is not available
SKIPPED [1] tests/test_map.py:963: C Map is not available
SKIPPED [1] tests/test_map.py:973: C Map is not available
SKIPPED [1] tests/test_map.py:992: C Map is not available
SKIPPED [1] tests/test_map.py:1016: C Map is not available
SKIPPED [1] tests/test_map.py:1031: C Map is not available
SKIPPED [1] tests/test_map.py:1054: C Map is not available
SKIPPED [1] tests/test_map.py:1078: C Map is not available
SKIPPED [1] tests/test_map.py:1291: C Map is not available
SKIPPED [1] tests/test_map.py:1341: C Map is not available
SKIPPED [1] tests/test_map.py:167: C Map is not available
SKIPPED [1] tests/test_map.py:257: C Map is not available
SKIPPED [1] tests/test_map.py:674: C Map is not available
SKIPPED [1] tests/test_map.py:692: C Map is not available
SKIPPED [1] tests/test_map.py:849: C Map is not available
SKIPPED [1] tests/test_map.py:856: C Map is not available
SKIPPED [1] tests/test_map.py:868: C Map is not available
SKIPPED [1] tests/test_none_keys.py:293: C Map is not available
SKIPPED [1] tests/test_none_keys.py:461: C Map is not available
SKIPPED [1] tests/test_none_keys.py:62: C Map is not available
SKIPPED [1] tests/test_none_keys.py:108: C Map is not available
SKIPPED [1] tests/test_none_keys.py:159: C Map is not available
SKIPPED [1] tests/test_none_keys.py:251: C Map is not available
SKIPPED [1] tests/test_none_keys.py:42: C Map is not available
SKIPPED [1] tests/test_none_keys.py:373: C Map is not available
SKIPPED [1] tests/test_none_keys.py:86: C Map is not available
FAILED tests/test_mypy.py::ImmuMypyTest::testMypyImmu
=========================================================== 1 failed, 75 passed, 77 skipped, 1 warning in 14.43s ===========================================================
pytest-xprocess reminder::Be sure to terminate the started process by running 'pytest --xkill' if you have not explicitly done so in your fixture with 'xprocess.getinfo(<process_name>).terminate()'.Can't use Map class as generic on Python <=3.6
On python <=3.6 it isn't possible to use Map as a generic type:
>>> from immutables import Map
>>> Map
<class 'immutables._map.Map'>
>>> Map[str, int]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'type' object is not subscriptableOn python 3.7+, the problem does not occur. The likely reason is the use of __class_getitem__, which is new in Python 3.7.
Relevant code snippet below:
Lines 3389 to 3394 in 82e5409
It looks like a metaclass will be needed to fix this. That, or inheriting directly from typing.Mapping. Not sure which is the preferred option.
immutables._map.MapMutation is not iterable / does not implement items()
I am wondering if this is by design or because it hasn't been useful up till now.
My use-case is implementing a middleware type of cache. The cache subscribes to some external data that gets updated and tracks it, and it allows clients to get an immutable view of the cache. The cache can be a nested immutables.Map dictionary. What works is to call mutate and finish for each new piece of data that comes in (N times each if the updated key is at depth N), but this can be too slow when the cache receives a large batch of updates. Using a profiler, I see my code spending ~35% of the time processing the incoming data to be added to the cache and ~65% of the time in the mutate/finish calls.
What I would like to implement is a lazy-finish protocol that calls mutate on updates as needed but not finish. Instead, we traverse the cache and make any needed finish calls when the client requests a view into the cache. What is preventing me from realizing this implementation (I think) is that the immutables._map.MapMutation objects that are returned by the un-finished mutate calls, are not iterable and they don't implement items(), but I need to finish the inner MapMutation objects before I finish the outer ones.
As a nasty hack, I can finish() an outer MapMutation to get its keys and use them to find and recurse into any inner MapMutation values to finish those, before I call finish() a second time on the outer MapMutation to really finish it this time.
Python 3.8 support?
Hi!
I'm one of the maintainers of the immutables feedstock at conda-forge. With the recent release of Python 3.8, conda-forge has started to build its packages for that new version.
Immutables builds cleanly, but one test fails on Python 3.8 (in Windows, Linux and OSX):
+ pytest tests
============================= test session starts ==============================
platform linux -- Python 3.8.0, pytest-3.2.2, py-1.8.0, pluggy-0.4.0
rootdir: $SRC_DIR, inifile:
collected 114 items
tests/test_map.py ...........................................................................................................F......
=================================== FAILURES ===================================
___________________________ CMapTest.test_map_pickle ___________________________
TypeError: cannot pickle 'immutables._map.MapMutation' object
During handling of the above exception, another exception occurred:
self = <tests.test_map.CMapTest testMethod=test_map_pickle>
def test_map_pickle(self):
h = self.Map(a=1, b=2)
for proto in range(pickle.HIGHEST_PROTOCOL):
p = pickle.dumps(h, proto)
uh = pickle.loads(p)
self.assertTrue(isinstance(uh, self.Map))
self.assertEqual(h, uh)
with self.assertRaisesRegex(TypeError, "can't pickle"):
> pickle.dumps(h.mutate())
E AssertionError: "can't pickle" does not match "cannot pickle 'immutables._map.MapMutation' object"
tests/test_map.py:1298: AssertionError
===================== 1 failed, 113 passed in 7.77 seconds =====================
Link: conda-forge/immutables-feedstock#10
I will open a PR shortly.
Binary wheels for Python 3.9
Potential buffer-overflow in map_node_array_assoc of _map.c
Description
When running our analysis tool on immutables, a potential buffer-overflow problem was reported:
missing boundary check when writing to "self->a_array" in the function map_node_array_assoc.
Details
map_node_array_assoc(MapNode_Array *self,
uint32_t shift, int32_t hash,
PyObject *key, PyObject *val, int* added_leaf,
uint64_t mutid)
{
............................
uint32_t idx = map_mask(hash, shift); ----> "hash" may depend on external values (from Python)
MapNode *node = self->a_array[idx]; ----> self->a_array: fixed size of 32
...........................
}
Optional call-path: set (Python) -> map_py_set -> map_assoc -> map_node_assoc -> map_node_array_assoc
Details in description
Suggestion
Add boundary check before write to "self->a_array"
Versions
the main branch
`immutables.Map.__str__` doesn't match `dict.__str__`
In [7]: d = {'a': 1}
In [8]: str(d)
Out[8]: "{'a': 1}"
In [9]: str(immutables.Map(d))
Out[9]: "<immutables.Map({'a': 1}) at 0x108ed56c0>"Segfault with Python 3.13.0b1
$ export CFLAGS='-Og -g -Wall'
$ python3.13 setup.py build_ext -i
running build_ext
building 'immutables._map' extension
creating build
creating build/temp.linux-x86_64-cpython-313
creating build/temp.linux-x86_64-cpython-313/immutables
x86_64-pc-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -Og -g -Wall -fPIC -DNDEBUG=1 -I/usr/include/python3.13 -c immutables/_map.c -o build/temp.linux-x86_64-cpython-313/immutables/_map.o -O2 -std=c99 -fsigned-char -Wall -Wsign-compare -Wconversion
immutables/_map.c: In function ‘map_node_bitmap_dump’:
immutables/_map.c:1287:12: warning: implicit declaration of function ‘_PyLong_Format’; did you mean ‘_PyLong_Copy’? [-Wimplicit-function-declaration]
1287 | tmp2 = _PyLong_Format(tmp1, 2);
| ^~~~~~~~~~~~~~
| _PyLong_Copy
immutables/_map.c:1287:10: warning: assignment to ‘PyObject *’ {aka ‘struct _object *’} from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
1287 | tmp2 = _PyLong_Format(tmp1, 2);
| ^
creating build/lib.linux-x86_64-cpython-313
creating build/lib.linux-x86_64-cpython-313/immutables
x86_64-pc-linux-gnu-gcc -shared -Og -g -Wall build/temp.linux-x86_64-cpython-313/immutables/_map.o -L/usr/lib64 -o build/lib.linux-x86_64-cpython-313/immutables/_map.cpython-313-x86_64-linux-gnu.so
copying build/lib.linux-x86_64-cpython-313/immutables/_map.cpython-313-x86_64-linux-gnu.so -> immutables
$ python -m pytest -s
========================================================= test session starts =========================================================
platform linux -- Python 3.13.0b1, pytest-8.2.0, pluggy-1.5.0
rootdir: /tmp/immutables
configfile: pyproject.toml
testpaths: tests
collected 159 items
tests/test_issue24.py .....Fatal Python error: Segmentation fault
Current thread 0x00007f08dab04740 (most recent call first):
File "/tmp/immutables/tests/test_issue24.py", line 121 in hamt_dump_check_first_return_second
File "/tmp/immutables/tests/test_issue24.py", line 141 in test_array_node_delete_in_place_count
File "/usr/lib/python3.13/unittest/case.py", line 606 in _callTestMethod
File "/usr/lib/python3.13/unittest/case.py", line 651 in run
File "/usr/lib/python3.13/unittest/case.py", line 707 in __call__
File "/tmp/venv/lib/python3.13/site-packages/_pytest/unittest.py", line 343 in runtest
File "/tmp/venv/lib/python3.13/site-packages/_pytest/runner.py", line 173 in pytest_runtest_call
File "/tmp/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/tmp/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/tmp/venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/tmp/venv/lib/python3.13/site-packages/_pytest/runner.py", line 241 in <lambda>
File "/tmp/venv/lib/python3.13/site-packages/_pytest/runner.py", line 341 in from_call
File "/tmp/venv/lib/python3.13/site-packages/_pytest/runner.py", line 240 in call_and_report
File "/tmp/venv/lib/python3.13/site-packages/_pytest/runner.py", line 135 in runtestprotocol
File "/tmp/venv/lib/python3.13/site-packages/_pytest/runner.py", line 116 in pytest_runtest_protocol
File "/tmp/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/tmp/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/tmp/venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/tmp/venv/lib/python3.13/site-packages/_pytest/main.py", line 364 in pytest_runtestloop
File "/tmp/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/tmp/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/tmp/venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/tmp/venv/lib/python3.13/site-packages/_pytest/main.py", line 339 in _main
File "/tmp/venv/lib/python3.13/site-packages/_pytest/main.py", line 285 in wrap_session
File "/tmp/venv/lib/python3.13/site-packages/_pytest/main.py", line 332 in pytest_cmdline_main
File "/tmp/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/tmp/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/tmp/venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/tmp/venv/lib/python3.13/site-packages/_pytest/config/__init__.py", line 178 in main
File "/tmp/venv/lib/python3.13/site-packages/_pytest/config/__init__.py", line 206 in console_main
File "/tmp/venv/lib/python3.13/site-packages/pytest/__main__.py", line 7 in <module>
File "<frozen runpy>", line 88 in _run_code
File "<frozen runpy>", line 198 in _run_module_as_main
Segmentation fault (core dumped)
$ coredumpctl gdb -1
[…]
#0 0x00007f08da29d2bc in ?? () from /usr/lib64/libc.so.6
#1 0x00007f08da24b926 in raise () from /usr/lib64/libc.so.6
#2 <signal handler called>
#3 0x00007f08da5f3a11 in PyObject_Str () from /usr/lib64/libpython3.13.so.1.0
#4 0x00007f08da5a6faf in PyUnicode_FromFormatV () from /usr/lib64/libpython3.13.so.1.0
#5 0x00007f08d8df7ef2 in _map_dump_format (writer=writer@entry=0x7ffe833cc340,
format=format@entry=0x7f08d8dfd03e "bitmap=%S id=%p):\n") at immutables/_map.c:537
#6 0x00007f08d8df8e44 in map_node_bitmap_dump (level=1, writer=0x7ffe833cc340, node=0x7f08d758cdb0) at immutables/_map.c:1292
#7 map_node_dump (node=0x7f08d758cdb0, writer=writer@entry=0x7ffe833cc340, level=level@entry=1) at immutables/_map.c:2261
#8 0x00007f08d8df8d71 in map_node_array_dump (level=0, writer=0x7ffe833cc340, node=0x7f08d7d4f480) at immutables/_map.c:2127
#9 map_node_dump (node=0x7f08d7d4f480, writer=writer@entry=0x7ffe833cc340, level=level@entry=0) at immutables/_map.c:2265
#10 0x00007f08d8df8f9f in map_dump (self=0x7f08d7563140) at immutables/_map.c:2620
#11 map_py_dump (self=0x7f08d7563140, args=<optimized out>) at immutables/_map.c:3191
#12 0x00007f08da5ddf97 in ?? () from /usr/lib64/libpython3.13.so.1.0
#13 0x00007f08da5a51b5 in PyObject_Vectorcall () from /usr/lib64/libpython3.13.so.1.0
#14 0x00007f08da5bb637 in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.13.so.1.0
#15 0x00007f08da601f66 in ?? () from /usr/lib64/libpython3.13.so.1.0
#16 0x00007f08da5f0427 in ?? () from /usr/lib64/libpython3.13.so.1.0
#17 0x00007f08da5bd27e in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.13.so.1.0
#18 0x00007f08da5a5dea in ?? () from /usr/lib64/libpython3.13.so.1.0
#19 0x00007f08da5e7301 in ?? () from /usr/lib64/libpython3.13.so.1.0
#20 0x00007f08da6ac8f3 in ?? () from /usr/lib64/libpython3.13.so.1.0
#21 0x00007f08da5a0db4 in _PyObject_MakeTpCall () from /usr/lib64/libpython3.13.so.1.0
#22 0x00007f08da5c3963 in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.13.so.1.0
#23 0x00007f08da5a5dea in ?? () from /usr/lib64/libpython3.13.so.1.0
#24 0x00007f08da5e7301 in ?? () from /usr/lib64/libpython3.13.so.1.0
#25 0x00007f08da6ac8f3 in ?? () from /usr/lib64/libpython3.13.so.1.0
#26 0x00007f08da5f0289 in ?? () from /usr/lib64/libpython3.13.so.1.0
#27 0x00007f08da5bd27e in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.13.so.1.0
#28 0x00007f08da5a5dea in ?? () from /usr/lib64/libpython3.13.so.1.0
#29 0x00007f08da5e7301 in ?? () from /usr/lib64/libpython3.13.so.1.0
#30 0x00007f08da6ac8f3 in ?? () from /usr/lib64/libpython3.13.so.1.0
#31 0x00007f08da5a0db4 in _PyObject_MakeTpCall () from /usr/lib64/libpython3.13.so.1.0
#32 0x00007f08da5c3963 in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.13.so.1.0
#33 0x00007f08da5a5dea in ?? () from /usr/lib64/libpython3.13.so.1.0
#34 0x00007f08da5e7301 in ?? () from /usr/lib64/libpython3.13.so.1.0
#35 0x00007f08da6ac8f3 in ?? () from /usr/lib64/libpython3.13.so.1.0
#36 0x00007f08da5a0db4 in _PyObject_MakeTpCall () from /usr/lib64/libpython3.13.so.1.0
#37 0x00007f08da5c3963 in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.13.so.1.0
#38 0x00007f08da5a5dea in ?? () from /usr/lib64/libpython3.13.so.1.0
#39 0x00007f08da5e7301 in ?? () from /usr/lib64/libpython3.13.so.1.0
#40 0x00007f08da6ac8f3 in ?? () from /usr/lib64/libpython3.13.so.1.0
#41 0x00007f08da5a0db4 in _PyObject_MakeTpCall () from /usr/lib64/libpython3.13.so.1.0
#42 0x00007f08da5c3963 in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.13.so.1.0
#43 0x00007f08da6590ad in PyEval_EvalCode () from /usr/lib64/libpython3.13.so.1.0
#44 0x00007f08da67469b in ?? () from /usr/lib64/libpython3.13.so.1.0
#45 0x00007f08da5cfcc6 in ?? () from /usr/lib64/libpython3.13.so.1.0
#46 0x00007f08da5a51b5 in PyObject_Vectorcall () from /usr/lib64/libpython3.13.so.1.0
#47 0x00007f08da5bb637 in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.13.so.1.0
#48 0x00007f08da68e1ec in ?? () from /usr/lib64/libpython3.13.so.1.0
#49 0x00007f08da68d9a3 in Py_RunMain () from /usr/lib64/libpython3.13.so.1.0
#50 0x00007f08da6411bb in Py_BytesMain () from /usr/lib64/libpython3.13.so.1.0
#51 0x00007f08da235350 in ?? () from /usr/lib64/libc.so.6
#52 0x00007f08da235409 in __libc_start_main () from /usr/lib64/libc.so.6
#53 0x0000562f9225b085 in _start ()
Cannot install on Windows-Python-2.7 due to: AttributeError: 'tuple' object has no attribute system
On Windows & Python-2.7.10, pip-installing fails with this:
> pip install immutables
Collecting immutables
Downloading https://files.pythonhosted.org/packages/dc/22/e8c6a0b77657612b21f45519b3c08067793371e62bf027af84d49ac9d3e9/immutables-0.6.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "c:\users\joe\appdata\local\temp\pip-install-5psh9p\immutables\setup.py", line 7, in <module>
if platform.uname().system != 'Windows':
AttributeError: 'tuple' object has no attribute 'system'
Now, i noticed that in the project's classifiers it is written only python-3,
but immutable is actually fetched as a transitive dependency of contextvars backported package.
Anyhow, the problem is that:
-
on Python 3,
platform.uname()returns a class "like" a named-tuple:>>> import platform >>> n = platform.uname(); n uname_result(system='Windows', node='hal', release='10', version='10.0.17134', machine='AMD64', processor='Intel64 Family 6 Model 69 Stepping 1, MonkeyIntel') >>> type(n).mro() [<class 'platform.uname_result'>, <class 'tuple'>, <class 'object'>]
-
while on Python-2, it is a simple tuple:
>>> n ('Windows', 'hal', '8', '6.2.9200', 'AMD64', 'Intel64 Family 6 Model 69 Stepping 1, MonkeyIntel') >>> type(n) <type 'tuple'>
Map type arguments aren't introspectible in runtime
Currently I don't see a way of getting the key and value types from a map type. Ideally, the Map.__class_getitem__ should return an object with __origin__ set to Map and __args__ set to a tuple of the key and value types, so that typing.get_origin() and typing.get_args() work with it.
Without this information, libraries like cattrs have no way of correctly serializing and deserializing Map instances.
python 3.8.2 has smal change in AssertionError wording
[ 11s] ======================================================================
[ 11s] FAIL: test_map_pickle (test_map.CMapTest)
[ 11s] ----------------------------------------------------------------------
[ 11s] TypeError: cannot pickle 'immutables._map.MapMutation' object
[ 11s]
[ 11s] During handling of the above exception, another exception occurred:
[ 11s]
[ 11s] Traceback (most recent call last):
[ 11s] File "/home/abuild/rpmbuild/BUILD/immutables-0.11/tests/test_map.py", line 1298, in test_map_pickle
[ 11s] pickle.dumps(h.mutate())
[ 11s] AssertionError: "can't pickle" does not match "cannot pickle 'immutables._map.MapMutation' object"
[ 11s]
[ 11s] ----------------------------------------------------------------------
sdist is missing tests/test-data/check-immu.test
The sdist package at PyPI is missing the tests/test-data/check-immu.test file. Without the file testing fails. Please add the missing tests/test-data/check-immu.test file to sdist to make downstream testing easier. Thank you.
Invalid pyproject.toml
Setuptools complains about a missing version tag in pyproject.toml:
[ 5s] + python3.10 -mpip wheel --no-deps --disable-pip-version-check --use-pep517 --no-build-isolation --progress-bar off --verbose . -w build/
[ 6s] Processing /home/abuild/rpmbuild/BUILD/immutables-0.18
[ 6s] Preparing metadata (pyproject.toml): started
[ 6s] Running command Preparing metadata (pyproject.toml)
[ 6s] configuration error: `project` must contain ['version'] properties
[ 6s] DESCRIPTION:
[ 6s] version should be statically defined in the ``version`` field
[ 6s]
[ 6s] GIVEN VALUE:
[ 6s] {
[ 6s] "name": "immutables",
[ 6s] "requires-python": ">=3.6"
[ 6s] }
[ 6s]
[ 6s] OFFENDING RULE: 'required'
[ 6s]
[ 6s] DEFINITION:
[ 6s] {
[ 6s] "required": [
[ 6s] "version"
[ 6s] ]
[ 6s] }
[ 6s] /usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py:125: _InvalidFile: The given `pyproject.toml` file is invalid and would be ignored.
[ 6s] !!
[ 6s]
[ 6s]
[ 6s] ############################
[ 6s] # Invalid `pyproject.toml` #
[ 6s] ############################
[ 6s]
[ 6s] Any configurations in `pyproject.toml` will be ignored.
[ 6s] Please note that future releases of setuptools will halt the build process
[ 6s] if an invalid file is given.
[ 6s]
[ 6s] To prevent setuptools from considering `pyproject.toml` please
[ 6s] DO NOT include the `[project]` or `[tool.setuptools]` tables in your file.
[ 6s]
[ 6s]
[ 6s] !!
[ 6s]
[ 6s] if _skip_bad_config(project_table, orig_setuptools_table, dist):
[ 6s] running dist_info
...
The build still succeeds and the version of the installed dist-info is correct, but I thought I pass along this warning.
The log is from using setuptools 63.2.0 on openSUSE Tumbleweed, but I also cross checked with setuptools 65.2.0 in a venv and pip wheel -v . which yields similar warnings.
32-bit Wheels for Windows?
Is it possible/easy to get 32-bit wheels for Windows up on PyPI?
I've got a project that targets a 32-bit machine (bleh) and indirectly uses immutables via the loguru package.
Edit: I forgot to mention that I specifically need py36 wheels on 32-bit windows.
Pattern matching isn't working for C implementation
For some reason pattern matching isn't working with the C implementation.
Minimal reproduction:
from immutables import Map
match Map({"foo": 123}):
case {"foo": 123 as exact}:
print(f"{exact=}")
case _:
print("no match")-> "no match"
With the Python implementation, matching works as expected:
from immutables.map import Map
match Map({"foo": 123}):
case {"foo": 123 as exact}:
print(f"{exact=}")
case _:
print("no match")-> "exact=123"
I'm posting here, but really I'm pondering whether this is a bug in Python. The spec PEP reads:
For a mapping pattern to succeed the subject must be a mapping, where being a mapping is defined as its class being one of the following:
- a class that inherits from collections.abc.Mapping
- a Python class that has been registered as a collections.abc.Mapping
- a builtin class that has its Py_TPFLAGS_MAPPING bit set
- a class that inherits from any of the above (including classes defined before a parent’s Mapping registration)
And I'm fairly certain Map should be fulfilling the second predicate. But perhaps things are special for non-Python implementations? Would the key be to set Py_TPFLAGS_MAPPING?
How about comparing performance to frozen dict?
How about comparing performance to frozendict?
`python setup.py test` is deprecated
test command is deprecated in setuptools (pypa/setuptools#1878). We should replace it with either tox, or simply pytest or python -m unittest
Make empty mapping a singleton
We might want to consider making the same optimization for empty mappings that CPython does for the builtin immutable types tuple and frozenset: make the empty mapping a singleton.
For reference, consider:
tuple() is tuple()
# True
frozenset() is frozenset()
# True
import immutables
immutables.Map() is immutables.Map()
# FalseInherit from Map
Any plans to do Map class inheritable? I want detect only possible keys and than I need to make new class, but now I get an error TypeError: type 'immutables._map.Map' is not an acceptable base type
No Python 3.5 Windows Wheel on pypi
There are wheels published for all three platforms (win, mac, linux) for python 3.6 to python 3.8 but for python 3.5, there are only wheels for mac and linux. https://pypi.org/project/immutables/#files
Is it possible to publish Python 3.5 wheel for Windows as well?
Setting from dict
It doesn't look like you support Map({'a':1, 'b':2})
Map.set also looks like it only accepts one pair of key,vals
Any reason why not? It's a bit tedious to set each key,val individually.
mypy integration
Collections should implement mypy Mapping generic to be properly typed
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.