List subtype inference doesn't consider `List` as a possible subtype about nbtlib HOT 18 CLOSED

Fixed in 6cb3b41. This was due to the fact that empty list tags don't have a concrete subtype. Now the subtype of empty list tags wil be inferred from the other list tags in the list.

from nbtlib.

Version 1.3.2 has the fix for compound keys that were interpreted as numbers so I'm closing the issue as I won't be fixing the list problem. I might be adding a "caveats" or "notes" section to the README later that will warn users that as opposed to the spec, nbtlib assumes lists to be deeply uniform.

from nbtlib.

My bad, this should be fixed in 1e7c90a.

from nbtlib.

Closing the issue as the fix is now in release 1.3.0.

from nbtlib.

it seems like it is not fixed completely

lists of a list may contain different type of value.

(by the way,keys start with a number is allowed. )

from nbtlib.

Hmm I think the list problem is going to be a bit tricky to work with. List tags are supposed to have a uniform subtype, meaning that each list element must be of the same type:

>>> parse_nbt("[1,2,hello]")
Traceback (most recent call last):
...
nbtlib.literal.InvalidLiteral: Item 'hello' is not a Int tag at position 10

The thing is that lists containing different types of elements have themselves a different type:

>>> integers = parse_nbt('[1,2,3]')
>>> strings = parse_nbt('[a,b,c]')
>>> type(integers) is List[Int]
True
>>> type(strings) is List[String]
True
>>> List[Int] is not List[String]
True

This means that lists that contain different types of elements can't be placed in another list:

>>> boom = List([integers, strings])
Traceback (most recent call last):
...
nbtlib.tag.IncompatibleItemType: String('a') should be a Int tag

This is not a parsing problem, it has to do with the fact that nbtlib requires list subtypes to be deeply uniform. Changing this behavior will break nbtlib.schema so I'm afraid I'm not going to be able to fix it. Schemas rely on being able to deeply inspect the subtype of lists to cast everything to the appropriate type:

>> Thing = schema('Thing', {
...     'bigList': List[List[schema('ThingElement', {
...         'value': Int
...     })]]
... })
>>> thing = Thing({'bigList': [[{'value': 5}], [], [], [{'value': -1}, {'value': -2}]]})
>>> type(thing['bigList'])
<class 'nbtlib.tag.List[List[ThingElement]]'>

In think that this problem is kind of an edge-case, though. I don't think you would ever run into it, I'm pretty sure Minecraft doesn't use lists that contain lists of different subtypes. IMO the advantages of being able to define schemas outweigh the slight spec mismatch.

from nbtlib.

However, I can do something about compound keys starting with a number.

from nbtlib.

I am 100% agree with you.
I guess that anyone who uses list this way must be mad.XD

I just try to find how reliable your amazing work is.

thanks a lot for fixing my issues these days though all of them are minor bug and not getting annoyed at my poor English.

from nbtlib.

Haha no problem, thanks for taking the time to test everything so thoroughly! English isn't my native language either so don't worry about it ;)

from nbtlib.

Alright so I fixed a few issues with number and string parsing and in 94f9906 you can now use compound keys that would usually be parsed as numbers.

from nbtlib.

I come across some thing interesting:

while boom = List([integers, strings]) leads to a Error,
boom = List[List]([integers, strings]) works well.

Have you fixed it already?

from nbtlib.

i have not found anything wrong with the List[List] object it return yet.

from nbtlib.

Oh that's interesting. I never really thought about it but the behavior actually makes perfect sense.

1. List[List]([integers, strings])

List tags can only contain instances of their respective subtype. If you're putting an object that's not an nbt tag into a list, nbtlib will attempt to cast the object to the list subtype (i.e.: subtype(obj)).

>>> integers = List[Int]([1, 2, 3])
>>> integers[0]
Int(1)

The reason why List[List]([integers, strings]) works is because integers and strings are already nbt tags so nbtlib simply checks that they're both instances of the combined list subtype.

>>> integers = List[Int]([1, 2, 3])
>>> strings = List[String](['a', 'b', 'c'])
>>> combined = List[List]([integers, strings])
>>> combined.subtype is List
True
>>> isinstance(integers, List)
True
>>> isinstance(strings, List)
True

2. List([integers, strings])

For convenience's sake, instantiating a list with the List class directly will infer the list subtype from the specified values.

>>> integers = List([1, 2, Int(3)])
>>> type(integers) is List[Int]
True
>>> integers
List[Int]([Int(1), Int(2), Int(3)])

Since List[List]([integers, strings]) works, this means that in the end, the only reason why List([integers, strings]) doesn't work is because the code that infers the subtype from the specified elements is currently a bit too strict.

3. Implications

In a previous comment, I said that nbtlib requires lists to be deeply uniform, but I didn't think about List[List], which is kind of embarrassing considering I wrote the code... 😅 Anyway, I'm going to make sure that List[List] can be used without any problem.

Converting List[List] instances to and from the binary format already works.

>>> type(combined)
<class 'nbtlib.tag.List[List]'>
>>> b = BytesIO()
>>> combined.write(b)
>>> b.seek(0)
0
>>> List.parse(b) == combined
True

Snbt serialization is already working as well.

>>> print(combined)
[[1, 2, 3], ["a", "b", "c"]]

However, parsing non-deeply uniform list tags from snbt currently raises an exception.

>>> parse_nbt('[[1, 2, 3], ["a", "b", "c"]]')
Traceback (most recent call last):
...
nbtlib.literal.parser.InvalidLiteral: Item '"a"' is not a Int tag at position 27

This is because the parser uses the List class directly to create the list tag instance, and therefore relies on inference to determine the subtype. By adjusting the inference code and making it so that List([integers, strings]) works, the problem will resolve itself.

I'm reopening the issue. nbtlib does in fact support non-deeply uniform list tags through List[List], the problem is simply that the code for inferring the subtype of a list doesn't consider List as a possible subtype.

from nbtlib.

Alright this should be fixed in 69f7c28.

from nbtlib.

The fix is now in release 1.4.5, closing the issue.

from nbtlib.

Sorry. =_=

I just found that

List([List[Int]([Int(1)]),List[List]([List[Int]([])])]) （or [[1], [[]]]）
the outermost List still does not get its subtype properly..........

thus,parse_nbt('[[1], [[]]]') couldn`t works too.

from nbtlib.

please reeopeen this issue

from nbtlib.

And... it's live in v1.4.6.

from nbtlib.