UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 101: invalid continuation byte about nbtlib HOT 8 CLOSED

https://minecraft.gamepedia.com/NBT_format

TAG_String : TAG_Short's payload length, then a UTF-8 string with size length.

from nbtlib.

are you able to parse this?
https://www.dropbox.com/s/dx2y14eg9gc7dxd/6a9c66bb-46dd-4e55-aefc-d1f48fabfa4a.dat?dl=0

from nbtlib.

As @ch-yx pointed out, nbt strings are supposed to be utf-8 encoded. Using iso-8859-1 doesn't fix the problem as it only silences the error by arbitrarily mapping bytes to characters from a completely different encoding.

I looked at the file you provided and there are books with 50 pages of random characters in your player's inventory. These books seem to have been generated by a clientside mod, and while the java code doesn't appear to have a problem with the resulting bytes, they're definitely not valid utf-8 encoded strings.

The books generated by the mod seem to enable an item duplication glitch in Minecraft, but result in non spec-compliant nbt files. I don't think I can do anything about this.

from nbtlib.

I am making my script to detect players who are using such books to dupe on the server. If using ISO-8859-1 isn't the correct solution then what do you suggest for others who come across this issue?
Should I report this as a bug to mojang?

from nbtlib.

raise an exception with raw bytes when cannot be decoded by utf-8？🤔

from nbtlib.

Hmm yeah I think the first thing to do would be to report it on the bug tracker. For your script, I guess that you could detect if someone has a glitched book by catching the UnicodeDecodeError when reading a player's dat file.

def has_book(player_dat):
    try:
        nbtlib.load(player_dat)
    except UnicodeDecodeError:
        return True
    else:
        return False

It's not particularly pretty but it could do the job.

from nbtlib.

I don't think there's a need for a custom exception here, as you can already get the raw bytes that couldn't be decoded through the UnicodeDecodeError.

try:
    nbtlib.load('6a9c66bb-46dd-4e55-aefc-d1f48fabfa4a.dat')
except UnicodeDecodeError as exc:
    print(exc.object)
    print(exc.start, exc.end)

https://docs.python.org/3/library/exceptions.html#UnicodeError

But yeah that could be useful in your script as well 👍

from nbtlib.

Thanks for the suggestions and hopefully it will help others who come across this same issue :)

from nbtlib.