"Content download not correct size" for 0005000e10112200 about funkiiu HOT 7 OPEN

Same with title ID 0005000e1010f500.

from funkiiu.

I've just done:
wget http://ccs.cdn.wup.shop.nintendo.net/ccs/download/0005000e10112200/00000001
and
wget http://ccs.cdn.wup.shop.nintendo.net/ccs/download/0005000e1010f500/00000001
and in both cases the file is size 32784. In other words, that seems to truly be the file size on the CDN, so your code isn't downloading it wrongly. The problem seems to be that the TMD says it is 32769 which doesn't match, assuming your code is reading the TMD correctly. I haven't checked the TMD parsing separately.

So the question is, what is the correct thing to do in this case? You could truncate the file to match the size the TMD says (or only download the first X bytes based on what the TMD says). Or you could just not bother checking the file sizes, maybe as an optional switch?

from funkiiu.

The file size mismatch problem has been discussed in https://gbatemp.net/threads/funkiiu-incompatibility-with-0005000e10102100-nintento-land-update.464552/ where it has been pointed out that the file size in the TMD is the file size of the decrypted content, not the encrypted content downloaded from the CDN. When decrypted, the file sizes are correct apparently, although I realise that FunKiiU doesn't do decryption so cannot verify that.

https://gbatemp.net/threads/funkiiu-verify-function-to-list-all-incompatible-game.464904/ has a modified version of your code with a nosizecheck parameter which would help with these awkward files.

Arguably, though, you shouldn't be using the TMD to verify sizes of downloaded encypted content at all, in other words, nosizecheck should be the default state, even though the size check does help detect incomplete downloads.

Tricky to know what to suggest, other than adding decryption as a verification step.

from funkiiu.

Thanks, I never knew this problem. I'll see what we can do.
But no decryption :)

from funkiiu.

I said previously that a "no size check" mode should be the default, as it would be more "correct", based on the principal that the size in the TMD isn't actually what one thinks it is.

Thinking about this further, given that 99% of titles have sizes in the TMD which do match the encrypted file sizes, I now think it actually would be good to check the sizes (to detect partial downloads) by default.

The question then is what to do about the titles with incorrect sizes in the TMD.

Maybe you could detect this at runtime, e.g. if after N download attempts the file size is wrong (but consistently wrong to the same size each time), go into a "no size check" mode.

Alternatively you could have a hardcoded list of title IDs with incorrect sizes, and use that to go into the different mode. Or try and persuade the titkekey site to store a flag against the title ID in their database, exposed via the JSON.

It might be worth bearing in mind that presumably the incorrect TMDs may be fixed by Nintendo in future, at least for patches and DLC if not for base titles, so a "blacklisted" title ID might not stay like it forever.

from funkiiu.

In light of the recent discontinuation of Wii U USB Helper, is there any likelihood that this issue could get re-explored?

It seems the downstream developers are aware of this issue too: https://github.com/dojafoja/FunKii-UI/issues/4

from funkiiu.

You can get the correct size with code like this (inside of download_file) but that's quick&dirty / there might be a way without opening two connections to the NUS (but I also can't re-use the connection reading the content-length as that would destroy download resuming).

            infile = urlopen(url)
            cl = infile.headers['Content-Length']
            if cl == None:
                expected_size = None
            else:
                expected_size = int(cl)
            infile.close()

from funkiiu.