non-ASCII characters in the URL breaks the decoder about proofpoint-url-decoder HOT 3 CLOSED

@agenbite great find, thank you for reporting this issue!

It looks like I did not account for handling the full set UTF-8 characters. I've tested a fix to this, and I should be able to update the scripts later today, after some testing.

Some details, if you are (or anyone else is) interested: the markers in the URL (i.e., **A) specify how many bytes (and not characters or list entries) should be replaced from the base64-encoded replacement string (w60). In this particular case, **A equals 2, and while the replacement string í has length 1, it takes 2 bytes to encode í with UTF-8 (UTF-8 encodes to 1-4 bytes).

The bug, then, was that I coded the URL markers to be the length of the string replacement and not the number of bytes, which led to popping from an empty list. The fix (at the moment) is build the replacement character mapping using the number of bytes by popping the character, calculating the number of bytes it is encoded to in UTF-8, and increment a counter by its size in bytes rather than string length.

from proofpoint-url-decoder.

It works beautifully, @cardi! Thank you so much for the quick response!!

from proofpoint-url-decoder.

@agenbite just pushed a fix. please give it a try and feel free to reopen this issue if the fix doesn't work. thanks again for reporting this!

from proofpoint-url-decoder.