Extra space in urldefense v3 about proofpoint-url-decoder HOT 2 OPEN

Thanks for the issue!

I have noticed the extra space in demangled URLs and from a couple emails I've checked, I've verified that in some cases there is an extra space inserted in emails that use quoted-printable encoding (i.e., Content-Transfer-Encoding: quoted-printable) with a Content-Type of `text/plain.

The extra space does not seem to be an issue with emails that are Base64-encoded or of Content-Type text/html, but I don't have many Base64-encoded examples to work with.

Two cases I've seen so far:

1. With angle brackets: <https[:]//urldefense.us/v2/url?u=[...]&e=3D >
2. Without angle brackets: https[:]//urldefense.us/v2/url?u=[...]&e=3D=20

=3D is =, and =20 is the space character: the space is encoded (escaped) if it is the last character of the encoded line.

(There are probably additional variants to these.)

Given that context, because the decoder does not process URLs (or emails) in their quoted-printable-encoded form, it will miss case (2), but ultimately the decoded URL does not look visibly off, as the URL is on a line by itself. (Even if we do process quoted-printables, we would still miss case (1).)

I think handling case (1) (which is much more noticeable) in a general way will require some careful thought, because it's not clear to me how to delimit a URL surrounded by angle brackets (<, >) (or other delimiters), as the space character is not part of the URL (if it was, it should be encoded as %20), without having to handle many different edge cases.

Thus, for now, I don't have a good fix for this issue, but I'll happily consider your (or anyone else's) suggestions if you have one.

from proofpoint-url-decoder.

Thank you very much for looking into this small issue! I understand that a cure could easily be worse than the disease, and anyway this is a bug of urldefense, not of your decoder. I could add an ugly hack to my pipeline (for example something like s/([^>]) >/\1>/g), which probably would have few side-effects, but since the issue is only cosmetical perhaps I could better leave all as it is. I will come back to you if I have a more sensible idea!

from proofpoint-url-decoder.