Sha256 Unicode test failed about crypto HOT 7 CLOSED

Here https://mothereff.in/byte-counter for éあ😂 get 3 characters and 9 Bytes as result.
Target: Interp

Total is 9 bytes (using Bytes.ofString) . So what should return length ? Bytes or characters ? In both cases 4 is current development release is wrong.

é = c3a9
あ = e38182
😂 = f09f9882

from crypto.

You should only use latest Haxe GIT version that has correct handling for Unicode Strings. I still think on some platforms str2blks does give invalid results because of surrogate pairs (#if utf16)

from crypto.

We should instead use haxe.io.Bytes.fromString that correctly gives UTF8 bytes. Another option would be a way to iterate on Unicode chars without surrogate pairs but we don't have such API atm.

from crypto.

from crypto.

The current Sha256 require length in bytes. Is it possible to get length in bytes instead count of character ?
The only way to do that now is

 var l = Bytes.ofString(s).length;

but this is some sort of hack. Not sure if anyone else will want to use bytes length instead of count character, but maybe it's possible to be add some other function for that.
About javascript according to specification can use UCS-2 and UTF-16 and most java frameworks used utf-16

from crypto.

What's the length in bytes ? it is encoding dependent. In order to know the length in bytes using UTF8 encoding then yes Bytes.ofString(s).length is one way. Another is to iterate on String char and check in how many bytes they will be encoded in UTF8

from crypto.

Fixed. Most SHA 256 use UTF8 as string input. I add Encoding param, so default should be UTF-8, but can use any of haxe support encoding.
Also I use Bytes.ofString and replace str2blks , so should work with UTF-16 .

Here are example for different encodings ( from stackoverflow https://stackoverflow.com/questions/39589999/correct-sha256-implementation-with-utf-8-characters?rq=1 )

"í" in Windows-1252 is byte ED, which hashes as:
f3df1f9c358ae8eceb8fce7c00614288d113ad55315f4ebb909774a7daadfc84

"í" in UTF-8 is bytes C3 AD, which hashes as:
127035a8ff26256ea0541b5add6dcc3ecdaeea603e606f84e0fd63492fbab2c5

"í" in UTF-16LE is bytes ED 00, which hashes as:
430e2ca27910b5ee6e0ec56a12b81325c763376cb8e25a60362dce9444424f95

from crypto.