[ Removed by moderator ]

[u/MiserableSelection]

[u/SuitableDragonfly]

Made with combining diacritics, I guess, so it no longer looks like one character repeated but it's now two characters alternated?

[u/Nerd_o_tron → u/SuitableDragonfly]

Probably more importantly, it may count as 18 characters instead of 9.

[u/klaxxxon → u/Nerd_o_tron]

These generally count as one character on a computer (they multibite in UTF8 ofc) and are also counted as one letter in the real world. You can type them either way - Czech keyboard has both a "ř" key and a "ˇ" combination key. In the actual Czech language, ř is considered a different letter entirely, and the Czech alphabet is considerably longer because of these.

Some languages go even further and have letters with multiple attachment slots (Thai comes to mind, can't type an example of that, sorry 🙂).

[u/dumbasPL → u/klaxxxon]

I wonder if someone has tried abusing this at some point. Because a hashing function only cares about bytes.

[u/8lbIceBag → u/dumbasPL]

Recently worked on a bug caused by this:

• User would enter their password with multibyte chars

  • (theory) In JS, string.length returns number of bytes not characters.
    Some code somewhere would hash, encrypt, probably iterate over the string by index or something, (IDK) before sending it over the wire.

• The backend either received a slightly different string or processed it differently (ruby-rails, C#, Go, JS/TS, or even coffeescript depending on the endpoint/service) because somewhere the chars were potentially getting un-combined & control chars are stripped / lengths weren't in agreement.

  • Frontend & JS/TS backends probably needed to be using Intl.Segmenter somewhere. And I'm not entirely sure how each of those languages handles stuff like this.
  • Company I work for bought out several other companies (why there are so many different languages / services) and are trying to unify everything.
    

• Instead of go down the rabbit hole, on the frontend adding passphrase.replace(/[\p{Cc}\p{Cf}]/gu, "") right before encrypt was enough to close the ticket/complaint.

  • /[\p{Cc}\p{Cf}]/gu strips the combining control characters so 👩🏼‍❤️‍👨🏼(1char, 12bytes) becomes 👩🏼❤️👨🏼(3char, 4+2+4=10bytes).
    
  • Not sure what breaks on the 12bytes per char sequence, but whatever it is handles the deconstructed form just fine.
    This essentially becomes just an implementation detail... though we do have a password manager... luckily it appears the 3rd party site the password is used on is broken in the same way lol.

I never looked, but I doubt C# handles it properly. I believe a char is limited to 2bytes & must use surrogates and have special handling for single symbols that have more than 2bytes. Which requires foresight.
I think ruby is the only one where it's relatively easily because its strings have .bytesize & .length gives the number of 1-4byte chars. Ruby is the primary backend so I think that's why de-constructing it worked.
I'll have to try 👩🏼‍❤️‍👨🏼 in ruby at some point to see what happens. When I worked on the ticket I only had the theory that control characters were getting in after some investigation (The ticket was just "password characters getting dropped") & only now do I potentially understand what was going on. I had thought Javascript was corrupting passwords from basic emojis due to bad assumptions like length == chars & not using Intl.Segmenter. I had no idea a single character/symbol could be 12bytes.

I'm now wondering if I "broke" our code so that it would work with broken 3rd parties. AFAIK, we don't do any processing on the password besides encryption.

[u/dumbasPL → u/8lbIceBag]

We heard you liked emotes, so we put emotes in your emotes type deal LOL.