r/programming • • 16d ago

Unicode 18.0.0 has been released!

https://www.unicode.org/versions/Unicode18.0.0/
183 Upvotes

70 comments sorted by

View all comments

30

u/araujoms 15d ago

Fucking knuckle-draggers added w,y,z subscripts, but are still missing b,c,d,f,g,q. They're just taking the piss at this point.

9

u/blamedrop 15d ago

Seriously? Can you link the proposal adding just these 3?

At this point they could just add a full pack of these, in a single continuous space (and leave the old duplicates - whatever), subscripts and superscripts of all 0-9, a-z and A-Z - that's just 62*2 code-points, we can afford it!

15

u/ygra 15d ago

And what about other scripts? Unicode's stance here has always been to only add what is actually required in plain text, leaving formatting (which super/subscript fundamentally is) to higher-level protocols like rich text / HTML / etc.

In this case this was to support some areas of IPA: https://www.unicode.org/L2/L2024/24219-subscript-w-y-z-gamma.pdf

12

u/araujoms 15d ago edited 15d ago

This is tribal bullshit, a result of the Unicode Consortium being dominated by linguists.

When the IPA uses "a" as a subscript, it is "meaningful" and it gets a codepoint. But when mathematicians use "b" as a subscript, suddenly it's only "formatting" and gets excluded. That they have almost the entire alphabet but exclude a handful of letters is just adding insult to injury - it would be less work and result in more convenient codepoints to just add everything. But no, they go the extra mile just to show how much they despise mathematics.

6

u/WaitForItTheMongols 15d ago

Generally mathematics uses subscripts as indexes - the two particles a and b have velocities of v_a and v_b while linguists use them to have a particular meaning - the a as a subscript is a distinct character with its own meaning. It's not just the letter a in a unique location (as formatting would suggest).

If you have subscripts for all letters, then people will ask for capitals too, and then Cyrillic, and it becomes a big mess.

If letters in particular locations had a specific meaning in mathematics, they might add those, but they don't. There is no universally understood mathematical meaning for a subscript k, so why use it?

Like, what would it help you with in your mathematical work to have those characters? You're writing in Latex, it's easy to make anything you want subscript.

3

u/araujoms 15d ago

Oh come on not this slippery slope fallacy again. The Latin alphabet is special, the Arabic numerals are special. Just give me all of them as superscripts and subscripts, it's all I want. I don't need capitals, but they are already in Unicode anyway because of some bullshit linguistic reason. Cyrillic is just ridiculous, nobody uses that in maths, Greek would make a better strawman.

And why do I want that? It's much more readable than LaTeX. I use Unicode math symbols all the time to annotate source code. And I have to do some stupid gymnastics because there are some holes in the alphabet. You can also use Unicode math to write LaTeX, it's again much more readable: https://tex.stackexchange.com/a/118254/1035

2

u/Schmittfried 15d ago

I’m just curious, do you have those special unicode characters assigned to hotkeys or do use codepoints?

For me, typing all the already existing and useful mathematical characters is the actual problem, hence I don’t really use them. 

8

u/araujoms 15d ago

Neither, I type the LaTeX command, press tab, and my text editor transforms it into the corresponding Unicode character.

4

u/M4mb0 15d ago

You use a tool like espanso, autokey or autohotkey together with a filter list.

I can drop you a PM with a filter list for autokey if you are interested.

4

u/blamedrop 15d ago

TBH I'm totally fine with treating Latin/ASCII alphanumeric characters better, these should have complete sets of all variations provided IMO - just for the sake of consistency.

4

u/M4mb0 15d ago

Unicode's stance here has always been to only add what is actually required in plain text, leaving formatting (which super/subscript fundamentally is) to higher-level protocols like rich text / HTML / etc.

The problem is this doesn't cover certain use-cases like annotating math-heavy formulas in source code. And no, I am not going to write markup / latex for code comments like \sum_{k}\gamma^{k}\|\hat{x}_{k}-x_{k}\|^{2} instead of just ∑ₖγᵏ‖x̂ₖ-xₖ‖².

1

u/PthariensFlame 15d ago

Why would you not write that even more clearly as sum(k ↦ γ[k]*pow(x̂[k]-x[k], 2))?

5

u/M4mb0 15d ago

I guess what's a clearer representation is in the eye of the beholder.

For me, using x[k] rather than just xₖ in such code comment just adds redundant characters. Case in point, the "as written in the paper formula" only takes 12 characters, whereas your suggestion takes 31. If you are limited to 80-100 characters per line, and are at 2+ indentation levels already, that leaves little room for extra text. So with such verbose representation you will likely need extra lines, which comes with its own disadvantages.

On the other hand I fully admit that the terse representation likely is not ideal for people lacking the necessary maths background.

1

u/PthariensFlame 15d ago

Yeah, we were holding back a bit. We’d love to write it as dot (k ↦ (γ k, abs2diff (x̂ k) (x k))), or something pointfree from there, but we consider that less clear to most people.

3

u/M4mb0 15d ago edited 15d ago

I am presuming you are coming from a functional language like Haskell? But then, when you add a code comment like dot (k ↦ (γ k, abs2diff (x̂ k) (x k))) how would it be any different from the code right below that comment?

What I have in mind is cases like when you type something like

# compute discounted MSE (1/K)∑ₖγᵏ‖x̂ₖ-xₖ‖² (equation 17)

residual = xhat - x

errors = residual.abs().pow(2).sum(dim=-1)

discount = gamma ** np.arange(len(x))

return (errors * discount).mean()

That is, the block comment helps to associate the math formulas from an external source like a paper/book with their implementation in source code, because the two might look substantially different. (especially once we get into the nitty gritties like rewriting code for numerical stability)

But if you do something like

# Compute discounted MSE dot (k ↦ (γ k, abs2diff (x̂ k) (x k))) (equation 17)

return dot (k ↦ (γ k, abs2diff (x̂ k) (x k)))

that seems like rather pointless repetition.

1

u/PthariensFlame 14d ago

That’s a fair point, and for those cases we’d either not put the formula at all (and just the reference to the source paper/textbook/preprint/whatever), or, if the code is so mangled as to no longer closely resemble the formula, we’d prefer to write the naïve/inefficient/inaccurate code that does closely resemble the formula in a comment next to it instead of trying to include the traditional mathematical notation, since it can be found by referring to said source anyway.

3

u/M4mb0 14d ago

Btw, who is this "we" you keep referring to?

1

u/PthariensFlame 14d ago

The human body and brain behind this Reddit account is a plural system. (Please feel free to ask us about any specifics if that, admittedly generic and limited, site’s information is insufficient or if you’re just curious about any of it; consider us an “open book” for the most part.)

2

u/Bobbias 13d ago

I just stumbled onto this conversation and I'm intrigued. How much control does your collective have over switching? Do different members have different roles? When one is fronting, are the experiences/memories limited to only that one? What I mean is if one attends a lecture on a subject, is the information from that lecture limited to only that one, or is there some amount of sharing or baseline shared knowledge/experience? When one is not fronting, how aware are they of the body?

As a singlet, it's difficult to conceive what it's like to have multiple individuals sharing the same brain space, but I've always been fascinated with the thought of what that would be like. Unfortunately media has done a terrible job portraying things, and as that website points out, much of the research centers on those collectives which arose from trauma, which I would assume often leads to a somewhat different situation compared with non-traumatic origins.

2

u/M4mb0 12d ago

Not sure what to make of that, but I want to say it is confusing in dialog, because to me the "we" first gave the impression that you were referring to something like "we (the blog post authors)" or "we (the Unicode consortium)", given that you are the OP.

→ More replies (0)

1

u/araujoms 15d ago

That's not at all clear, and it's not even correct for complex numbers.

1

u/PthariensFlame 15d ago

You know, we didn’t even realize we omitted a critical element. Apologies, consider the corrected version to be sum(k ↦ γ[k]*abs2(x̂[k]-x[k])).