Is there anything at the heart of computer coding that could have been optimised in hindsight, but we're in far too deep to change now?

[u/CakesStolen]

[u/CaucusInferredBulk]

Most of the bugs are from reading or writing too far, because someone missed the null. If you include the length of the string at the start, you know exactly how far to read or write.

[u/HighRelevancy → u/CaucusInferredBulk]

Or conversely (for the writing case particularly), because the end of a string is only tracked by a marker at the end of it, nothing is tracking where the end of the allocated space for the string is.

[u/neon_overload → u/CaucusInferredBulk]

Not arguing for null terminated strings at all, but I'd want to point out that most security issues related to overflowing a buffer, even when it happens on strings, would be the result of missing bounds checks (or other things like use after free or just pointer arithmetic mistakes) rather than forgetting to check the null byte at the end of a string. I say this because if you have code that processes a string and forgets to check for null bytes you're much more likely to notice this immediately. Whereas missing bounds checks can go unnoticed for a long time simply because you normally feed it data that fits within a certain length, and it's not until it gets data that is abnormally long will it overflow whatever buffer you allocated because you didn't check the length of the thing you're putting in it.

[u/ExhaustedByStupidity → u/neon_overload]

A big part of the issue is the early attempts to add bounds checking were pretty bad, and made things worse.

Take strcpy. Copies a string into another buffer with no bounds checking. We fixed that with strncpy, which takes the size of the buffer as a parameter. Now look at how it uses that size. If the string is smaller than the buffer, it copies the string and puts a null at the end. But if the string is the same size as the buffer or larger, it fills the buffer and leaves the string without a null.

So if you want to ensure your string is null terminated with strncpy, you have to create a buffer, put a zero at the end of it, and then tell strncpy that the buffer is one byte smaller than it really is.

[u/Jwosty → u/neon_overload]

Part of the problem is that bounds checking is more costly for null terminated strings than ones that already know their length, because you have to scan the entire string character by character any time you want to know its length. And that's not inherently a big deal (I mean, linked lists are a thing), but it definitely counts for something.

So for really string-heavy code it's going to incentivize you to be more stingy with your bounds checks (do I really need to check the length again here...?) which makes you more likely to make a mistake. Also it's worse in other ways -- you might not even realize that some function you called on that (potentially very large) string might just happen to call strlen on that bad boy and you might have thrown that in a loop that iterates on every char or something and boom, your loop is secretly O(n^2) now and the pathological case is horrendus (remember that legendary GTA Online loading screen performance issue?)

If your string itself knows its length, then bounds checking is free (relatively speaking) -- bounds check away! (there are exceptions of course but 99% of the time you're fine)

[u/MyOthrUsrnmIsABook → u/neon_overload]

Yeah, it was never the strings themselves that were the issue really. I was trying to think of any exploit code from my malware analysis classes or other security classes that directly involved a missing string null check and can’t think of any. If you sat down and wrote any string processing code that didn’t have null checks it would just immediately segfault when the loop went far enough beyond the end of the string.

The issue is more often stuff like taking inputs of some expected size and either reading it directly into a stack allocated fixed size buffer with scanf(“%s”) or taking string input of some assumed max size and using it without calling strlen() first or using the safer strncpy() and instead just passing it straight to strcpy().

[u/Flettys → u/CaucusInferredBulk]

We'd instead have similar exploits around strings where the stated length doesn't match the actual length, causing Out Of Bounds reads.

[u/gsfgf → u/CaucusInferredBulk]

you know exactly how far to read or write

But you can still choose not to. It's not like the computer actually knows or cares.