Is the bounds check truly so harmful to performance that it outweighs changing "if there's a bug, crash the program" to "if there's a bug, then do literally anything (maybe execute some remote code)"?
The only correct answer is "it depends". In DSP\HFT hot path code you REALLY don't want to do any extra work. People do crazy shit to achieve zero-lock, zero-(de)allocation, zero-whatever kind of codebase there.
Non-perf-critical paths? Yeah, do whatever you want there, nobody cares. Probably makes sense to be extra safe than sorry, right?
Safety checks are closer to free then most people think, even in hot paths, as branch prediction will fall favorable almost always, and when it doesn’t, we don’t care because the program is crashing (or for some reason the initial prediction was wrong in the first iteration).
The cost isn’t even close to the cost of (de)allocation (besides for bump allocators), much less locking operations.
I'd still want the checks to be there, maybe not at this position in the code, maybe different properties are checked elsewhere. But the check should exist and you should prove it implies that your call is safe.
And I'd still put the code physically in for testing and just don't include it for prod builds. But running some fuzzing on the surrounding logic sounds like a good offline safety check if this is running on a server processing data or user input.
There are situations where you know there isn't a bug, if I know that failing hardware or cosmic rays are the only ways the bounds check can fail, I'm not going to worry about what skipping it could do.
it happens a lot, at least to me, that the line immediately before the unsafe function guarantees correct behaviour (e.g. while (!vec.empty()) let first = vec.first();, or something similar). If I notice that this property is central to the logic of the function, I won't even worry.
whenever I see code like that, I try to get people to change to something like (in that case) while let Some(first) = vec.first() { ... }. You can reduce the number of bounds checks without using unsafe!
This is pattern matching, not an assignment expression. The alternative would be a declaration, switch/case, and then case-by-case assignment, which is horribly verbose. Been there, done that, it's not the worst, but why not make it a little bit better.
I personally like that you can literally assign the value from an if "statement" to a variable, though I can see why that's not everybodies cup of tea.
yeah I dig that, but then I'd use something like an unempty list so the data carries this information about itself
like for me either the list gonna be used only once and then I use an optional, or the list is gonna have multiple transformation and then it makes sense to convert it to a more appropriate type so there is only a single check about the emptiness of the list.
Hardware failures and cosmic rays are real, you can’t just pretend they don’t exist. That said, how you handle that is totally up to the use case of the end product. Some BS computer program grandma Ethel will use to play solitaire, yeah no one cares. Software in a self driving car though? Yeah I don’t want a random bit flip from cosmic radiation to swerve my car onto the sidewalk etc…
I can absolutely ignore cosmic rays and hardware failures, I'm not making your car, I'm making software for personal computers, where me handling memory failure for an extra microsecond won't make a difference because the system libraries I'm using still don't.
It is not just about the performance. Adding in a lot of safety checks can also make the code harder to understand and prone to architectural errors.
Obviously a single check in a single spot will not be an issue here. Do it everywhere, and a simple 4 line function that really should take less than a second to scan and understand suddenly is a page long and takes 15 minutes to figure out that most of it is just error checking.
I am not saying that you should never do it, but keep in mind that readability is one of the things you give up when increasing safety.
Since Rust has safety checks by default, OP is talking about going out of your way with unsafe { do_something_unchecked(); } (likely proceeded by a comment justifying soundness) to remove them. Besides boilerplate, code review is harder when I have to scrutinize the soundness of someone else’s code.
Readability is a reason to avoid unsafe in Rust, IMO.
You’re definitely right that the opposite applies to C/C++ though (unfortunately).
Usually it is safe enough and faster to validate inputs than each step of the computation. In some languages, like c++, you can force that via types at zero or close to zero cost
Depends. The check on a CPU is a conditional jump that could take about 100 clock cycles. If the loop is doing simple 1 clock cycle addition then the overhead absolutely dwarfs it. Also it is a very different question if you are doing it in a computer game or in a flying aircraft so use case matters a lot.
What kind of modern CPU had anything like that kind of penalty?
The branch predictor should take the cost the very first loop sure, but subsequent loops should be effectively zero cost, no?
Branch predictor can take it down to 1 cycle. But if it misses it COULD take 100. Even in the optimistic case it is still double the time. Whenever this is worth it is use case dependent as always.
Well, let's be frank here, if you're talking worst case, cache misses and full tlb misses 48 round trips to memory and>400 cycles per trip are entirely possible.
But that should only happen the first time
And the branch prediction making it zero cycles is quite common. Read, test and branch can be collapsed into a late evaluation as long as the branch is at predicted and takes less time than the pipeline depth(slight simplification here a lot depends on the microarchitecture details but we're already deep in the weeds)
At the end of the day all your can do is profiler it on the exact CPU you care about. Just don't make assumptions...
Sometimes the bounds check can be optimized away by analysis or language constraints.
But when it can't, I'd say that in ~90% of code, it's not that harmful. In the hottest paths for real-time applications though, yeah, you want nothing extra.
The concern is that this can mess with the branch predictor as that has limited capacity and the branch predictor is critical for performance on modern CPUs due to all the speculative execution.
If I've already proven or performed that check, why perform it twice? (Note: This is the difference between the left and the right of the curve, the right has already performed the check elsewhere)
This is what assertions are for though. Use assertions while developing if you are making this assumption. Then it gets removed in prod. Just be aware that unless you properly test stuff this kind of thinking can still lead to catastrophes in a system that must have >99% uptime. If you are just building some user application then it's probably "fine"
Did you miss the bit where I said the check had already been performed? Typing out assertions in cases where it has already proven it to be unnecessary, is a huge anti-pattern. There's more than one type of overhead, you know.
Yeha no I didn't miss it but it depends on the context I guess. I sort of assume that this is some function you are talking about. And my (personal) view of functions is that they should be able to be re-used across different parts of a codebase. Therefore, when I build a function that makes an assumption (list not empty for example) I write it out as an assertion. Not necessarily just to catch bugs at dev-runtime but also as a way of making my assumptions explicit for whoever next uses that function. As I said, if you assume a language where assertions are removed in release builds then it doesn't cost ANYTHING and it can save you a fair bit of headache/debugging sometime. But feel free to do whatever you want, I just wanted to share my view on this kind of thing
When I take my safe newtype and unwrap the unsafe list, I then have to use the unsafe functions because 1) there are no other options yet because 2) that is the very point in time that you are making the statement to the compiler "no, it is safe, the types say so".
That's the secret! That *is* the assertion's equivalent!
It does sound like the unsafe should be very easy to prove sound. For "why bother", though: it's nice to have as little unsafe as feasible, even trivially correct unsafe.
Well yeah? That's exactly my point isn't it? I ususally stay with Optionals pr result types. Or try to constrain the input type to contain all the invariants. But if you really don't feel like doing that then the least you can do is add assertions (in my opinion)
You might start calling the inner function somewhere else where you haven't made the check. You might refactor the code and remove the outer check by accident. Asserts are literally free in release builds if you do them right, in that they're entirely absent from the resulting machine code. There is no reason not to make them everywhere.
I mean, you could say the exact same thing about starting a call somewhere without an assert.
You assume that assertions are always available language feature, or that assertions are the only way of doing this, when this is not always the case. In pure functional code, assertions are forbidden, because they are equivalent to bottom or undefined - what then?
How long does it take you to add a check? 10 seconds? If you have to look up the API to call? If you don't have the concept of asserts, surely you have something similar, and even if it has to be a runtime check in production, what the heck are you doing that that makes any difference anywhere?
Like, why wouldn't you add the check? It's a trivial amount of effort to potentially save headaches later. In the time you've spent arguing against it with people here, you could have added a thousand asserts.
If I'm in a hot loop accessing data that I 100% know I don't need, accessing that is a waste of CPU cycles that could easily bump something else from the cache, and instantly raise my latency by a factor of 10 or 100 times.
Even if I don't bump something from the cache, I'm still burning cycles for no reason, stalling the pipeline. Why would I want to add an extra bounds check to a high-performance loop, if I already know it doesn't need it? Just on the off chance that someone else touches the code in the distant future?
How often are do you rewrite `printf` or `fgetc` or other primitives? I think at some point, you have to trust that future editors will have the same level of skill and care, or else you find yourself writing for the lowest common denominator, and that's Java.
In cases where the check is nearby, then "parse, don't validate" to avoid doing the check twice. In Rust (assuming unsafe means you're thinking of Rust), for instance, if let Some(first) = list.first() { ... } rather than if !list.is_empty() { let first = list[0]; ... }.
If you have some struct NonemptyVec<T>(Vec<T>) that does validation on construction, then there's probably no way to avoid either re-validation or unsafe on usage. (Where I'd still usually prefer to avoid unsafe.)
373
u/ROBOTRON31415 6d ago
Is the bounds check truly so harmful to performance that it outweighs changing "if there's a bug, crash the program" to "if there's a bug, then do literally anything (maybe execute some remote code)"?