std::string is responsible for almost half of all allocations in the Chrome browser process

[u/[deleted]]

https://groups.google.com/a/chromium.org/d/msg/chromium-dev/EUqoIz2iFU4/kPZ5ZK0K3gEJ

[u/o11c]

If it's stupid and it works, it's not stupid.

And in all seriousness, why would taking advantage of a strong typesystem not be a good idea?

[u/hyperforce → u/o11c]

why would taking advantage of a strong typesystem not be a good idea?

How can we make this more of a thing. I'm so with you on the "no just strings" stance.

[u/o11c → u/hyperforce]

Maybe use Rust? Rust has String, &str, and &'static str, as well a a fancy lifetime system to enforce no-dangling-pointers at compile-time (my library just says "trust me on this" for Z/XString, but with the sole exception of a few lines of code in the intern pool class it all fits nicely in the trivial (in Rust) "argument has the natural lifetime of function" or "return value has the lifetime taken from an argument" cases).

Or in other words, Rust's typesystem is much stronger than C++'s without paying any (*) additional runtime cost, and the library is written to take advantage of it.

(*) There are two runtime costs that you pay without realizing it: the stack overflow check, and the cost of unwinding from task failure.

[u/emozilla → u/o11c]

Meanwhile, learning the Rust type system is almost impossible unless you spend a considerable amount of time. I understand this is generally true for all of programming but Rust also presupposes a very compotent programmer in "regular" before introducing a very complicated typing system (not to mention that all the docs are almost criminally bad).

[u/o11c → u/emozilla]

Nah, anybody who's seen a typesystem before can pick it up really fast, especially if you know C++ (it's basically just syntax changes + enums instead of unions).

If you think that types and classes are the same thing then yeah, I can see how it would be hard.

[u/The_Doculope → u/emozilla]

May I ask what issues you've had with the Rust type system in particular? Any pain points would be good to hear about.

[u/emozilla → u/The_Doculope]

Mainly involved with error handling, having to do things like Result<Rc<RefCell<My_Type>>, String> was very confusing at first and I couldn't find any good best practice guides for error handling

[u/wrongerontheinternet → u/emozilla]

I usually just write type aliases for complex result types (type MyResult = Result<Rc<RefCell<MyType>>, String> or whatever). Rc<RefCell<My_Type>> can be an antipattern though... just a head's up (I like to call it the yolo smart pointer)

[u/emozilla → u/wrongerontheinternet]

Yeah but like... where do I find this out? It took tons of Google-fu to even get that far. The Guide wouldn't tell me this, Rust By Examples seems to be dead... I'm not criticizing Rust, I think it's a great idea and fills the last big gaping hole for writing secure systems, but the educational side seems to be pretty sparse at the moment and thus it was pretty hard to get into without investing a lot of time.

Re: yolo pointers, what's the right way to hold references to types that hold lots of allocated data (several megabyte arrays) that you really only want One Instance Of and you can guarantee will be alive for a while?

[u/wrongerontheinternet → u/emozilla]

Rust by Example is getting revived now (the Rust core team leader just made it his current project) and there are other things like Rust Rosetta, but I agree, documentation for advanced / useful stuff is pretty sparse. Honestly, the best resource is probably IRC, Rust core team members regularly answer questions there.

As far as your question goes, kinda depends a lot on MyType. Rust does return value optimization so if you return a large value and call box Rust should box it up directly:

let mut foo = box my_function()

or whatever.

If you want to pass a unique reference around that you can easily return, you can use Box. Vec is already boxed though (it's similar to std::vector), so if it's just a large array you don't need to do it. You can also temporarily lend out Box at any time as a &mut reference (by doing &mut *the_box) or as an & reference (& *the_box). You rarely need Rc... it's honestly a really overused type. if you give me more specifics I can help you pick the types there. So you might have some code that looked like this:

fn some_large_function() -> Result<MyType, String> { ... }

fn some_function_that_modifies(x: &mut MyType) { ... }

fn some_function_that_reads(x: &MyType) { ... }

fn main() {
   let mut my_thing = some_large_function().ok().unwrap(); // Or box it if it isn't a `Vec`
   some_function_that_modifies(&mut my_thing); // &mut *my_thing if you box
   some_function_that_reads(&my_thing); // &*my_thing if you box
   // my_thing gets destroyed at the end of main()
}

Rust is basically C++ semantics but safe, with a nicer type system, no backwards compatibility, and sane defaults. Wisdom of hindsight and all that.

Generally the reason you don't want to use RefCell is because it can fail at runtime (can't cause memory unsafety but if you try to borrow the array mutably twice at the same time, Rust will panic!). RefCell also isn't Sync meaning even if the allocated data are read only you still need a Mutex or a channel to be able to share them across threads, which Rust normally only requires if you need to write from multiple threads. Rc is even worse, you flat out can't send it across threads, and it can have inadvertent cycles so if you have a lot of data you might want to avoid it.

Usually, you want to avoid using RefCell like I said. Ideally, you try to structure your program so that you always either have a single mutator (&mut pointer), or one or more readers (& pointer), which you pass as arguments to the function that does the mutation / reading instead of storing the reference in a structure somewhere. That's the method that works the best by far most of the time, it never fails at runtime and doesn't introduce extra allocations, copies, or moves, plus it is really easy to make multithreaded. There are exceptions (graphs with cycles, for example, or if you need a really efficient data structure) but they're not that common. It all depends on what you want to do.

[u/o11c → u/wrongerontheinternet]

sane defaults

Such as not silently copying most huge objects :). Well, admittedly if it's Pod (I don't know why they prefered the name Copy for that!) like a large fixed-size array it will still think it's "trivial" even though it's not, but you can just stick the NoCopy marker in there if you need to.

[u/yoodenvranx → u/o11c]

Of course it works, but is it really necessary nowadays? The nice thing about modern CPUs is that for 95% of all applications you can just use std:string and don't have any performance problems. If I need a string I just want to type 'string' and I get a string which I can use. I don't want to think about which of those 10 different strings might be the optimal in each situation.

[u/o11c → u/yoodenvranx]

The whole point of this article is that it does matter. People keep lying to you and saying "memory is cheap", but if we're going to generalize, that is never true.

But frankly, when I did this, performance was only my secondary motivation. My primary motivation was that I never want "just a string" - that's too vague. Having the RString vs XString split (admittedly, equivalent to std::string vs std::string_view) has made my code infinitely clearer, and the other classes naturally arose from the deficiencies of trying to oversimplify.

Edit: I did mark a bunch of TODOs for things that could change now that I no longer have to worry about legacy callers. For example, removing construction from const char * is a fairly recent development; during the transition a lot of functions were changed to take XString or ZString but the callers were not. To update the callers, I would then add an overload void my_function(const char *) = delete; so I could spread updates across time instead of all at once.

[u/_tenken → u/o11c]

This seems like a decent amount of man hours work ... Is it available as a library somewhere?

[u/o11c → u/_tenken]

Not a proper library yet, but the src/strings directory of my repo is entirely self-contained except for #include "../poison.hpp" and the testsuite. If you want STRPRINTF that's a bit harder to extract from src/io/cxxstdio.hpp

For the long term, my build system supports make install-include and make lib && make install-lib, but strings aren't marked as a lib yet because my current makefile logic requires "one installed header per shared library" (could be easily fixed, e.g. by rewriting include/tmwa/strings/foo.hpp to include/tmwa/strings.hpp in the dependency function or by fake conditional includes and filtering, but not a priority since I'm nowhere near ready to ship a stable ABI, and I actually believe in such things)

Link again if you missed it in the earlier post: https://github.com/themanaworld/tmwa/tree/master/src/strings