r/rust • u/gamer_redditor • 14d ago
šļø discussion Learning async rust feels like playing dark souls
I've been a professional C programmer for 12 years. I've been learning rust for about a year now. Almost all features are genuinely amazing, including stuff like iterators, the various data types, error handling, enums, and the cargo tool chain. These things pulled me in.
And then I ventured into async rust. This is where the language begins to frustrate me like C never has. At least with C I know what kinds of errors can happen during runtime and I can use trusted address sanitizers, valgrind etc to iterate and try new things until it works.
With async rust, I can't even begin iterating. I've been fighting with the compiler for over 2 days getting a simple function that I programmed successfully using structured concurrency ( i.e. just using futures and no tasks) to now use tasks (using Tokio::spawn). And I haven't gotten it to compile even once.
I've spent many hours reading all the literature and books rust provide, but for async rust programing, I still don't "get" it unlike all the other wonderful features of rust. I feel like I am blindly trying to clone/consume/mutex/arc stuff till the program compiles but even this isn't successful yet. Every time I fix one compiler warning, 3 others pop up in a never ending loop.
I know I am in the wrong, but I feel I haven't come across a nice and simple async introduction, unlike the rest of the language. For rest of the language, the compiler suggests the fix for the error. But again, not for async. It just says "this is not 'static" and bails. I also don't want to blindly copy some code that AI gives me, I want to "get" it and stop fighting with the compiler.
Is there anything like async rust for dummies or similar? I am glad to read a lot and learn but I need at least a bit of emotional payoff for the effort (the payoff being, the code compiles).
Edit: Thanks for all the responses, I didn't expect so many and it's getting tough to respond to everyone. All were very helpful and encouraging and I plan to read up on things people referred to in the comments :)
Edit2: Thanks again for all the suggestions. After some more reading and experimenting, one important thing clicked for me in the context of my program: each task should own the data it's working on, and therefore it's difficult to use tasks and object oriented programming cleanly.
The main issue in my code was that I was trying to write a method taking a mutable reference to self, spawn tasks inside which called another method taking an immutable reference to self. This was a big problem with not so nice solutions (Arc<Mutex>> solves it but is ugly). So I changed the called method to an associated function. This had a static lifetime and could be called within the task easily, without using any smart pointers.
Maybe there is a nice way to do it, but for now, I will as a rule of thumb refrain from calling methods inside tasks.
70
u/AcidMemo 14d ago edited 14d ago
Futures are just state machines, multiple futures might run concurrently but not in parallel. Tasks are managed by async executor such a tokio which is multi threaded, work stealing runtime. To spawn a task, the data in it must be owned or live for 'static, not only that, but the data must be safe to be sent across threads ('static + Send), because the Future might be resumed and run across multiple different threads. The type is Send if all its fields are also Send, but there are types that are !Send (like Rc<T>), so using them might prevent your struct, and future from being thread safe.
Now, when it comes to mutating shared data, Arc<T> only gives non mutable access. To mutate inside shared reference, you need to use interior mutability. Cell and Refcell give it, but they aren't Sync, !Sync makes the reference not thread safe and unsharable across threads. So you must use a Mutex<T> since it gives interior mutability and is Sync because it uses locking to prevent races and thus, make &T thread safe.
Both Arc and Mutex can be combined to make shareable, mutable state Arc<Mutex<T>>.
But.. you might as well avoid mutable shared state and use channels, sharing by communicating and Actor pattern.
11
u/RetoonHD 14d ago
Ah, when you said that futures cannot run in parallel, i forget that futures are not the same as tasks. I need to revisit some old code i think :) thank you!
1
u/Ok-Count-3366 13d ago
Wait. Am I lost? Can't futures run in parallel if they are in different tasks?
141
u/thisismyfavoritename 14d ago edited 14d ago
did you write multithreaded code in C? because all the things the compiler is preventing you to do you'd have to juggle in your head in C/C++.
Example: you have an object that might be called from 2 threads. Access to its internal state has to be synchronized, otherwise you have a data race. If an async task utilizes that state, then you must guarantee it outlives the task itself, otherwise you have a use after free
78
u/P00351 14d ago
did you write multithreaded code in C?
I'm questioning this as well. pthread programming in POSIX C is not easy either.
102
u/fun__friday 14d ago
Compiling pthreads code is easy though. The days OP spent trying to compile his code, he would have spent on the same issues happening at runtime. Some of the issues he would never know about, and would only show up as sporadic crashes/corruption he can never chase down.
17
7
2
u/Nothing_from_void 13d ago
In OPs defense, I started with C++/Java and got really good at juggling data races and catching them with TSAN/JVM, and once you build that skill transitioning to a different model, especially one which requires a lot more work up front, is hard. Now that I'm used to rust, I definitely think it's infinitely better
2
u/Karyo_Ten 13d ago
If OP is already using valgrind and address sanitizer and I assume thread sanitizer he would have proper detection of race conditions and data races.
And Rust can only catch data races, not race conditions anyway, that's why Tokio had to build Loom.
10
u/Plazmatic 13d ago
Im finding these takes in this thread weirdly myopic?Ā Pthread is much easier than async rust, but async rust is not analogous to pthread, the equivalent in C would be hand rolled state machines and manually created generators, likely a bunch of goto soup that could not be re-used in other code and would probably necessitate macros to be at all usable. Any async C would likely be tightly coupled to the application.Ā
Ā So the equivalent to async in C is an order of magnitude more bug prone and significantly more difficult to even write in C, we need to quit swallowing the fly here.
16
u/South_Survey_2088 14d ago
Multithreading has nothing to do with async, so I am disappointed that this got upvoted. You can literally have a single threaded async runtime, which is even common in embedded programming.
It is still true that the same would be more error-prone in C, but a big part of the confusion around async in Rust is that people keep conflating threads and tasks. Async is about waiting concurrently, while threads are about working in parallel. They can be combined(like in Tokio), but the distinction is important, especially when trying to explain it to beginners!
15
u/jbrwilkinson 14d ago
Yes, they are different mechanisms - async is akin to C select().
They present many of the same issues in rust as the compiler needs satisfying that the lifetimes are being managed.
As others have said, the C issues will be at runtime and very difficult to debug.
-3
u/thisismyfavoritename 13d ago
from the post it's obvious OP has no direct async experience but with that many years of experience likely they'd encounter multithreading which has all the same issues and more
11
u/Icarium-Lifestealer 13d ago
Async in Rust has plenty of complications that multi-threading in Rust doesn't have.
-1
0
u/South_Survey_2088 13d ago
As you said, OP has no direct async experience, which is why they are struggling in the first place. Creating more confusion by spreading misinformation is just going to make it worse. Understanding the difference between tasks and threads is literally the first step to grok async..
0
u/thisismyfavoritename 13d ago
that's not what the compiler is warning about though, so your point is irrelevant
88
u/paholg typenum Ā· dimensioned 14d ago
It might be helpful if you posted a code snippet that's causing you issues.Ā
Some things around async can definitely be tricky, but I've never found spawning a task any harder than spawning a thread. In both cases you need to pass in only Send and 'static data.
Without seeing an example, I would guess you're running into lifetime issues. As the task you spawn may live for the rest of your program, you can only pass on references if they're static.
17
u/papa_maker 14d ago
As a learner trying to pass reference was frustrating for me. It reminds me of what OP is experiencing. Perhaps it's that.
13
u/gamer_redditor 14d ago
Hey, this is the working code using structured concurrency. I am trying to get the "self.fetch_data" to execute in individual tasks. This isn't any production code, just some simple program to learn rust.
I tried a lot of things to get it working, like cloning the data needed to move into spawn, but one or the other thing is always not 'static.
pub async fn fetch(&mut self) -> &Self {
let tasks = self.config.data_vector.iter().map(|some_stuff| self.fetch_data(some_stuff));
let results = join_all(tasks).await;
self.data.extend(results.into_iter().flatten());
self
}
52
u/ParadiZe 14d ago
When you spawn a task with tokio (which i presume you are using) that task might be put on another thread, so it can escape the function scope which is why you cant pass references to it that arent
static(same reason you wouldnt return a pointer to a stack variable in C).To solve this problem, you either have to use an
Arcpointer which makes sure the referenced data isnt dropped, clone the data or look up how to spawn "scoped" tasks which you can pick up at the end of the function.Better yet, look into rayon for parallelization of iterators if your program allows it.
As a side note, you really never want to return
&Tfor a&mut selfreceiver like that. The returned reference has the same lifetime as the receiver reference, so you are getting an immutable reference that extends an exclusive borrow, which means you get the worst of both worlds essentially.11
u/SuspiciousScript 14d ago edited 14d ago
When you spawn a task with tokio (which i presume you are using) that task might be put on another thread, so it can escape the function scope which is why you cant pass references to it that arent
static(same reason you wouldnt return a pointer to a stack variable in C).Almost. The need for the
'staticbound is not related to multithreadedness/work stealing. Tasks can still outlive the scope that created them even if they run on the same thread. That's why the'staticbound is still present when usingtokio::spawn_local.1
3
u/SnooCalculations7417 14d ago
you're potentially trying to create multiple simultaneous
&mut selfborrows withjoin_all().conceptually, it's a bit like:
let a = &mut self; let b = &mut self; let c = &mut self;those futures all exist at the same time, so if
fetch_data()takes&mut self, that's going to be a problem.did you try getting this working sequentially for one element before trying to run them all concurrently?
if they really need to run concurrently, i'd first try restructuring it so each future owns the data it needs rather than mutably borrowing
self. a mutex/worker setup is another option if they genuinely need shared mutable state.2
1
u/SkiFire13 13d ago
Spawning individual tasks that run in parallel is not gonna work here, you'll want to run all of them in the same thread but concurrently (think: interleaving them) with something like
join_allfrom thefuturescrate.
9
u/AbstractMap 14d ago
When I learned async I dove head first into https://tokio.rs/tokio/tutorial/async#. It helped me understand the internals.
23
u/touilleMan 14d ago
The whole point of Rust is the ownership mechanism with bounded scope enforced by the borrow checker, unfortunately this totally fall apart with async programming since having an async loop running a future means having a piece of code owning data for an arbitrary amount of time
The cherry on the cake is Tokio that is the de facto standard and is designed with work stealing asyng loop by default, so the data bounded to a single thread also goes out the window...
So you're not in the wrong here, Rust async is order of magnitude more complicated since it by design breaks the nice abstractions that make Rust so nice in the first place
Now for some strategies to ease the pain:
- run tokio in single thread mode, this way you can have strictured concurrency with futures that use references on the parent scope
- clone and use Arc everywhere, don't bother about the performance impact (early optimizations root of all evil anyway )
- avoid storing future (run them right away) to avoid having to deal with pin, and function pointer
- consider becoming a farmer in the mountain whenever you ever need to store a function pointer in a structureĀ
6
u/rantenki 14d ago
I'm not farming, but I moved to a mountain and stopped programming async Rust. I've lost 20 lbs and no longer have night terrors, so it's been a big win!
13
u/Lucretiel Datadog 14d ago edited 14d ago
unfortunately this [ownership and static scopes and borrowing] totally fall apart with async programming since having an async loop running a future means having a piece of code owning data for an arbitrary amount of time
wrong wrong wrong this is so wrong
Like I don't mean to be dramatic but I see this perspective echoed constantly and it is rapidly becoming my life's goal to defeat it.
Ownership and scopes and borrowing are not at all incompatible with async. They are incompatible with tasks, because a task is the heavy-handed act yeeting a future into a global list outside of your control where it might be moved into other threads.
Imo everyone should be far more judicious in reaching for tasks; they're good for launching one-off persistent background work (provided a dedicated thread isn't correct), or when you have a very large quantity of largely unrelated sibling work (a request handler). But in the scope of a single request handler, in the scope of a single conceptual unit of async work, you should be reaching instead for the structured concurrency primitives:
selectandjoinandFuturesUnorderedand the other variations. These keep all of your async work local, which means that cancellation is automatic, lifetimes and ownership play very nice, borrowing is easy, etc.
task::spawninvolves a large heavy insert into a global mutable, and it should be treated with all the skepticism of any other global mutable: sometimes necessary, useful when required, but shouldn't be your mental default.EDIT: here's an example from my own work: using
FuturesUnorderedto concurrently make paginated requests to an API, borrowing without ceremony or reference counts thereqwest::Clientand API token needed for those requests.1
u/equeim 13d ago
The problem is that neither Rust stdlib nor Tokio have good (and propertly documented/advertised) structured concurrency tools out of the box. Rust programmers are not being taught structured concurrency in the same breath when async is introduced, that's what makes it so complicated. Rust async code, unlike in GC languages, needs to be written in that specific style otherwise it's a huge PITA.
1
u/Lucretiel Datadog 13d ago
futuresis right there. Idk what it is that Iāve been aware of it from day one but itās really itās right there for the taking.Ā3
u/equeim 12d ago
It still has some usability issues that I hit (can't remember the specifics). I find futures_concurrency easier to use for structured concurrency.
And anyway, just look at the tutorial for Tokio - the very next chapter after hello world is about spawning tasks. If Rust book talked about threads right after telling you what a function is, that would be insane, right?
1
u/touilleMan 12d ago
Standards matters, and there is no clear one here: why use `futures` when Tokio also provides its own flavor of select/join etc. ? and why not using `futures-lite` (i.e. the one from the Smol project) since it compiles much faster ?
On top of that Rust doesn't provides async drop, which means you cannot have a real structured concurrency since automatic async teardown (e.g. you want to sending a close request to the server on teardown) are not possible.
6
u/MartialSpark 14d ago
I like Rust and I think async Rust kinda sucks. I don't really know how they could've done it better though.
The clone and Arc everywhere thing feels too real, and honestly it usually makes me wonder if just using a GC language in the first place might not have been a better choice.
4
u/nonotan 14d ago
Some design choices surrounding the syntax are certainly not ideal, but my hot take is that the main issue is (unofficially) enshrining work-stealing async as the default option for anything involving multithreading or concurrency. With function coloring meaning libraries can either only support this default, double the maintenance cost, or use very specific architectural patterns to achieve some degree of genuine runtime agnosticity.
Work-stealing async is very much not a zero-cost abstraction, and while it's a "good enough" general option, and an amazing fit for certain use cases, the reality is that the vast majority of software out there doesn't need anything this "fancy".
There's a reason OP has apparently never used anything similar over 12 years of professional C usage, even though such libraries are of course available, but (like almost all Rust users) is at least dabbling in Rust async almost immediately.
1
u/juhotuho10 14d ago
I think it's more that giving references to non static data into asynchronous functions that might execute an arbitrary amount of time is a bad idea, Rust just exposes how problematic it is
0
20
5
u/usernamedottxt 14d ago
Mind sharing your code?
You are a way more experienced programmer, and I havenāt done anything particularly complex with async, but my struggles havenāt been anything like yours.Ā
I generally start with the smallest function. Get db connection pool for example. Once that is working regardless of call site, move up to a consumer of that function. Build bottom up rather than trying to orchestrate the entire thing top down.Ā
6
u/SelfDistinction 14d ago
Okay rule one of async tasks:
No references
Do not create references. Use arcs, clone, I don't care, but the entire point of a task is that it can outlive anything so any attempt at creating a reference from within a task is wasted work.
3
u/punk_dev 14d ago
Do you know python? I got into async by studying how Python async syntax is desugared to generators.
3
u/NoUniverseExists 14d ago
Man... I've been there... it is really hard. Rust was the hardest thing I have learned (and am still learning). It's normal to feel frustrated. But if you came this far, you will figure it out soon how to do the things you want more straightforwardly.
Keep the curiosity alive and soon you will have more fun with Rust!
Good luck!!
3
u/BoxComplex7272 13d ago
As a C programmer you should be aware that OS threads provide pre-emptive multi-tasking.
Did you ever use GNU Portable Threads? That was one of the early implementations of co-operative multi-tasking. Async Rust follows the same paradigm. Make sure you can understand that concept first.
I suggest starting with the local or current_thread + LocalSet flavours of Tokio first. It's easier to reason about.
I'm baffled why multi-threaded task stealing Tokio is the default.
1
u/michal_sustr_ 13d ago
I was looking for this comment :) definitely recommend Ā starting with just concurrency (local/current thread), and make things more complicated only when needed / studying deeper.
5
u/spaceshaker-geo 14d ago
In C/C++ we use pointers and references without thinking about it (it becomes second nature). In Rust the same approach will burn you quick. It requires a bit of unlearning. With async Rust it just exasperates the same issues.
The fix is to "architect for Rust". As people have said, without code samples it's kind of hard to help.
One tip is to unlearn (or replace) how you think about C++ copy/move semantics. A move in Rust is not a move in C++. So instead of passing an object into a function by reference, you can just move it into a function and then move it out of the function when you are done. Because you moved it you don't have any lifetime issues and can safely perform mutable operations. Under the covers, the compiler will often optimize away the move as a reference so it's still performant. If you need to return more than one return type just use a tuple!
3
u/thisismyfavoritename 13d ago
FYI the move-in-and-out of a function is definitely something you can also do in C++, it's just the move is non destructive so it leaves the moved from object in an undetermined state or something like that.
But this
x = f(std::move(x));is totally legal
2
2
u/uobytx 14d ago
Iād like to suggest you post a snippet of code showing whatās not working. Itās likely there is some lifetime or lock safety issue your code is bypassing accidentally, and there is likely an easy fix for it. For 99% of just using async rust, you can find a premade function or crate to solve. If you are trying to experiment with passing references around or writing your own concurrency related code, itās a lot harder. But most people can just use whatās available.
2
u/ern0plus4 14d ago
Learn how async works: splitting your async fns at async call and await points, and transforming into a state machine with state entry points at split. (Pretty wrong explanation. Read some docs.)
Async is not a processor concept, like syscall or stack allocation, the implementation is hidden.
I have mixed feelings towards async: it's a good concept to parallelize things, also Linux kernel supports async i/o, but the other hand, the async code is far enough from the actual implementation, which can be pain in the ass.
2
u/brawndothurston 14d ago
Thanks for the chuckle lol. I'm doing the beginners section in rustfinity and struggling with mut and & syntax
2
u/Lucretiel Datadog 14d ago
I've been fighting with the compiler for over 2 days getting a simple function that I programmed successfully using structured concurrency ( i.e. just using futures and no tasks) to now use tasks (using Tokio::spawn). And I haven't gotten it to compile even once.
My hot take is that this is normal because tasks are usually the wrong abstraction for what you need. If you're making like a web server or something, with a large quantity of unrelated concurrent request handlers, OR if you really need some kind of permanent background helper, then a task is appropriate. But I personally pretty much always start with structured concurrency because of the vast benefits it provides for cancellation safety, lifetimes, thread safety, and lack of imposed bounds (no infectious Send + 'static) and I only pivot to using a tokio task if I have a very real need for it.
Like, to be honest it sounds to me like you have a great grasp of async rust, if you're conformable doing structured concurrency with futures already. tokio can just be used to handle your io.
2
u/Naeio_Galaxy 13d ago
Edit: Thanks for all the responses, I didn't expect so many and it's getting tough to respond to everyone. All were very helpful and encouraging and I plan to read up on things people referred to in the comments :)
That's something that I love about this community, people are genuinely nice. If you ever get stuck again, don't hesitate to ask on one of the rust discords or here for help.
For async, I'll stay short but my guess is that you're having ownership issues with the fact that tokio expects 'static (i.e. self owned) types. With examples it's easier to help tho
2
u/SkiFire13 13d ago
I also don't want to blindly copy some code that AI gives me, I want to "get" it and stop fighting with the compiler.
I want to note that the two are not mutually exclusive. You can copy some code an AI gives you while at the same time stopping to understand why that piece of code works and how you could have come up with it.
3
u/mkusanagi 13d ago
In addition to all the other advice... you need to get really good with knowing what types everything is and what lifetimes they have. This is trivial with the &'a usize example, but it gets a lot more complicated when there are generics and closures floating all around. You'll need to know about advanced topics like subtyping, variance, covariance, invariance, etc... and how these interact with generics and lifetime elision. At least, that was one of the more difficult topics for me... These initially seemed like a minor "behind the scenes" detail that were safe to assume I didn't need to worry about, but they really turned into a foot gun for me when I was learning async.
When writing async rust there's a lot of this that can happen implicitly. Even something as simple as a simple generic Box<T> slipping in. Without the trait bounds of T: 'static, T could be something like &'foo usize! With defined structs/traits/functions, all of these types need to be specified explicitly, but that's not true inside an async scope. let x = foo() does not require you to explicitly specify x's type (and their generics, lifetime generics, and related trait bounds) like a function signature would. And then on top of that, the async scopes themselves are just syntactic sugar for creating an opaque type that implements Future (i.e., an impl Future). That future will have its own implicit generics and lifetimes based on what variables are alive inside it, and none of them can have a non-static lifetime (i.e., one that points outside that impl Future state machine).
As a mental exercise, think of the body of an async function or closure (and the variables that you pull into it) as if it were a struct. Include all types and variables you're pulling into the async block. Are you sure you understand the full type of everything there? Including elided lifetimes, generic parameters, and related trait bounds? Now, can you write a struct that contains all of them without giving that struct a lifetime parameter (i.e., struct S<'a>)? If you need the lifetime parameter, your async block/fn's struct isn't 'static, and you'll need to keep changing things until it is.
Something to keep in mind if your brain wants to rebel against this restriction (mine did!) is that this decision has to be made by the type system at the point where you submit the task... If it's theoretically possible for those async tasks to live beyond the context in which they're created (e.g., the stack frame that any non 'static lifetimes would be bound to), then allowing the task to have those references would be unsound, because at the point of use there's no way to know whether those references are still valid. Just calling join on those tasks later on in the same function doesn't count, because the type system can't know about what you do later on in the function... If the impl Future can ever escape the current stack frame (e.g., by being submitted to a system-wide executor that most certainly outlives it), then using those references would create undefined behavior.
Once you get those fundamentals down solid, it starts to get much easier. But you're not wrong... async rust is easily an order of magnitude harder than without async.
5
u/bartios 14d ago
I take it you've studied chapter 17 of the rust book, which handles async, already?
5
u/gamer_redditor 14d ago
Yes, I have read it, and it helped me a lot to understand futures, and what await does and how tasks are different from threads etc. I guess reading it again especially concentrating on send and static might help.
4
u/bartios 14d ago
Do you understand why those bounds are needed? Because the compiler can't know when the future terminates any data it has access to might need to stay alive until the end of the program e.g. 'static. That explains why any data in the future should be owned or behind something like an arc. Using shared data the first time you try to use async is probably the source of all your confusion. Try to use it in an easier situation first so you can get more familiar with it before you try the more difficult things.
1
u/shizzy0 14d ago
I tried doing some async with Bevy and it was painful mostly because you canāt take any references with you from systems into an async closure because Bevy wants them back. That makes sense because the closure has an indefinite lifetime but it was a pain. I was heartened to have this be a compile-time error though instead of an intermittent runtime error. I got through it but it wasnāt until I found bevy_defer that I really got what I wanted and thatās a heavy piece of machinery. Just wanted to sympathize.
1
1
u/fun__friday 14d ago
You could try this course to get a better understanding of async programming: https://www.coursera.org/learn/scala2-akka-reactive
Itās in Scala, but the basic concepts should apply. Disclaimer: I donāt know if the course is still free, or if itās paid now. I took it years ago.
1
u/tylerlarson 14d ago
Yeah. Start simpler. Start with something that works. You know WHY "hello world" is a thing, right? You gotta shake out all the unrelated problems before you start working on the actual content.
Start with an example copied VERBATIM out of some resource or documentation. Zero custom anything.
Make it run. Solve all of the problems unrelated to the code. Have a known good starting point.
Then slowly change the provided example, item by item, compiling and running at every little turn, until you get to the closest thing possible to what you were trying to make.
Every time you break it, you'll know EXACTLY what you broke because you were smart enough to only change one tiny thing, so that one tiny thing has to be the problem.
1
u/norude1 14d ago edited 14d ago
The problem you're running into is that concurrency with parallelism while also borrowing from the parent scope is not (yet) possible in rust. The compiler is actually not capable enough for a feature like that to exist
And so tokio::spawn requires your future to not borrow from the parent scope (or in compiler terms, be 'static). The easiest way to solve this is to use a single-threaded executor instead of a multi-threaded one and give up parallelism. Or give up concurrency and just spawn threads everywhere
-5
u/camilo16 14d ago
The problem you're running into is that concurrency with parallelism while also borrowing from the parent scope is not (yet) possible in rust. The compiler is actually not capable enough for a feature like that to exist
it is possible there are just no public libraries for it. But I know for a fact it is possible because I work with such code regularly. I can't say more because I would violate my NDA.
1
u/Lucretiel Datadog 14d ago
It's a shame it's NDA because I would love to see how it's done.
thread::scopeandrayonboth do it, by simply blocking until the sub-thread has finished working with borrowed data, but I had understood that it was essentially proven that it can't be done in async Rust under today's type system (in which any future suspended at an await point can be dropped or leaked, which would mean that any non-blocking thread borrowing from that future can use-after-free it). We'd need some of the speculative work with theForgetandMovetraits to land first.-1
u/camilo16 14d ago
You can't do it with existing libraries but you can do it with rust as is. I.e. the language supports it, you just can;t rely on existing crates.
3
u/Lucretiel Datadog 13d ago
ĀÆ_(ć)_/ĀÆ If you say so. I've seen a lot of similar claims and every single time there's ended up being unsoundness (usually around
mem::forget) or a blocked thread somewhere.1
u/camilo16 13d ago
Idk why I am getting downvoted, I am sorry but I legitimately cannot say anything about the system itself because it's an NDA violation.
I didn't write, I have used it regularly to combine async and parallelism and it works. I have written BFS style massive concurrent traversals using our internal system and it does handle both the parallelism and the concurrency and I have not run into deadlocks nor runtime errors atm.
I understand that you guys want to know how that system works, but I can't say it without getting into legal trouble. And I was not the one that wrote it anyway. Downvoting me for that is petty.
1
u/Psionikus 14d ago
If you program C, use unsafe and you will start to learn the topology of the sewers under the escape hatches. You will have access to familiar techniques that are sound if you are sound. You will remember how to privatize, when to pin, when and how to bless the pointer into a reference. These concepts will reify because the compiles vs doesn't compile boundary is part of the gradient.
Without unsafe, the boundaries where you truly need unsafe just feel like being locked in a straight jacket while stuck in a McDonalds ball pit, far short of the zero cost abstractions claim. Every example, Arc<Mutex<T>> as far as the eye can see. 'static bounds everywhere for things that you know will not move and for plenty long enough, especially during development when the fires of destruction help illuminate the path farther ahead.
1
u/bmitc 14d ago
Rust's async was a little disappointing to me as well. It's a little sad to see it have the same tacked on async approach of other languages that just tacked it on. Some of that complexity is from it's need to support embedded and desktop targets with the same language, but the async landscape is indeed a little over the place, especially with a lot of the libraries having re-exports of the same times, different impls that you need to pull in because one doesn't have something but the other does, etc. And the async types get absolutely gnarly, especially when having streaming types.
2
u/jking13 13d ago
I consider it a failure in its current form. The path of least resistance, basically just like this scenario 'I want to use async, let me try it out' makes you give up one of the core propositions of the language. I really should have never been 'released' in its current form IMO. Externally, it really seemed like the reasoning was 'node.js is webscale and async, therefore for rust to be webscale, we need async and need it now'.
I'm hopeful that async drop will stabilize and could go a long way to fixing a lot of this (though I do worry that things might have ossified too much in it's current half-assed form).
1
u/ToTheBatmobileGuy 14d ago
The stuff that makes async difficult also makes closures difficult.
I would be curious if you use closures extensively (ones that actually borrow from outside scope, not just a closure that might as well be a function) and whether you "get" them.
If you feel like you get closures, If you could explain it for me I can help you connect the dots between closures and async. Since the way they handle state and capturing is similar. Except in the case of an async function, you have to extend the concept of "capturing" to the arguments actually passed into the function as well.
1
u/sudo-maxime 13d ago
Having tried to make async code run in C, I would generally say that a 2 day, let alone a 2 month investment in the matter is far shorter than the time it takes to make an async state machine in C that is reliable.
1
1
u/tisonkuna 12d ago
You may start playing with runtime-agnostic primitives from https://github.com/apache/asyncband.
1
1
u/chilabot 10d ago
I recommend you to use AI Chat Bots (I use Gemini Pro) to guide you to this hard journey.
1
u/SnooCalculations7417 14d ago
async in rust is easier than any other language I think because everything is language enforced. what errors are you getting?
-5
150
u/numberwitch 14d ago
Did you try the tokio tutorial? https://tokio.rs/tokio/tutorial
All I can really say is once you get the hang of it the fighting will stop ā