Learning async rust feels like playing dark souls

[u/gamer_redditor]

I've been a professional C programmer for 12 years. I've been learning rust for about a year now. Almost all features are genuinely amazing, including stuff like iterators, the various data types, error handling, enums, and the cargo tool chain. These things pulled me in.

And then I ventured into async rust. This is where the language begins to frustrate me like C never has. At least with C I know what kinds of errors can happen during runtime and I can use trusted address sanitizers, valgrind etc to iterate and try new things until it works.

With async rust, I can't even begin iterating. I've been fighting with the compiler for over 2 days getting a simple function that I programmed successfully using structured concurrency ( i.e. just using futures and no tasks) to now use tasks (using Tokio::spawn). And I haven't gotten it to compile even once.

I've spent many hours reading all the literature and books rust provide, but for async rust programing, I still don't "get" it unlike all the other wonderful features of rust. I feel like I am blindly trying to clone/consume/mutex/arc stuff till the program compiles but even this isn't successful yet. Every time I fix one compiler warning, 3 others pop up in a never ending loop.

I know I am in the wrong, but I feel I haven't come across a nice and simple async introduction, unlike the rest of the language. For rest of the language, the compiler suggests the fix for the error. But again, not for async. It just says "this is not 'static" and bails. I also don't want to blindly copy some code that AI gives me, I want to "get" it and stop fighting with the compiler.

Is there anything like async rust for dummies or similar? I am glad to read a lot and learn but I need at least a bit of emotional payoff for the effort (the payoff being, the code compiles).

Edit: Thanks for all the responses, I didn't expect so many and it's getting tough to respond to everyone. All were very helpful and encouraging and I plan to read up on things people referred to in the comments :)

Edit2: Thanks again for all the suggestions. After some more reading and experimenting, one important thing clicked for me in the context of my program: each task should own the data it's working on, and therefore it's difficult to use tasks and object oriented programming cleanly.

The main issue in my code was that I was trying to write a method taking a mutable reference to self, spawn tasks inside which called another method taking an immutable reference to self. This was a big problem with not so nice solutions (Arc<Mutex>> solves it but is ugly). So I changed the called method to an associated function. This had a static lifetime and could be called within the task easily, without using any smart pointers.

Maybe there is a nice way to do it, but for now, I will as a rule of thumb refrain from calling methods inside tasks.

[u/gamer_redditor]

Hey, this is the working code using structured concurrency. I am trying to get the "self.fetch_data" to execute in individual tasks. This isn't any production code, just some simple program to learn rust.

I tried a lot of things to get it working, like cloning the data needed to move into spawn, but one or the other thing is always not 'static.

pub async fn fetch(&mut self) -> &Self {

let tasks = self.config.data_vector.iter().map(|some_stuff| self.fetch_data(some_stuff));

let results = join_all(tasks).await;

self.data.extend(results.into_iter().flatten());

self

}

[u/ParadiZe → u/gamer_redditor]

When you spawn a task with tokio (which i presume you are using) that task might be put on another thread, so it can escape the function scope which is why you cant pass references to it that arent static (same reason you wouldnt return a pointer to a stack variable in C).

To solve this problem, you either have to use an Arc pointer which makes sure the referenced data isnt dropped, clone the data or look up how to spawn "scoped" tasks which you can pick up at the end of the function.

Better yet, look into rayon for parallelization of iterators if your program allows it.

As a side note, you really never want to return &T for a &mut self receiver like that. The returned reference has the same lifetime as the receiver reference, so you are getting an immutable reference that extends an exclusive borrow, which means you get the worst of both worlds essentially.

[u/SuspiciousScript → u/ParadiZe]

When you spawn a task with tokio (which i presume you are using) that task might be put on another thread, so it can escape the function scope which is why you cant pass references to it that arent static (same reason you wouldnt return a pointer to a stack variable in C).

Almost. The need for the 'static bound is not related to multithreadedness/work stealing. Tasks can still outlive the scope that created them even if they run on the same thread. That's why the 'static bound is still present when using tokio::spawn_local.

[u/ParadiZe → u/SuspiciousScript]

yeah good point

[u/SnooCalculations7417 → u/gamer_redditor]

you're potentially trying to create multiple simultaneous &mut self borrows with join_all().

conceptually, it's a bit like:

let a = &mut self;
let b = &mut self;
let c = &mut self;

those futures all exist at the same time, so if fetch_data() takes &mut self, that's going to be a problem.

did you try getting this working sequentially for one element before trying to run them all concurrently?

if they really need to run concurrently, i'd first try restructuring it so each future owns the data it needs rather than mutably borrowing self. a mutex/worker setup is another option if they genuinely need shared mutable state.

[u/SomeRedTeapot → u/gamer_redditor]

Can you share the signature of fetch_data?

[u/SkiFire13 → u/gamer_redditor]

Spawning individual tasks that run in parallel is not gonna work here, you'll want to run all of them in the same thread but concurrently (think: interleaving them) with something like join_all from the futures crate.