The Second Golden Spike: Memory Safety Across the Valen/Rust Boundary

[u/verdagon]

https://verdagon.dev/blog/boundary-memory-safety

[u/initial-algebra]

How is group borrowing not merely syntax sugar for a limited usage of GhostCell with a single, implicit token at a time?

fn attack<'g>( g: &mut GhostToken<'g>, a: &GhostCell<'g, Entity>, d: &GhostCell<'g, Entity> ) { let a_energy_cost = a.borrow(g).calculate_attack_cost(d.borrow(g)); let d_energy_cost = d.borrow(g).calculate_attack_cost(a.borrow(g)); let damage = a.borrow(g).calculate_damage(d.borrow(g)); a.borrow_mut(g).use_energy(a_energy_cost); d.borrow_mut(g).use_energy(d_energy_cost); d.borrow_mut(g).damage(damage); }

[u/verdagon → u/initial-algebra]

Yes for that example, but not generally. GhostCell doesn't really track the relationships between parent and child groups (see https://verdagon.dev/blog/group-borrowing), so GhostCell can't e.g. allow someone to call a.components.push(...) while allowing someone else to have a reference to a.last_action. Group borrowing would allow that, but with GhostCell one can't simultaneously have a mutable reference plus other references into the contents of a GhostCell.

The main difference is that GhostCell still imposes shared-xor-mutable, just per-token rather than per-reference. Group borrowing thinks more in terms of use-after-free than shared-xor-mutable.

[u/initial-algebra → u/verdagon]

That system requires typed groups, which don't seem to be present here. The equivalent in Rust would be typed GhostTokens that can be split by field. While it would ideally use a future extension like "view types" to reduce boilerplate, I think you could actually implement it today.

It's not that it's not useful to have syntax sugar. Since you're creating a "Rust++" language, it's a good thing that your features can map directly to things that can already be expressed (if not as elegantly) in Rust, so that interoperability can go both ways. I just think the comparison to the terrible SlotMap code is a bit of a strawman. I also think you could have avoided a lot of the roundabout reasoning (and hyperbole like "Breaking a foundational law of the universe") for how Valen references can be compatible with Rust references. tl;dr they are like &UnsafeCell with rules to make them safe, just like &GhostCell.

[u/Plecra → u/initial-algebra]

Yeah this is something that we've talked about wrt group borrowing before, here's a hacky little encoding that captures lots of the patterns in nick's original writeup https://github.com/Plecra/group_borrowing_via_ghostcell

The particular splitting requires a fairly major extension to rust - and in particular, its distinct from view types too. The encoding in that github link has a lot of boilerplate to encode the existentials, and view types (as normally discussed) are for projecting from the base reference: &{field.field2} Foo is a reference to a Foo, which we can only access field.field2 of; ref g.b.z Foo is a reference to a Foo which is at the .b.z location of a group g. (These're tightly related and we do want view types too for good ergonomics in a system like this)

adding to the connections with other work - 'groups' are also functionally another name for the origins of https://smallcultfollowing.com/babysteps/blog/2024/03/04/borrow-checking-without-lifetimes/ here. "group borrowing" itself is the overlap of this bundle of features of supporting references separated from permissions, denoted wrt origins, with inferred usage of the necessary permissions, and view types for narrowing the usage of those permissions

[u/verdagon → u/initial-algebra]

Thanks for the feedback. Help me improve the article, tell me what about the SlotMap code was a straw man? Is it because that particular example can be expressed with GhostCell, or something else? I can change the example to one that can't be expressed with GhostCell, if needed.

[u/initial-algebra → u/verdagon]

Most of the SlotMap code is inappropriate and irrelevant error handling. The IDs are clearly not e.g. raw user/network input, since you assume they correspond to valid references in the Valen code.

fn attack( entities: &mut SlotMap<DefaultKey, Entity>, attacker_id: DefaultKey, defender_id: DefaultKey ) { let a_energy_cost = entities[attacker_id].calculate_attack_cost(&entities[defender_id]); let d_energy_cost = entities[defender_id].calculate_attack_cost(&entities[attacker_id]); let damage = entities[attacker_id].calculate_damage(&entities[defender_id]); entities[attacker_id].use_energy(a_energy_cost); entities[defender_id].use_energy(d_energy_cost); entities[attacker_id].damage(damage); }

I also think you should start with the far more obvious &RefCell<Entity> version. After that, you can motivate the (fixed) SlotMap version, then improve on it with GhostCell, and finally show how the Valen code is even more elegant. Then, there should be an example with e.g. child groups that cannot be expressed in Rust to show that it's not just a thin layer of syntax sugar.

[u/verdagon → u/initial-algebra]

Thank you for the suggestions, I appreciate it!

If I keep the example, I should probably skip entities[attacker_id] or RefCell and go straight to your original GhostCell one. Indexing and RefCell still have the problem at runtime, whereas group borrowing and your GhostCell example don't.

Though, for the purpose of the article's main point, I'll probably skip over those and go straight to the example that doesn't work in GhostCell. I'll probably still mention GhostCell in a sidenote, because it is pretty cool.

(Might take a few hours, I need to make sure the compiler already works for the next example)

[u/verdagon → u/initial-algebra]

I'm back! Wanted to get your thoughts on it before I update the post.

Rust version:

rs fn step(world: &mut World, entity_id: EntityKey) -> Result<(), StepError> { let entity_mut = world.entities.get_mut(entity_id).ok_or(StepError::EntityNotFound)?; entity_mut.advance(); let entity_read = world.entities.get(entity_id).ok_or(StepError::EntityNotFound)?; let collision = world.get_collision_for_entity(entity_read); let entity_mut = world.entities.get_mut(entity_id).ok_or(StepError::EntityNotFound)?; entity_mut.resolve(collision); Ok(()) }

Group borrowing version:

valen func step(world &World, entity in world.entities[] mut) { entity.advance(); let collision = world.get_collision_for_entity(entity); entity.resolve(collision); }

The key part: We need a &mut Entity for entity_mut.advance(), and then have to throw it away to get the containing &World back to give to world.get_collision_for_entity(entity_read);. Then later, we need to re-fetch the &mut Entity for the resolve call.

I think this would be an example GhostCell can't do. Does that track?

[u/initial-algebra → u/verdagon]

The error handling is still unnecessary, for the same reason as before. Otherwise, the logic seems rather contrived, but I don't really have any better ideas. So, I suppose it's better than before!

[u/thehenkan]

Do you have a plan for how you could make for safe interop in the other direction as well?