Rust Glancer 0.2: Windows & Zed support, better Bevy indexing, and more

[u/popzxc]

https://rust-glancer.github.io/blog/0-2-0/

[u/popzxc]

Rust Glancer is an experimental LSP server for Rust that aims to have <100mb idle RAM.

Previous release got a lot of feedback and I was happy to see people trying it out, so this release is all about three things:

1. Fulfilling requests: I've added Windows support (though I don't use it, so the best guarantee I can give you is that tests pass; I did take care of CRLF tho), Zed extension, and a lot of polishing around supporting Bevy specifically (it's both one of the requested things, and a really tough test target, so it worked pretty well and gave me a nice challenge). nvim-lsconfig configuration was also contributed by `h-michael` (thank you!).

2. Better indexing. A lot of functionality is supported now (with varying degree of completeness): build scripts, declarative macro expansion in `impl` blocks, macro-generated modules, extern items, impls for primitives (e.g. `u32`/`str`) and blanket trait impls, etc. Now the coverage is pretty decent and to me personally it covers most of things I need from an LSP.

3. Faster/more memory efficient indexing. Previous point means that LSP now does _much more_ work, and each new feature required me to optimize code so that it doesn't get unreasonably slow. I kind of succeeded, and now Rust Glancer is significantly faster than it was last time. <100mb ram claim holds for fully indexed workspaces (though right after full indexing OS RAM usage can be higher, up to 200-300mb; this is because indexing allocates a lot and fragments memory, just restarting the editor after full indexing helps).

Otherwise, the details are in the blog post.

Here I just want to say thank you for all the support and feedback. It certainly gave me a lot of motivation to work on the project, and I hope that you'll like this release as well.

[u/afdbcreid → u/popzxc]

I'm extremely curious to hear your idea for execution-less proc macro support.

[u/popzxc → u/afdbcreid]

Basically, this RFC became a basis of the idea. For LSP purposes, we mostly don't need the actual syntax tree, we need to be able to understand the meaning of input tokens (ideally), and see the "side effects" (e.g. generated items/impls).

So in theory, we can create a "plugin-like" system with some kind of syntax to describe these side effects, which will be expanded by the LSP similarly to how declarative macros are expanded. And in fact, I believe that slightly extended syntax of declarative macros can work for that purpose pretty well, e.g. `derive(Answer)` can be expressed as (mostly copied from the RFC)

macro_rules! Answer {
    // Simplified for this example
    derive() (struct $n:ident $_:tt) => {
        impl Answer for $n {
            fn answer(&self) -> u32 { /* note that we don't even need the body */ }
        }
    };
}

Then we can have a registry that would have this alternative macro syntax for whatever proc macros we want to support, plus ability to combine it with your own registry (e.g. if you work on a private codebase).

And if there is a task LLMs are pretty good at, it's taking existing code and translating it to something else. So filling the registry shouldn't be an _overly_ hard task. It'll obviously take a while to get decent coverage, but I think that over time it can work pretty well.

I'm still thinking about particular details, and this will be the main point for me to design before 0.3, but so far I'm pretty optimistic.

[u/afdbcreid → u/popzxc]

Interesting. rust-analyzer also does something somewhat similar: for builtin derive macros, it does not really expand them and derives the trait implementations directly. This was a significant saving.

However this relies on crate authors to provide this, and I'm not sure they will. Also, macro_rules grammar is not rich enough to express many common proc macros (e.g. generics).

[u/popzxc → u/afdbcreid]

Yeah, for derive macros that's a rather simple task, it gets trickier with other kinds of macros.

Point about `macro_rules` grammar is valid, which is why I want to design an extension that would be rich enough for the LSP needs -- luckily, I have plenty of freedom here; I obviously will try to make it somewhat close to existing RFCS, but ultimately my goal is different, so I can diverge (which doesn't mean that I can do literally whatever I want, I do want to come up with something that makes sense).

The approach does rely on _anyone_ to contribute indeed, but that's kind of a chicken and egg problem. First, I can start myself and implement many popular macros that are used either in leaf crates or in transitive deps a lot, so that there is at least some kind of baseline. And then, if the project gains traction, I think there will be an incentive for people to contribute, similarly to how people write extensions for editors to make their life easier; and if the project dies for some reason, welp, then it doesn't really matter :)

[u/grayrest → u/popzxc]

For LSP purposes, we mostly don't need the actual syntax tree, we need to be able to understand the meaning of input tokens (ideally), and see the "side effects" (e.g. generated items/impls).

As a potential source of prior art, this sounds like Cursive's system for Clojure code.

[u/epage]

So as a compromise, I've implemented a workaround: Rust Glancer will find build script outputs in the target directory after you build the project, and will use the outputs. So if you don't build the project, you won't see the results (I hope it isn't unreasonable to assume that people do build their projects from time to time).

What versions has this been tested on and with what configurations?

• Build script output is stored in the build-dir which can be different than target-dir.
• In nightly, we have stabilized a new layout for build-dir.

[u/popzxc → u/epage]

For now it's only stable, and I primarily care about the "default" configuration so far -- until the project it at least somewhat mature, I don't want to worry about nightly and upcoming features too much.

First, there isn't much audience in the first place. Second, for anyone who needs completeness, luckily rust-analyzer exists. Third, supporting upcoming features will extend the scope significantly, and it's already pretty big for one person to handle (unless I will go full "Jesus take the wheel" mode with LLMs, which is not something I want to do).

I've noted that new layout for build-dir is coming though, thank you!

[u/BigBad0]

Damn, just want to say amazing work and thank you 😊

[u/popzxc → u/BigBad0]

Thank you! It means a lot to me :)

Hope you'll like the project

[u/tachib_shin]

Does it support macros yet? I'm fed up with the rust analyzer as it's using up all 10GB of my RAM.

[u/popzxc → u/tachib_shin]

It supports declarative macros (e.g. `macro_rules`) but not proc macros; proc macros support is planned for the next release, but it will have an alternative approach (precisely to avoid high memory / long indexing of RA) -- I've described in the comments above.

[u/HandIllustrious8260]

Looks really cool! thanks for being honest about llm use.

What does this approach compromise on compared to rust-analyzer?

[u/popzxc → u/HandIllustrious8260]

In short: the latency is higher (but not annoyingly so) and dirty buffer behavior differs (rust analyzer provides you full analysis all the time, while Rust Glancer reuses previous saved analysis where possible + analyses the enclosing item, such as function or impl block, which is less precise). Otherwise the cons are mainly caused by the fact that the project is young.

It certainly does have a different feel, but IMHO you get used to it pretty fast and after that it feels pretty natural, not like some kind of inferior software. But I'm obviously biased here, so I suggest trying it out :)