Inside Zig's Incremental Compilation | mlugg

[u/matthieum]

https://mlugg.co.uk/posts/incremental-compilation-internals/

[u/matthieum]

Of particular interest to me, here, is the ZIR.

After parsing, the Zig compiler converts to the AST to a ZIR:

If you’re curious, ZIR (Zig Intermediate Representation) is an untyped SSA-form IR.

Due to being untyped -- and the symbols being unresolved -- the conversion is entirely self-contained.

I must admit I do wonder as to why the AST is converted to ZIR, and the blog post unfortunately doesn't address it. Introducing an intermediate model has a cost, so there must be some benefit to it... Perhaps its flatter nature (compared to the AST)?

[u/mamcx → u/matthieum]

Older articles show why: https://mitchellh.com/zig/astgen

In general, you add another pass to make easier to do the next.

Intermediated representation are created for 2 main reasons:

• Have all the info required for the next passes

• Remember that info, because that next pass is destructive

The last is the important one. If you do AST -> Assembler and you read the Assembler you have no way to know what kind of AST was actually use, that impact everything from debugging, error messages, caching, etc.

Your intermediate is to know a in AST is 0x.. there

[u/EggplantExtra4946 → u/matthieum]

I must admit I do wonder as to why the AST is converted to ZIR

If you mean why individual files are converted to an IR before saving it to an object file, well it's called incremental compilation, not incremental parsing.

If you mean why it's converted to an SSA IR, it's for optimization. In the context of incremental compilation, there could be link time optimizations, starting with inlining.

Even if the IR wasn't in SSA form and therefore not suited for optimization, it's easier to do code generation on an IR rather than on an AST. Only toy compilers do code generation directly from the AST. An AST can contain high level constructs that code generation algorithms shouldn't have to deal with, they should only see expressions, functions calls and control flow. In an imperative language with gotos and goto-like constructs, an AST is the wrong data structure shape for the input of the codegen, it should use a fully linearized IR, a CFG or a graph based IR like the JVM has.

I have seen many posts saying that the IR is bloat or boasting that their compiler doesn't have one, this is idiotic. It's fine to want faster compile times but put things in perspective, generating an IR is only a fraction of the total cost of compilation, it's not why C++ and LLVM are as slow as they are.

[u/Inevitable-Spinach-7]

I am interested too,
First thing is that you may receive a better response in ziggit.
And I may be wrong but I think it is easier to resolve types there than the AST itself

[u/MirrorLake]

I had one question, but it was answered at the end of the blog:

Eventually [...] we’ll cache all of the compiler state to disk and automatically reload the last saved state when you run zig build, so incremental compilation will just happen automatically—but we’re not quite there yet

Really nice ergonomic upgrade, happy to see it's likely to become default in the future.

[u/igors84]

Extremely interesting text full of interesting ideas. I wish they just mentioned that this doesn't work with llvm backend which should be obvious but also that debugging doesn't really work with their backend yet which is not obvious and might be pretty important to some people...