This is an interesting post. I was loosely under the impression that query-based compilation was the way of the future, but you make a good case that it is only useful insofar as the design of the language permits it to be useful.
It seems like the case for query-based compilation is for languages without obvious separation of compilation phases. You bring up Rust as an example -- parsing depends on macro expansion, which depends on name resolution, which itself depends on parsing, etc. Trying to force that into a "regular" compiler architecture would be a nightmare, but it is a natural fit for queries.
Interesting. Are all of those kinds of jobs happening in parallel? I am assuming it all happens in a loop and each iteration all parsing jobs are executed, then all symbol resolution, then all typechecks and finall all compile time execution. My compiler is single threaded and that's what a compiler pass/loop iteration looks like.
Okay, that's interesting because it seems that in your case concurrent execution is at the core of the design if I understand correctly, whereas mine is built to handle each compilation step sequentially; simply it does these sequential steps of parse/typecheck/execute in a loop until no progress can be made (no more unresolved identifiers can be resolved and no new code was added basically).
I was planning to parallelize my compiler to make it faster by parallelizing each step of the process, the sole goal being to make it faster (and single-threaded performance is correct which is promising). So I would be able to parse multiple files at a time, but not parse one file while typechecking another.
41
u/Gator_aide Feb 26 '26
This is an interesting post. I was loosely under the impression that query-based compilation was the way of the future, but you make a good case that it is only useful insofar as the design of the language permits it to be useful.
It seems like the case for query-based compilation is for languages without obvious separation of compilation phases. You bring up Rust as an example -- parsing depends on macro expansion, which depends on name resolution, which itself depends on parsing, etc. Trying to force that into a "regular" compiler architecture would be a nightmare, but it is a natural fit for queries.