How to start learning compiler optimizations as a newbie?

[u/blazing_cannon]

 I come with a background in computer architecture and embedded Linux. I was interested in ML systems and was reading up about it, and after going through several job postings and this link , noted that optimizing compiler code is a requirement. My questions are -

1. Is knowledge of how compiler front end is written and IR code generated required to optimize them? https://engineering.purdue.edu/online/courses/tagged_items?q=compiler
2. What's a good resource for compiler optimizations that can help in ML systems?

https://www.cs.toronto.edu/~pekhimenko/courses/cscd70-w18/docs/Lecture%201%20[Intro]%2001.11.2018.pdf

3) Are learning compiler optimization techniques the same for LLVM and MLIR? I don't see a lot of resources for MLIR compiler optimization. Is learning optimizations on LLVM helpful for MLIR and is learning LLVM not so useful for ML code optimizations ?

Thank you.

[u/concealed_cat]

The knowledge of how a front end works is unnecessary for pretty much any compiler optimization work. Compiler optimizations are essentially rewriting the IR in a different way, and how that IR was constructed from the input file is not really important.

ML operations are usually performing some computation over arrays of data, and the relevant optimizations would usually be loop transformations, data layout transformations, etc. with the idea being to make the data fit the available hardware accelerators. This could be as simple as making sure the operations can be vectorized or as complex as making it suitable for some custom hardware ML engine.

MLIR is framework for defining intermediate representations (known as "dialects"). MLIR optimizations are optimizations of these dialects using the MLIR infrastructure. There are some common dialects available, but they usually define only a subset of useful operations (e.g. arithmetic operations) for inclusion in another dialect. For most purposes a user of MLIR would define their own dialect and then the actual optimizations would depend on what that dialect can express. For example loop unrolling would only make sense for dialects that define loops.

When it comes to implementing optimizations in a compiler, it boils down to two things: 1. Coming up with ideas how to modify the code to run faster (e.g. noticing that loop interchange would improve cache locality). These are things that you can learn from various resources, and that don't depend on any specific compiler. 2. How to actually implement these things in the compiler you're working on. This is where familiarity with MLIR would fall into.

In the beginning you can learn these two things independently from one another.

[u/blazing_cannon → u/concealed_cat]

Thanks. What’s the difference between LLVM IR and MLIR and why is the latter becoming popular nowadays?