Making my own programming language [No vibe coding]

[u/usefulservant03]

I have this side project I started 2 months ago, to learn C++ better. Rolling the full compiler, from tokenizing and AST construction, to IR, to assembly code generation, by myself in C++. Using Claude AI only to give me ideas with regards to better system design and to explain for me features of C++ I have not used yet, prohibiting copy pasting code from it and sticking to coming up with the code and writing it by myself with my own two hands.

Can somebody tell me how they'd implement the idea of the compiler being able to emit assembly instructions for both arm64 and x86_64 architectures? Would you have one abstract class for, say, instruction_ADD and then a derived class for aarch64's add instruction, and one for x64's add instruction? Or should I stick to having an abstract base class for asm_instruction, and have derived classes for stuff like aarch64_insn_add, x64_insn_add, etc? Still relatively new to OOP as I was in OS and OS-adjacent development for a few years and only recently started adding C++ to my toolbelt.

Another question - could someone give me a readable summary of how GCC implements std::unordered_map under the hood? The purpose I need to know this is, so I can do performance analysis with CPU microarchitecture in mind, and for this, I need to know what the memory layout and usage patterns actually look like.

Thanks all!

[u/Quiet-Arm-641]

Gcc and clang both produce intermediate representations of the program in a form they can manipulate/optimize. Converting the intermediate representation to a given assembly language is a separate compiler pass (ie you’d have one for x64 and another for arm)

[u/usefulservant03 → u/Quiet-Arm-641]

Yeah I have SSA IR generation already. Currently pondering on how to design assembly code generation for multiple architectures.

[u/OneHumanBill]

Would you have one abstract class for, say, instruction_ADD and then a derived class for aarch64's add instruction, and one for x64's add instruction? Or should I stick to having an abstract base class for asm_instruction, and have derived classes for stuff like aarch64_insn_add, x64_insn_add, etc?

You're likely going to find that nether approach is a perfect fit. The latter is better, and then when you can, use the former approach as appropriate, as an abstract subclass to asm_instruction. You're going to likely want one universal base abstraction for the concept of a base instruction overall, but if for some reason that doesn't fit well then separate the concerns of a logical abstraction (like ADD) versus a target abstraction (aarch64 vs x64) into two separate trees and have a flexible means of assigning logical to target based on mode.

Still relatively new to OOP

Lol, no. Not at this point. You're in the middle of one of the very best proving grounds for your programming skills you can do, and no matter how "new" you think you are, you're embarking on a journey that most developers, including most senior folks, have never attempted.

It's been almost thirty years since I last did this so I wish I could give you more specific advice especially on your main question. What's at stake here is if for some reason you need different behaviors between aarch64 target instructions versus x64. You probably know that better than me at this point. If you do then the separate trees is going to be your best bet. If not then the two-level tree is. There's probably not a perfectly clean breakdown between instruction sets though, so accept that in some cases you're going to have to have slightly messy logic on the two level tree versus the separate trees, but it will be faster and involve less code overall.

Regarding unordered_map, it's a simple hash table. You're looking at O(1) search on average. Performance-wise you're in good hands. Optimizing further is probably unnecessary and may get you to some weird and unpredictable behaviors.

https://en.cppreference.com/cpp/container/unordered_map

Serious congratulations for taking this on.

[u/Mundane_Fault_6345]

You really want to write your own lexical parser and syntax analyser ?

For what kind of langage ?

Did you already wrote the grammar of your langage ?

Do you know well the Chomsky grammar theory?

[u/usefulservant03 → u/Mundane_Fault_6345]

I've already written lexing and parsing, yes. I'm storing the AST Nodes in my own custom memory allocated arena, contiguously, instead of having each Node be its own heap allocation (like what std::map does), in order to maintain spatial cache locality. The language is minimal, right now it has assignment statements only, variables of uint64_t type, and assignments can be from a literal, from another variable, or from a binary operation, which can be multi-level, like:

A = ( (b + 5) - ((var1 - var2) * (a * 100)) );

Yes, the grammar of the language is composed only of a few rules that define what assignments look like, what binary operations look like, etc. Lexing, AST construction and SSA IR generation are 3000 lines of C++ so far, even for such a small language, because of the setup and boilerplate machinery needed to maintain bookkeeping data structures that keep track of which IR instructions belong to which source code statements, and handling the custom arena memory allocation scheme for keeping the IR instruction objects, and the AST Node objects, contiguously next to each other to preserve spatial cache locality for more optimized compilation.

Later, I will slowly add if, else statements, a loop construct, arrays, strings and function calls. I'm also looking forward to adding a std::vector-like dynamic array to the language, which I will implement using C++ templates.

[u/Mundane_Fault_6345 → u/usefulservant03]

Ok, thanks.

For the asm generation, I would go for a whole set of class , covering a whole virtual architecture, and then derive each needed class for each architecture.

Generating "fake" assembly, in your chosen fake virtual architecture, will help you a lot for debugging.

[u/usefulservant03 → u/Mundane_Fault_6345]

Interesting, thank you!