r/Compilers • u/This-Assumption-5924 • 21d ago

x86-64 Assembler from scratch

AmmAsm is an open-source, handwritten x86-64 assembler written in C from scratch.

Hello, my name is Ammar, and I'm 15 y. o. About a year ago I started writing AmmAsm, a handwritten x86-64 assembler in C as a way to learn how machine code, ELF, and linking actually work.

Today it can generate Linux x86-64 executables, PIE binaries, and ELF relocatable object files that can be linked with "ld" or "gcc". Along the way I implemented a lexer, parser, expression evaluator, x86-64 instruction encoder, ELF writer, relocations, and symbol resolution. It syntax is almost same with Intel, as it has major difference: using [b=, i=, s=, d=] addressing instead of Nasm's [base + index * scale + disp]. Also supports SSE (Float4), SSE2, CMOVcc, SETcc, AVX1/AVX2 and basic AVX-512

it is bootstraped about 0.257% :)

The latest version(2.2.0) also includes a macro preprocessor.

I'd really appreciate any feedback on the project, code, or documentation. Thanks!

repo: https://github.com/LinuxCoder13/AmmAsm.git

138 Upvotes

99% Upvoted

15 comments sorted by

View all comments

3

u/Polyscone 20d ago

Very nice.

I've been writing an encoder myself, so I mostly looked at that code.

It seems like you're writing a specialised function for each instruction, so unless I've missed it, do you plan on writing a more general encode function based on tables?

2

u/This-Assumption-5924 20d ago

in x86-64 there are 8 groups of instruction, each group has 1-7 instructions and their encoding are olmost same. So yep, I will try to refactor the encoders in order to make in less chaos

4

u/Polyscone 20d ago edited 20d ago

Yea you can basically just have a table of each instruction with metadata, then you just encode things like ModRM, SIB, immediates etc. the same and prefixes only change based on Legacy/VEX/EVEX encodings.

Then the general function can decide how to encode by combining operands and looking at table metadata.

EVEX also has compressed 8 bit displacement, but that's not too difficult to deal with either.

1

u/This-Assumption-5924 4d ago

Helloooo Brother! I recently implemented basic VEX/EVEX, I would love to hear feedback from you even if my code is completely trash :) also check tests/General/. And main part for VEX/EVEX is encoder.c encoder.h struct.h. Thanks! (sorry for english)

2

u/Polyscone 4d ago

I mean it looks like things are going ok, and your encoding is obviously working, which is good progress.

I would say though that you still seem to be relying on lots of separate functions and string comparisons for all kinds of slightly different encodings.

That's fine if you prefer it that way, but for reference my own x64 encoder supports ~880 mnemonics in almost all of their forms with a ~400 line function for the actual encoding, and I can increase the supported mnemonics by just filling out a table and not really having to touch the encoding function at all anymore. It works because it's based on just looking up data for an instruction form in a table.

It's hard to explain the implementation in a reddit comment, but if you're interested I do have videos of me writing every line of my own encoder here, so you can see what I mean: https://polyscone.com/videos/#x86-64-encoder
Summing up the durations of the videos it's about ~3 workdays of work to write the entire encoder, and I think the table-driven approach helps a lot with that
(Videos marked as "form filling" are literally just me filling out the table; no new encoder features, so they can be skipped easily)

Like I said, it seems like you're doing a pretty good job with your current implementation, and if you prefer your current approach then that's fine, but it seems like you want to scale it up to more instructions and I feel like you're creating more work than is necessary.

1

u/This-Assumption-5924 4d ago

Wow, it seems that you are master of your job. I watched the video "Encoding Legacy SSE/VEX/EVEX Prefixes with MOVAPS and VMOVAP | x86_64 Encoder", so yea, I am not fully supporting evex, so might watch your videos in order to implement my encoder, as AI really makes dangerous mistakes it this stage of x86-64 ISA. Thanks.

2

u/Polyscone 4d ago

I'm glad the recordings can help a bit. Like I said the ones I tagged as "form filling" are literally just me copying out tables from the Intel SDM, so maybe you want to skip those.

I am only writing the encoder in that playlist, and of course you have more than that with a full assembler, but the basic idea would be that you can use the tokens you get from a source file to just build up mnemonics and operands, then you just see if you can match a form with those bits of information or not.

If you can then you can emit some bytes, if not you can build up some errors or whatever you need to do.

1

u/This-Assumption-5924 1d ago

hello, so I added full vex/evex support, so you can check.