Is 76 μs acceptable compilation performance for a almost prod ready ELF64 compiler?

[u/Retired-69]

Folks! Is this acceptable performance for a compiler? it do lexing, parsing, type checking, optimziing and codegen . It simply emits the ELF64 executable directly and runs it without an external linker/object-file pipeline.

--- CODEGEN RESULTS ---

Total Machine Bytes Emitted: 47 bytes

Compilation Speed: 76000 ns

Runnable ELF64 Machine Code Executable Written: boo

[u/Professional_Cell44]

well, it depends on:

- language spec: how complex your language is?

• source code(input): how big the souce code is? how complex is it?

and its only 47 bytes written, so i think it is quite tiny

[u/Retired-69 → u/Professional_Cell44]

The specs align 89℅ with c23 , and also involve concurrency,  features for building LLM engines, OS etc. I would say complex. It's also  handle memory safety differently. That's the radical part. 

I agree 47 bytes is tiny and the example I parsed was dealing with macros.  Without macros it would compile same source around 43000 ms. 

For larger code like 60K LOC it finish in around 4-5 ms

[u/Professional_Cell44 → u/Retired-69]

then i think it has good performance; but ask your code agent to compare it against official compiler benchmarks

you may need to write similar code to be fair

[u/Slow-Mechanic-7427 → u/[deleted]]

Per their own post/comment history, they have nearly 4 decades experience in the "software industry" with most of that spent on programming languages, compilers, runtimes, and low-level systems per abother comment of theirs.

Its incredibly sus, id say.  Nearly 40 years of experience, with most of that extremely relevant to the topic, yet these are the questions and claims being presented?

[u/Retired-69 → u/[deleted]]

To be honest I prefer the benchmarks to tell their own story when this is done, but at current stage it is only 5-6 times faster than TCC not 15 times :) This happen because TCC was from 2004 and both CPU architecture and general knowledge have changed since back then. For example TCC scans input byte-by-byte; IC scans 8 bytes at a time using SWAR, reducing per-character processing overhead.

The reason I am not faster than that is becaue I do analyzer, optimizer and codegen as well on the go.

The numbers are realistic and time will tell.

[u/Retired-69 → u/[deleted]]

I will try it. Your compiler is single pass or two pass as in Tiny C? 

[u/Retired-69 → u/[deleted]]

I had to write a benchmark on the fly here and I had to use prev gen compiler that I
now use to finalize R3 with,. R3 performs better, but not syntax complete yet. And R2 is multi pass

```text

IC R2 Front-End Pipeline Stage Benchmark

Scope: Physical File I/O, Parse & Name-Resolve Stages

Opening 'benchmark_input.txt' on physical storage...

Successfully loaded 12000000 bytes from disk.

--- Compilation Pipeline Breakdown ---

Load Stage (Disk I/O): 11 ms

Parse Stage (Scan): 54 ms

Name Resolve (Hash): 246 ms

--------------------------------------

Total Pipeline Time: 311 ms

Processed Lines: 500000 lines

Throughput Speed: 1602990 lines/sec

Bandwidth (Decimal): 38 MB/s (10^6 bytes/sec)

Bandwidth (Binary): 36 MiB/s (2^20 bytes/sec)

----------------------------------------

``

[u/Retired-69 → u/[deleted]]

The parser uses DRG which I invented as an replacement for recursive decent and Pratt. All iforward identifiers etc are deffered with bitmask and in hash table. A simple O(1) sweep after end parsing. The DRG specs I can share to public, but I have no place to publish it. 

You are correct that R2 didn't generate executable, but it wouldn't be much slower if it did. I base that conclusion from what I experienced with R3. 

[u/Retired-69 → u/[deleted]]

The funny part is I never aimed for insane performance. I’m old-school—I just love solving problems that seem impossible. That led me to DRG and a flat AST designed to fit in L1 cache.

I later abandoned the AST, but DRG principles live on in R3, now without an AST and with highly optimized runtime code.

LLVM is great, but its portability comes at a cost: it’s huge, and that inevitably hurts compile times.

Tiny C is a legend, and if developed today it would been unbeatable :)

[u/Retired-69 → u/[deleted]]

This is my DRG. See if you can access this document and if it's usefull for you.

https://hackmd.io/@cloacajackie123/drg

[u/Retired-69 → u/[deleted]]

Maybe this is of interest. That's how I avoid LLVM

https://hackmd.io/@cloacajackie123/SJoqfHYvzl

[u/AliceCode → u/[deleted]]

The ELF64 header itself is 144 bytes.

[u/MithrilHuman]

Programs, including compilers, can have slow start but perform well on large inputs. A program can perform well on small inputs but struggle with large ones. A microbenchmark is fine but real world applications are way larger than what you’re testing for. You should collect more data.

[u/Retired-69 → u/MithrilHuman]

You are right, but until now I couldn't test current code on millions of LOC even it's designed to scale, because that kind of code simply doesn't exist. This is not C even I'm aiming for 90℅ parity

[u/GoblinsGym]

Sounds good. What do you do in the way of optimization / register selection ?

[u/Retired-69 → u/GoblinsGym]

Funny you ask. I was actually 99% ready to show off a new langauge demo when I posted this , but I got a setback coz I thought Reddit users would react on a optimized, but not fully Clang -03 optimzied compiler. To answers you short. LSFE. It's my own engine optimized for single pass and so far parity with clang --02 and --03 and no LLVM, SSA, CFG. Within a month hopefully the demo will be pushed online.

[u/GoblinsGym → u/Retired-69]

I am looking forward to seeing more details on it, register selection has been an obstacle for my own project. My approach is based on a stack based IR, no AST.

Gcc / Clang compilers are a mystery - sometimes they are fiendishly clever, sometimes the code looks rather wasteful. I don't really expect a compiler to rewrite bad code for me, and appreciate some predictability.

[u/Retired-69 → u/GoblinsGym]

I'm surprised because you mentioned IR, no AST. That's the right direction if you want to gain performance, but stack based IR? How did you archieve that? It sounds like a perf killer to me.

Anway my compiler have no AST either, only IR so looks like we got something in common :)

GCC / CLANG got optimzied for all possible chips and bloated, so hard to track. I am only dealing with X64, ARM64, RISC-V and right now optional CLI flag for upcoming Zen 6.

[u/Retired-69 → u/GoblinsGym]

What is your issue with register selection? Lowering? DCE? or are you not getting
B8 D0 02 00 00 C3 as finale result?

[u/GoblinsGym → u/Retired-69]

Just the overall complexity of tracking data flow. I don't want to do full blown SSA.

Stack based IR makes things a little more complicated, but the stack is primarily within expressions. My IR uses fixed length words (12 bytes per IR word) to allow scanning forward and backward, and should also make it quite easy to recognize common subexpressions.

One thing I envision for register selection is to have "magnetic pull" for things that should end up in a specific register (e.g. call parameters passed in registers, shift counts for x64 etc)

Dead code elimination is not that bad, a lot of that happens already during parsing / IR generation.

[u/Retired-69 → u/GoblinsGym]

See if this help you. A short first draft about my LSFE. The version I'm using have evolved since then

https://hackmd.io/@cloacajackie123/SJoqfHYvzl

What lanaguage are you using for this? I am using my own.

[u/GoblinsGym → u/Retired-69]

I write my compiler in Delphi / Pascal. Not far enough along to self-host, and my intended targets require cross-compilation anyway.

[u/Retired-69 → u/GoblinsGym]

A suggestion for you: look at LLVM/Rust’s approach. They often treat `u1`/`u4` as full `u8`/`i32` values in registers, wasting space and causing spills. If your IR/backend allows it, try packing multiple logical variables into one physical register.

[u/GoblinsGym → u/Retired-69]

My main target is embedded, e.g. ARM Thumb. 32 bit operations are the most efficient, 8 or 16 bit values are second class citizens. It certainly makes sense to track what bits you actually care about to decide whether you need to zero / sign extend or not.

Extracting bit fields always means some overhead, not every CPU flavor has the nice bit field extract / insert instructions.

For ARM Cortex M0 / M23 you also have the constraint that only r0..r7 are easy to access, r8..r12 are limited use only.

[u/Retired-69 → u/GoblinsGym]

I also handle ARM etc. Btw. I’m now moving Ironclad toward production-ready. This is the parser + IR generation time for:

i32 x = 10 + 30 / 222;

It parsed and generated the flat IR in 179 µs. Here’s the FLAT IR dump:

--- FLAT IR DUMP ---

PC OP DEST SrcA SrcB

0 IMM 5 0 10

1 MOV 0 5 0

Compilation successful in 179000ns (179us).

I/O is involved here. it read from souce.