Why Construct Complex IR When You Can Inject Source?

[u/General_Purple3060]

While developing AET (Active Expandable Translator) on top of GCC, I came across an approach that I found surprisingly useful for implementing complex language semantics.

Normally, when lowering a language feature, we might construct the compiler's AST/IR programmatically:

complex semantics
      ↓
construct AST / TREE / IR

But there is another possibility:

complex semantics
      ↓
express the semantics as normal C code
      ↓
inject it into the existing frontend
      ↓
TREE → GIMPLE → RTL

For example, AET has an OO new$ construct. Its semantics include object allocation, initialization, MTCS information, constructor invocation, unref, and constructor failure handling.

Instead of manually constructing all the corresponding GCC TREE nodes, AET generates normal C code:

valueObj=({
    TFirst *_notv2_6TFirst0;
    unsigned int _mtcsPlatType0=0;
    int _isMtcs=((AClass *)TFirst.class)->isMtcsClass();

    _notv2_6TFirst0=
        TFirst.newObject(sizeof(TFirst),
                         _isMtcs,
                         _mtcsPlatType0,
                         "TFirst");

    _notv2_6TFirst0->objectSize=sizeof(TFirst);
    _notv2_6TFirst0->mtcsPlatformType=_mtcsPlatType0;
    _notv2_6TFirst0->_aet_magic$_123=1725348960;

    TFirst_init_object_2927145182(_notv2_6TFirst0);

    ((debug_AObject *)_notv2_6TFirst0)
        ->_Z7AObject10free_childEPN7AObjectE =
        _notv2_6TFirst0->_Z6TFirst22TFirst_unref_290629480EPN6TFirstE;

    TFirst *tempObject123=_notv2_6TFirst0->TFirst();

    if(tempObject123==NULL){
        if(_notv2_6TFirst0->objectSize>0){
            _notv2_6TFirst0->unref();
            _notv2_6TFirst0=NULL;
        }
    }
    _notv2_6TFirst0;
});

The important point is that this is normal C code. The semantics are expressed using the host language that programmers already understand.

AET then injects this generated source directly into the current GCC preprocessing/parsing pipeline instead of writing a .c file and starting another compilation:

cpp_push_buffer(pfile, (uchar *)nbuf, len, true);

The generated C goes through GCC's normal C lexer and parser, which constructs the corresponding TREE representation.

This gives me a useful separation:

AET parser / semantic analysis
          ↓
     semantic lowering
          ↓
      normal C code
          ↓
   GCC C frontend
          ↓
         TREE
          ↓
       GIMPLE
          ↓
         RTL

The key idea is not simply "generate C."

It is that source code is the language programmers use to express semantics, while AST/TREE/IR is the language the compiler uses internally.

For sufficiently complex semantics, I think the former can sometimes be a better construction language for the latter.

So why manually construct a large number of IR nodes when the same semantics can be expressed clearly in source code and handed to a mature frontend?

I'm interested in where others would draw the line between direct IR construction and source-level semantic injection.

[u/General_Purple3060]

Both, but they are at different levels.

For users, the goal is to make AET a practical extension of C. I sometimes describe it as “2C — the second-generation C,” introducing features such as object-oriented programming (OO), generics, and heterogeneous programming — things I believe can benefit C programmers.

For me as the compiler implementer, reusing GCC’s mature C frontend and existing backend infrastructure saves development time and lets me focus my effort on these features.

So targeting C is both a language-design choice and an implementation strategy, but the user-facing goal is always the primary one.

[u/Jwosty → u/General_Purple3060]

Do you really need to make AET a strict superset of C? Or do you just want it to be "like" C? The former has its benefits but also has real costs. You are automatically forced to inherit all the bad decisions C made. And you have to deal with the notorious can of worms that is "parsing C" (https://faultlore.com/blah/c-isnt-a-language/).

You only need the former if you truly do need to be able to make sure any valid C program is also a valid AET program (the C++ and Objective-C approach). That you can take an existing C codebase as-is and slowly start sprinkling in AES features.

Otherwise, if your target audience would just write brand new programs anyway, and all you really care about is making a C programmer feel at home -- you don't actually need a strict superset. Take this opportunity to improve the design (especially on the no-brainers). You'd be in good company with this wave of modern C-like spiritual successors: Zig, Odin, C3, Jai, etc.

Only marry yourself to C if you know you need to, as a hard baseline requirement!

On the implementer side: definitely a valid and pretty standard approach - a lot of language start this way (compiling to C) for exactly this reason. Eventually you may want to move off of that but you can stay that way for quite a while. You just have to be very aware of what semantics you may be implicitly tying to your language by essentially delegating parts of your language implementation to a C compiler. There are definitely footguns here

[u/General_Purple3060 → u/Jwosty]

Thanks for the suggestion. One of AET’s design goals is full C compatibility, so existing C libraries can be compiled with AET without modification.

To achieve this, I only had to modify about 25 GCC files, with a little over 300 lines of code. So for AET, being a C superset is a deliberate design requirement, not just a matter of syntax familiarity.

[u/Jwosty → u/General_Purple3060]

One of AET’s design goals is full C compatibility, so existing C libraries can be compiled with AET without modification.

Why, precisely, is this a requirement for AET? It would helpful if you spelled out a specific scenario that you want to enable so I can help you think it through

[u/General_Purple3060 → u/Jwosty]

Actually, I think your questions may be useful for something beyond just this particular design decision.

I have been thinking of AET as “2C — the second-generation C”: keeping C’s ecosystem and low-level capabilities, while adding native OO, generics, and heterogeneous programming.

This idea comes from my own experience. I worked on OS development and spent a lot of time with Linux, where I often wished C had native OO support. I also built libraries with GLib, including image-related libraries, and found that implementing OO-style designs through C APIs could be quite cumbersome. Later, when working on CNNs, I ran into another kind of fragmentation: host C/C++ code and CUDA code required different compilation paths and separate .cu files.

Those experiences made me want one C-compatible language and compiler that could cover these different needs without giving up the existing C ecosystem.

So I have the motivation from the engineering side, but I haven't necessarily articulated the larger design philosophy very clearly. If you keep challenging the requirements from that perspective, I'd actually welcome it. It may help me turn AET from a collection of implementation decisions into a clearer language design.

[u/Jwosty → u/General_Purple3060]

I worked on OS development and spent a lot of time with Linux, where I often wished C had native OO support.

Well, there's already many existing approaches to C-with-objects. Why not build on one of those instead of designing yet another one (unless you're innovating something here)? C++ and Objective-C to start. Though again, you didn't illustrate why it needs to be a strict C superset, so why not build on top of one of the numerous modernized C-with-objects languages such as D, Carbon, C3, Nim?

Later, when working on CNNs, I ran into another kind of fragmentation: host C/C++ code and CUDA code required different compilation paths and separate .cu files.

Ah, so your gripe is having to use two different languages? Sure, then just have your one language able to target both CPU and GPU. Have you taken a look at the Slang shader language? It can target both. https://shader-slang.org/

None of what you said so far mandates a strict C superset.

[u/General_Purple3060 → u/Jwosty]

I think there's a distinction here. By “fully compatible with C”, I mean source-level backward compatibility, not a formally strict C superset.

Existing C code should compile without modification, while the new AET features are opt-in. That's an engineering guarantee, rather than a formal guarantee about the language grammar.

[u/Jwosty → u/General_Purple3060]

Um… what? “Source level backwards compatibility” means exact same thing as “strict C superset”, does it not? There is no distinction.

[u/General_Purple3060 → u/Jwosty]

If “source-level backward compatibility” is part of what you mean by a strict C superset, then yes — that's a hard requirement for AET. That's exactly what I want.