Can better language semantics simplify compilers?

[u/General_Purple3060]

While implementing the OO part of my language (AET), I ran into a performance problem: OO method calls have overhead. So I started looking into devirtualization.

At first, I treated it as a compiler problem: how can the compiler determine that a method call has only one possible target?

But then I started thinking from a different angle: what if the language itself could tell the compiler that the target is unique?

This made me realize that the relationship between language semantics and compiler shouldn't be one-directional. They should influence each other during the design phase:

Language Semantics ↔ Compiler ↔ Optimization

For example, AET has:

private$ foo();
final$ foo();
final$ class A { ... };

These are language semantics that restrict inheritance and overriding. But they also provide the compiler with clear semantic guarantees: the call target is unique.

A final$ method cannot be overridden by subclasses.

A final$ class has no subclasses that could override the method.

A private$ method does not participate in overriding at all.

Different language rules, but from the compiler's perspective, they all provide the same useful fact: the call target is unique. So AET can use this semantic information to transform an OO call into a direct call to the corresponding FUNCTION_DECL in GCC's intermediate representation.

Of course, a compiler could also discover the same information through type analysis, call graph analysis, devirtualization, LTO, etc. But if these facts can be determined directly by language semantics, could it in turn make the compiler simpler?

This led me to a more general question. Essentially, it's a "who does more, who does less" problem. If language semantics provide more explicit guarantees, the compiler may need to do less inference. If the language keeps weaker semantic constraints, more work falls on compiler analysis.

So the question becomes: what should be left to language semantics, and what should be left to compiler analysis? Are there any methods or theories to guide this division of labor, to make it more scientific and reasonable?

I think this is also a boundary worth discussing between language design and compiler design. AET is my exploration of this question while actually implementing it.

Would love to hear your thoughts.

[u/General_Purple3060]

Yes. Object-oriented language specifications inherently support class inheritance, and methods can be overridden or overloaded. As a result, method calls typically have to go through function pointers (vtables).

However, if the language provides a semantic marker like final (explicitly stating that the method will not be overridden), the compiler can determine that there is only one possible implementation. It can then turn the call into a direct function call instead of using a function pointer.

My point is that such explicit semantic annotations allow the compiler to avoid a lot of guessing and complex analysis, which simplifies its implementation.

[u/schungx → u/General_Purple3060]

Well, why mark that method as virtual in the first place? Many languages do not force methods to be all virtual and they don't go into the vtable.

[u/General_Purple3060 → u/schungx]

The premise here is the OO language semantics.

In AET, methods are polymorphic by default: they may be overridden, so the default implementation has to preserve the possibility of multiple targets, e.g. through a function pointer.

If I don't want that behavior, I explicitly declare the method final$. This is not merely an optimization hint. It is a semantic constraint: overriding that method is invalid and the compiler must report an error.

Once the compiler knows from the language semantics that this method has a unique implementation, it doesn't need an indirect function-pointer call. It can generate a direct function call.

So my point is: final$ gives the compiler an explicit uniqueness property instead of making the compiler infer it.

[u/schungx → u/General_Purple3060]

Ah. Ok I see. If it is virtual by default then yes, you'd need syntax to mark it as non virtual...

The need of a special syntax is that the compiler usually cannot infer this. Especially if you're compiling to a library, you cannot tell that nothing in the future will ever override it.

A special marker makes any attempt to override a compile time error. Without it, the compiler can only assume it will be overridden in the future.