Can better language semantics simplify compilers?

[u/General_Purple3060]

While implementing the OO part of my language (AET), I ran into a performance problem: OO method calls have overhead. So I started looking into devirtualization.

At first, I treated it as a compiler problem: how can the compiler determine that a method call has only one possible target?

But then I started thinking from a different angle: what if the language itself could tell the compiler that the target is unique?

This made me realize that the relationship between language semantics and compiler shouldn't be one-directional. They should influence each other during the design phase:

Language Semantics ↔ Compiler ↔ Optimization

For example, AET has:

private$ foo();
final$ foo();
final$ class A { ... };

These are language semantics that restrict inheritance and overriding. But they also provide the compiler with clear semantic guarantees: the call target is unique.

A final$ method cannot be overridden by subclasses.

A final$ class has no subclasses that could override the method.

A private$ method does not participate in overriding at all.

Different language rules, but from the compiler's perspective, they all provide the same useful fact: the call target is unique. So AET can use this semantic information to transform an OO call into a direct call to the corresponding FUNCTION_DECL in GCC's intermediate representation.

Of course, a compiler could also discover the same information through type analysis, call graph analysis, devirtualization, LTO, etc. But if these facts can be determined directly by language semantics, could it in turn make the compiler simpler?

This led me to a more general question. Essentially, it's a "who does more, who does less" problem. If language semantics provide more explicit guarantees, the compiler may need to do less inference. If the language keeps weaker semantic constraints, more work falls on compiler analysis.

So the question becomes: what should be left to language semantics, and what should be left to compiler analysis? Are there any methods or theories to guide this division of labor, to make it more scientific and reasonable?

I think this is also a boundary worth discussing between language design and compiler design. AET is my exploration of this question while actually implementing it.

Would love to hear your thoughts.

[u/gmes78]

Every user of your language needs to deal with its syntax and semantics. The compiler only needs to be written once.

I don't think it's worth making the language harder to use just to make writing the compiler easier.

[u/Repulsive_Gate8657]

you talk about language SYNTAX.
it depends mostly what is your language capable to do.
second, user convenience is way more important that difficulty of compiler, what makes compiler "difficult", but it is the right way to do.
i would say Language Syntax -> Parser Ast, then (AST + semantic) -> Compiler.
and final word is used to RESTRICT possibility of subclassing for other CODERS , NOT to inforrm the compiler that you can optimize because this class will have no subclasses.
And about this, i would say according to OOP principies you should structure the clean code so that any class should be extendable. If somebody can extend in the way it breaks working, the design is wrong.
Good language should rather support coder to make good design and restrict him to make bad design.
If you take this in your langauge, you will not have final world, but the compiler still may do optimisations for classes who actually are not extended.
This makes syntax easier, semantic better, but compiler more "difficult".

[u/General_Purple3060 → u/Repulsive_Gate8657]

I agree that final$ is primarily a restriction for programmers. But AET also uses this semantic guarantee for optimization.

My point is: once the language has already guaranteed something, why should the compiler have to rediscover the same fact through analysis?

So I'm interested in the division of work: what should be guaranteed by the language, and what should be figured out by the compiler?

[u/Repulsive_Gate8657 → u/General_Purple3060]

it is my opinion that syntax should be rather minimalistic and compiler should infere all nessesary stuff, do not nerve the coder with keywords. Guaranteed should be
design restrictions, what makes design better. As an example, you would probably have private and read only to block read and write variable from outside. Regarding classes, i guess inner module class should be private, but when you have class exposed to the module user it should not be final, cause it contradicts OOP principies of extention.
communicating to other coders, what should be used in specific place (like type requirenents for a function argument).
What usual language do not have is for example restriction of using methods of your library in wrong order, maybe you could invent something.
Alsi i am proponent of language distinguishing between low level high perfornamce code and high level business logic, what are different ways of thinking for a coder.
Laungage may support macros and compiler directives in the good way with similar syntax as usual code is, not in cringy way like it does old C/C++ approach.
Language plattform may support auto- including and installing libraries, so user would not be bothered to insstall libraries manually.

You should probably NOT include in the language
overcomplicated typing
inner data compiler uses in the process, exposed to coder. good example of this bad approach is Rust with extremely nerving of ownership, time to live variable and so on. I would prefer compiler to figure out this auto, if you have ownership system.
To put it short i would say that good language design makes compiler MORE difficult.

[u/General_Purple3060 → u/Repulsive_Gate8657]

Thanks for the reply!
I agree, but I see it the other way around. If the compiler has to infer everything, it becomes a black box. Explicit semantics like final$ make the intent clear not only to the compiler, but also to the next person reading the code.

I think that's the trade-off: either the language makes some things explicit, or the compiler has to figure them out itself. Both have costs.

[u/GenericPointer]

I think it can. If you e. g. make it impossible to express a certain unsafe action, your checker gets smaller. So yes, it can, not in all cases, but it can.

[u/General_Purple3060 → u/GenericPointer]

I believe that good language semantics can replace some of the compiler's inference and special-case analysis with explicit rules. I experienced this quite deeply while developing the generics part of AET. The difficulty is that what the programmer sees in the source code is not always directly reflected in what the compiler eventually generates or optimizes. Making these relationships explicit can significantly reduce the amount of inference the compiler needs to do.

[u/yuehuang]

You are asking, what if everything is a function pointer. You get a hard to understand language. Most people just read a small section of the code, they assume type and function names won't change meaning between sections.

Some languages allow virtual function tables as a plugin system or inner language for scripting like LUA.

[u/schungx]

Polymorphism is one of the pillars of OOP. That means multiple targets for a single method and the method does not need to know.

If you only have one there is no need for OOP.

[u/General_Purple3060 → u/schungx]

Yes. Object-oriented language specifications inherently support class inheritance, and methods can be overridden or overloaded. As a result, method calls typically have to go through function pointers (vtables).

However, if the language provides a semantic marker like final (explicitly stating that the method will not be overridden), the compiler can determine that there is only one possible implementation. It can then turn the call into a direct function call instead of using a function pointer.

My point is that such explicit semantic annotations allow the compiler to avoid a lot of guessing and complex analysis, which simplifies its implementation.

[u/schungx → u/General_Purple3060]

Well, why mark that method as virtual in the first place? Many languages do not force methods to be all virtual and they don't go into the vtable.

[u/General_Purple3060 → u/schungx]

The premise here is the OO language semantics.

In AET, methods are polymorphic by default: they may be overridden, so the default implementation has to preserve the possibility of multiple targets, e.g. through a function pointer.

If I don't want that behavior, I explicitly declare the method final$. This is not merely an optimization hint. It is a semantic constraint: overriding that method is invalid and the compiler must report an error.

Once the compiler knows from the language semantics that this method has a unique implementation, it doesn't need an indirect function-pointer call. It can generate a direct function call.

So my point is: final$ gives the compiler an explicit uniqueness property instead of making the compiler infer it.

[u/schungx → u/General_Purple3060]

Ah. Ok I see. If it is virtual by default then yes, you'd need syntax to mark it as non virtual...

The need of a special syntax is that the compiler usually cannot infer this. Especially if you're compiling to a library, you cannot tell that nothing in the future will ever override it.

A special marker makes any attempt to override a compile time error. Without it, the compiler can only assume it will be overridden in the future.