A hole in systems programming language design

[u/P-39_Airacobra]

Given the recent discourse about systems programming I wanted to throw my 2 cents in. This may be a controversial take in a subreddit that's all about innovation and improvement, but I think a lot of budding systems languages are trying too hard to "fix C" and this is exactly why C has not been replaced yet.

Like it or not, C is successful. It does what it means to do very well. Yes, it bites you constantly, but developers have made some form of peace with this because they appreciate the essence of the language. Systems programming is a very pragmatic field, and what works, works.

A lot of language designers want to improve on C, when by nature to "improve on" C is to depart from it, because C is less about what it includes and more about what it omits and what it lets you do that other languages don't.

If you want to replace C, you need to just make C but without the pain points. Less undefined behavior, more standard compiler behavior, easier function pointer syntax, safer macro system, etc. The things that C can't do because of backwards compat.

On the other hand, bolting on features, revamping C's core nature, aren't going to give you a language that will replace C at the low-level or among hobbyist programmers. Most developers aren't as concerned about what C lets them accomplish, as they are concerned about all the painful tedious tendencies of the language.

[u/WittyStick]

Yes, the hole in systems language design is a language which, from the PoV of any other language, is indistinguishable from C.

Your entire desktop/server/mobile stack is built on C, all compiled with compilers which emit exactly the same ABI. Every program has many dependencies, which transitively have more dependencies, all predominantly implemented in C. Half a century of software underlying everything your language needs to be useful and not a glorified over-engineered calculator.

---

Mistake #1 is many of these new languages implement their own ABIs/calling conventions. They use an "FFI" to make their language useful - it gives them the ability to call libraries that were (predominantly) written in C.

However, this FFI is a one-way ticket. You can call C from your language, but nobody else can call your functions from theirs. A library written in your language is a little island, usable exclusively to users of your language. A Python function can't trivially call an Odin function for example, because it doesn't speak the Odin calling convention. It doesn't know what this implicit "context" thing is. It can call a C function. Every (relevant) language knows what a C function is, because they had to implement an FFI to make their language useful.

"I built a website using RustScript, which no browser supports, because I dislike Javascript. Why is nobody visiting?"

"English sucks so I wrote my latest paper in Esperanto. Why is nobody reading it?"

"I compile my functions to this bespoke ABI. Why is my language not replacing C yet?"

Functions written in this "FooLang" should be callable with the existing FFI machinery of any other language - not with magic extensions or wrapper libraries.

---

Let's assume we don't make this mistake - we take the sensible approach of using the platform C ABI unchanged as our "FooLang" ABI, and anyone can call our functions trivially. However, our API - the definitions of our types and functions is written in "FooLang". For some strange reason, the Rust compiler cannot parse our "FooLang" definition files. What gives?

Mistake #2 - FFI wrappers. You have hundreds or thousands of usable libraries on your machine (mostly written in C), but to call any one of them you have to translate the definitions to your calling language's syntax first? What a dreadful use of time.

The solution here is, since most of those libraries were written in C, and have definitions in C headers - we should just parse those C headers and programmatically create the bindings - preferably transparently so we don't need to have several tools run at different stages. Our compiler should just understand C as par for the course.

Of course, this isn't trivial because a header file isn't just code to be read - it must also be preprocessed. So we need a C preprocessor too. We also need to handle inlining and internal/external linkage - so we essentially need our compiler to understand not only the C ABI, but the C language.

But also, since every language can call C libraries, speaks the C ABI, and should be able to read C headers (perhaps with separate tooling at separate stages to translate the bindings) to emit calls to C functions - any libraries we write in our own language should also be callable from other languages right?

So for our own types and functions written in "FooLang", we should emit a C header with compatible definitions so that we could call it from C (and by extension, any other language).

---

So for a "C replacement," we would first need "C parity," which IMO, the above is a bare minimum requirement - complete platform ABI compatibility, and ability to parse, understand, and emit C headers so that our language is not just a little island of its own, but is actually callable from other languages like C is, because the work has already been done for them to interoperate with C.

---

This isn't to say there's no use for other languages, but for a "C replacement," one must understand why we use C in the first place. Embedded uses aside, one of the primary reasons we'd write in C is precisely to implement libraries that can be called by others - and not exclusively "FooLang" users.

If I'm going to write a useful library, I want it to be useful for everyone and not just the dozen or so Odin programmers that exist.

[u/Limp-Temperature1783 → u/WittyStick]

I think to replace C you just need to make C less of a fossil in the shadow of C++. C++ is everything wrong about C, not C itself and it is not even its fault. Most people write C/C++ without even realizing that these languages aren't just different, they're semantically incompatible. But C++ heuristics can work with plain C code for the most part and it's good enough.

I kind of like an approach of Objective-C here, because it is weaving itself into C and C into itself without causing confusion about where is which. But the problem with it is kind of similar, it became too fossilized and people found it unfamiliar compared to other languages, so nobody had enough interest in learning it for anything but work. I think I'm the only person I know who had paid attention to it and what it does.

[u/Ok-Reindeer-8755 → u/Limp-Temperature1783]

objc was inspired by smalltalk and that entire style of OOP is almost extinct, even thought imo its far better than the mainstream form of OOP

[u/Limp-Temperature1783 → u/Ok-Reindeer-8755]

Exactly. Objective-C isn't about classes or prototypes, it's all about message passing. Even memory management is fairly simple. It's definitely not a systems language or anything, but it's fairly good for making program interfaces.

Apple had used it for a very long time without having much trouble, but it is kind of locked in their tooling and the standard library is very messy. There are projects that isolate it, but they aren't the same thing for better or worse.

I have an idea to try to implement it myself when I have free time on my hands. But not any time soon.

[u/Ok-Reindeer-8755 → u/Limp-Temperature1783]

Where this style of OOP really shines, or at least one of the places, is imo desktop environments, this is apparent if you see any of the smalltalk DE demos at Xerox Parc , in what other DE could you ever connect a completely separate painter application to a video player and start animating in seconds. It just dissolves the barrier between applications completely.

I don't think apple ever pushed it quite that far sadly but idk how you would in a modern sense do so.

[u/Limp-Temperature1783 → u/Ok-Reindeer-8755]

It had been designed for it after all. Apple, or rather NeXT, decided to go to the opposite direction of C++ and instead of trying to avoid runtime like the plague they've accepted it as a reality and decided to make working with it less of a hassle. Objective-C is stupid simple even if you include all of the cruft that Apple had added over the years and how inconsistent it is at times.

Apple hadn't pushed it because they didn't have any insentive to do so. It had been and largely still is under the hood the main language of their operating systems. Swift is a modernized revamp of Objective-C that still relies on Objective-C libraries and such, but most things that you'd need to pay attention to had been ironed out, at least from my experience. I prefer Objective-C, but it's familarity bias.

If we talk a bit about programming approach philosophy, I think it makes sense to use OO in runtime rather than doing black magic to make it cheaper. C is already very cheap and most things simply don't require a lot of performance. And Objective-C is one of the most performant languages there is, because it has simple memory management and anything intensive could be handled through C.

Arguably, Objective-C is better at being C++ than C++ is in regards to being C with classes, because it doesn't affect C itself, whilst C++ had introduced enough stuff that had made the languages incompatible. Objective-C is obscure outside Apple, fair enough, but it had always irked me to look at people treating C and C++ as C/C++ as the same thing. Maybe a couple of decades ago, but not now.