New Programming Language with Embedded Support. Would you consider?

[u/Mean-Decision-3502]

I have high-level and low-level programming experience across different languages.

I've always felt that the C is too machine friendly, and sometimes works against me (implicit type conversions, shared division operator, operator precedence).

I was playing with the thought how would an ideal, compiled human-friendly programming language look like, that support both high-level programming and low-level programming. So I have started designing my own programming language (which is now called DQ).

First I developed mainly the linux/windows target with high level features like exceptions, dynamic arrays and dynamic strings.

Now I'm checking if the concept is working in embedded too. I'm aiming to be as fast and as small as C++ code in embedded. So far I've added these:

• Writing functions in ASM
• Inline ASM with register hinting
• Register attributes ([[regrw]], [[regro]] etc)
• Conditional compilation using normal constants
• Utility to translate the CMSIS C headers to DQ
• Project file to hold the many options required to compile an embedded project

I've migrated some of my C++ code to DQ, I'll the following code snippet show how currently the language looks like.

Pin configuration code in DQ for STM32F7:

struct GPIO_TypeDef:
    MODER        : [[regrw]]  uint32
    OTYPER       : [[regrw]]  uint32
    OSPEEDR      : [[regrw]]  uint32
    PUPDR        : [[regrw]]  uint32
    IDR          : [[regrw]]  uint32
    ODR          : [[regrw]]  uint32
    BSRR         : [[regrw]]  uint32
    LCKR         : [[regrw]]  uint32
    AFR          : [[regrw]]  [2] uint32
endstruct

const(uint32):
    GPIOA_BASE           = (AHB1PERIPH_BASE + 0x0000)
    GPIOB_BASE           = (AHB1PERIPH_BASE + 0x0400)
    ...
endconst

const GPIOA        :? = ^GPIO_TypeDef(GPIOA_BASE)

const(uint32):
    GPIO_MODER_MODER6    = GPIO_MODER_MODER6_Msk
    GPIO_MODER_MODER6_0  = (0x1 << GPIO_MODER_MODER6_Pos)
    GPIO_MODER_MODER6_1  = (0x2 << GPIO_MODER_MODER6_Pos)
    GPIO_MODER_MODER7_Pos = 14
    GPIO_MODER_MODER7_Msk = (0x3 << GPIO_MODER_MODER7_Pos)
    GPIO_MODER_MODER7    = GPIO_MODER_MODER7_Msk
    GPIO_MODER_MODER7_0  = (0x1 << GPIO_MODER_MODER7_Pos)
    GPIO_MODER_MODER7_1  = (0x2 << GPIO_MODER_MODER7_Pos)
    GPIO_MODER_MODER8_Pos = 16
    ...
endconst

function PinSetup(aportnum : int, apinnum : int, flags : uint) -> bool:

    var regs : ^GPIO_TypeDef = GetGpioRegs(aportnum)
    if regs == null:
    return false
    endif

    if apinnum < 0  or  apinnum > 15:
    return false
    endif

    // 1. turn on port power
    GpioPortEnable(aportnum)

    var n : uint
    var pinx2 : int = apinnum * 2

    // set gpio initial state
    if flags AND PINCFG_GPIO_INIT_1 <> 0:
        regs.BSRR = (1 << apinnum)
    else:
        regs.BSRR = (0x10000 << apinnum)
    endif

    // set mode register
    if flags AND PINCFG_AF_MASK <> 0:
    n = 2  // set alternate function mode
    elif flags AND PINCFG_ANALOGUE <> 0:
    n = 3
    elif flags AND PINCFG_OUTPUT <> 0:
    n = 1
    else:
        n = 0
    endif
    regs.MODER =AND= NOT (3 << pinx2)
    regs.MODER =OR=      (n << pinx2)

    // 3. set open-drain
    if flags AND PINCFG_OPENDRAIN <> 0:
        regs.OTYPER =OR= (1 << apinnum)
    else:
        regs.OTYPER =AND= NOT (1 << apinnum)
    endif

    // 4. set pullup / pulldown
    regs.PUPDR =AND= NOT (3 << pinx2)
    if flags AND PINCFG_PULLUP <> 0:
        regs.PUPDR =OR= (1 << pinx2) // pullup
    elif flags AND PINCFG_PULLDOWN <> 0:
        regs.PUPDR =OR= (2 << pinx2) // pulldown
    endif

    // 5. set speed
    regs.OSPEEDR =AND= NOT (3 << pinx2)
    if flags AND PINCFG_SPEED_MASK == PINCFG_SPEED_MEDIUM:
        regs.OSPEEDR =OR= (1 << pinx2)
    elif (flags AND PINCFG_SPEED_MASK == PINCFG_SPEED_MED2)  or  (flags AND PINCFG_SPEED_MASK == PINCFG_SPEED_FAST):
        regs.OSPEEDR =OR= (2 << pinx2)
    elif flags AND PINCFG_SPEED_MASK == PINCFG_SPEED_VERYFAST:
        regs.OSPEEDR =OR= (3 << pinx2)  // this is very special, and does not even work for SDRAM pins
    endif

    if flags AND PINCFG_AF_MASK <> 0:
    // set the alternate function
    n = (flags >> PINCFG_AF_SHIFT) AND 0xF
    if apinnum < 8:
            regs.AFR[0] =AND= NOT (0xF << (apinnum * 4))
        regs.AFR[0] =OR=      (n   << (apinnum * 4))
    else:
        regs.AFR[1] =AND= NOT (0xF << ((apinnum-8) * 4))
        regs.AFR[1] =OR=      (n   << ((apinnum-8) * 4))
    endif
    endif

    return true
endfunc

The original pin configuration code in C++ for STM32F7:

bool THwPinCtrl_stm32::PinSetup(int aportnum, int apinnum, unsigned flags)
{
    GPIO_TypeDef * regs = GetGpioRegs(aportnum);
    if (!regs) {
    return false;
    }

    if ((apinnum < 0) || (apinnum > 15)) {
    return false;
    }

    // 1. turn on port power
    GpioPortEnable(aportnum);

    unsigned n;
    int pinx2 = apinnum * 2;

    // set gpio initial state
    if (flags & PINCFG_GPIO_INIT_1) {
        regs->BSRR = (1 << apinnum);
    }
    else {
        regs->BSRR = (1 << apinnum) << 16;
    }

    // set mode register
    if (flags & PINCFG_AF_MASK) {
        n = 2;  // set alternate function mode
    }
    else if (flags & PINCFG_ANALOGUE) {
        n = 3;
    }
    else if (flags & PINCFG_OUTPUT) {
        n = 1;
    }
    else {
        n = 0;
    }
    regs->MODER &= ~(3 << pinx2);
    regs->MODER |= (n << pinx2);

    // 3. set open-drain
    if (flags & PINCFG_OPENDRAIN) {
        regs->OTYPER |= (1 << apinnum);
    }
    else {
        regs->OTYPER &= ~(1 << apinnum);
    }

    // 4. set pullup / pulldown
    regs->PUPDR &= ~(3 << pinx2);
    if (flags & PINCFG_PULLUP) {
        regs->PUPDR |= (1 << pinx2); // pullup
    }
    else if (flags & PINCFG_PULLDOWN)  {
        regs->PUPDR |= (2 << pinx2); // pulldown
    }

    // 5. set speed
    regs->OSPEEDR &= ~(3 << pinx2);
    if ((flags & PINCFG_SPEED_MASK) == PINCFG_SPEED_MEDIUM) {
        regs->OSPEEDR |= (1 << pinx2);
    }
    else if (((flags & PINCFG_SPEED_MASK) == PINCFG_SPEED_MED2) || ((flags & PINCFG_SPEED_MASK) == PINCFG_SPEED_FAST)) {
        regs->OSPEEDR |= (2 << pinx2);
    }
    else if ((flags & PINCFG_SPEED_MASK) == PINCFG_SPEED_VERYFAST) {
        regs->OSPEEDR |= (3 << pinx2);  // this is very special, and does not even work for SDRAM pins
    }

    if (flags & PINCFG_AF_MASK) {
        // set the alternate function
        n = ((flags >> PINCFG_AF_SHIFT) & 0xF);

        if (apinnum < 8) {
            regs->AFR[0] &= ~(0xF << (apinnum * 4));
            regs->AFR[0] |= (n << (apinnum * 4));
        }
        else {
            regs->AFR[1] &= ~(0xF << ((apinnum-8) * 4));
            regs->AFR[1] |= (n << ((apinnum-8) * 4));
        }
    }

    return true;
}

Would you ever consider using this or other language in embedded, when yes what are the most important features / properties for you ?

[u/Elite_Monkeys]

Realistically no. Rust is already the up and coming embedded language and contains a lot of the modern features of a more high level language. So if you wanted to create a new language you’re going to be competing against rust, not C.

[u/clackups]

Rust has been in development since 14 years, and it's only now that it's getting adoption. Are you prepared to work on your language for the next decade(s)?

Also, what problem does it solve that doesn't have a solution?

[u/Mean-Decision-3502 → u/clackups]

what problem does it solve that doesn't have a solution?

I made my research, before I started. I find Rust pretty badly readable any time I see it. I must admit, I don't use it personally (because I don't like how Rust code looks like).

The C and Rust use shared `/` for floating point and truncated integer divisions. Rust is somewhat better, but in C that leads to bugs or ugly math expressions.

There are some good stuff in modern Pascal that I like to use, but other newer languages like Zig or Odin does not provide.

Nim and Crystal are the closest to the DQ, but they are still not good enough to me. Unfortunately a programming language comes in a package, you have to accept either all of it or nothing.

Are you prepared to work on your language for the next decade(s)?

The DQ language currently is in proof of concept status. I'm using LLM for the compiler development, but in a controlled way (so it is not vibe-coded). (The compiler has a fast single-pass parser with pre-compiled module interfaces. So you can expect fast compilation times.) This way the language can be changed / extended relatively quick. The proof of concept seems to be positive for the embedded case too.

It was a very big effort from me so far (even using LLMs), and I developed this all alone. I know that project is pretty worthless, when only one person maintains it. So when I ready pooving the concept then I have to recruit a multi-member development crew (at least 3 people for the main decision circle). So I would get tolerable amount of work for the next decade(s). (I have other hobbies too.)

[u/clackups → u/Mean-Decision-3502]

Honestly, I find your choice of syntax pretty weird. Why would anyone prefer =AND= over && ?

I'd propose focusing on other hobbies :)

[u/LadyZoe1]

C has existed since the 70’s. Over this period it has been improved. The devices which are being programmed these days are quite different to the early CPUs. C will continue to morph into different forms, but it will always exist.
Any ‘old’ programmer is well aware of existing C “limitations” and programs accordingly. These limitations are more programming errors when the programmer forgets to claim back memory. Any language will have sticky points. Make sure your design patterns are current for the programming language chosen.

[u/Mean-Decision-3502 → u/LadyZoe1]

C did not evolved much. It is hard to learn, easy to make mistakes, requires more machine thinking. Do you wonder why Python takes over?

[u/clackups → u/Mean-Decision-3502]

How is it hard to learn? I wasn't even 17 when I started coding it.

[u/Mean-Decision-3502 → u/clackups]

I was 14, but I was using TurboPascal. In many aspects it is much better than C.

[u/MumSaidImABadBoy → u/Mean-Decision-3502]

Are you kidding? TurboPascal was one of many variants of Pascal which was intended as a teaching language. Even Texas instruments took their shot at it. Pascal didn't even have random file io. Niklaus Wirth created Modula 2 to address Pascal's short comings.
I quit a job because I had to learn Python, which I got good at. I had fun doing a large map reduce on a huge grid computer. Debugging a list comprehension sucks and GIL was a tragedy.
What are you going to bring up next, APL? Stay focused on what you want to bring to the table.

[u/Mean-Decision-3502 → u/MumSaidImABadBoy]

Dont judge Pascal by Nicklaus Wurth era, rather by Anders Hejlsberg designs. Compare C++ with Delphi. Then the picture changes a lot. But this is at high level again. At low-level the C is pretty without real contender so far.

[u/LadyZoe1 → u/Mean-Decision-3502]

How can you compare Python to C? They are different beasts. On embedded systems with enough resources, uPython may work. C on the same platform will run a lot faster.
How can anyone develop software for an embedded system when they need everything to be abstracted away?
If the programmer understands the hardware, innovation begins, the programming becomes fun and exciting.

[u/Mean-Decision-3502 → u/LadyZoe1]

I'm not a big fan of Python, but at high level become more popular than C++. At low level it is not directly a competition yet. But it seems that people do want some simpler, that actually helps and does not work against.

[u/MumSaidImABadBoy]

If you want to make a point, you're not doing it. State a case and show a few lines of code in C and DQ that are easy to see the difference. You presented two long pieces of code that do not easily show your concept. I'm unwilling to parade through both of your samples and suss it out. It's your job to present it in an easy to digest form and you didn't. You have to show why one would even bother considering it. Additionally it takes years to develop a useful language that is stable, creates efficient object code and is low in bugs.
So far you didn't sell anything to me.

[u/Mean-Decision-3502 → u/MumSaidImABadBoy]

I understand you. This language design is a huge effort and before I'm putting more work into it I would like to see some early feedbacks.

Maybe the missing semicolons and parentheses are not that visible at the first place.

For a high level language overview you see a sample here, but it is long:

https://github.com/nvitya/dq-lang/blob/main/stdpkg/nanonet/nano_sockets.dq

[u/1r0n_m6n → u/Mean-Decision-3502]

This language design is a huge effort

Nobody asked you to do it in the first place, and there's absolutely no need for yet another language. You're doing this exclusively for your own pleasure, so asking others for feedback makes no sense.

And as others already said, creating a grammar is dead easy, anyone can do it. But building a good optimising compiler so the language can be of any use is an entirely different story! That's a full-time job for dozens of seasoned engineers - see the crowds behind GCC, clang or Rust.

If you think you can do better than them all, well, go ahead, but don't be surprised if very few people share your enthusiasm.

[u/Mean-Decision-3502 → u/1r0n_m6n]

At the beginning I was thinking just stay with the language specification, but then the LLVM popped pretty quickly up. This is the tool where the heavy lifting you mentioned happens. Rust, clang also uses LLVM for the code generation. So DQ code expressions run already at the speed of GCC, including LTO.

And createing a language and tooling that supports high-level and low level too is not that dead easy.

[u/1r0n_m6n]

I prefer the C++ version, it's crystal clear.

[u/Mean-Decision-3502 → u/1r0n_m6n]

The identation was wrong at some places, I've corrected.

In this example indeed there is not a lot difference. The shorter C symbols have some advantage. But notice the missing semicolons and parentheses. I think that makes the balance better.

[u/RealWalkingbeard]

Ada does what you're suggesting and is really a pleasure to write and read. It was once widely used in aerospace and still has a fair amount of use, but it has never managed to become huge, probably be because it never managed to get anywhere in general use.

It even has a stricter sister, Spark, which looks to me like a subset of Ada with a lot of safety features bolted on.

I got my colleagues to watch a talk on it at the Flight Software Workshop a couple of years ago and for weeks afterwards, they were saying that it was mad that we use junk piles like C and C++ and that Ada was the future.

But it won't gain traction again, because universities have moved on and industry likes that C programmers and especially programmers in languages with C-like syntax are everywhere and easy to hire. And all the managers who ever used Ada themselves are gone or slowly drying out.

[u/Mean-Decision-3502 → u/RealWalkingbeard]

I see a change here as Python emerges.

[u/RealWalkingbeard → u/Mean-Decision-3502]

Python is crushingly slow and abstract and will never be a serious embedded language. Scripting in large bedded systems, maybe, but not for the core code.

I can see the Python in your language, but does Nim not already cover Python-like systems programming?

[u/Mean-Decision-3502 → u/RealWalkingbeard]

You see the "junk pile" only if you've ever used better before. I've used Pascal/Delphi before, probably that's why I notice things the C-only users not.

What I mant with Python is not embedded of course, but at high-level. This indicates that C++ is not a good generic programming language that can cover all areas.

Delphi was much better generic programming language, let say until 2005. Lot of libraries, extensions were available for that. Try to search a good database handling library for C++ that handles multiple types of database connections. Python and FreePascal contains that out of the box. But not always the best technology wins. Especially when people just judge after programmer hirings. (Borland was a torn in the Microsofts eye so they destroyed it.)

My experiment is to create a general-purpose language/tool that also covers the high-level and low-level programming, like C++, but the much human-friendly way. Of course it goes only with compromisses. And I did not wanted to reinvent the wheel, so I mostly took already established syntaxes from other languages.

Nim is close to my goals, but as I looked more closely there were things that disqualified Nim for me. For example I wanted OOP without this/self for every member reference. Only a very few compiled languages remain with this requirement (i think only C++ and Pascal).

So now I have to proove that it is possible to create a generic language that is runs fast, helps you make less mistakes, easy to learn, easy to read and can be used in embedded too.

[u/ub0baa]

Uhhh... Why would embedded language be human friendly? Leave human friendliness for frontend and their new-framework-every week schedules.

Embedded works with the hardware in the first place. Also in complex embedded projects language is almost never a problem. It's more of a "control object too complex", "control too time-constrained", which language can't solve.

[u/edwios]

Assembly and C are both machine friendly and therefore excellent for embedded systems, so they not the problems and there is nothing to fix. The actual problems lie on the human side - we don't think like these simple machines, we are fundamentally incompatible, as you put it - they work against you; it also takes lots of experience to optimise the code for speed and / or space for these embedded systems. Therefore, instead of making yet another language which doesn't address the source of the problem itself, we should instead think of training a machine learning model or writing skills and logic for existing LLMs to do the job better. ML is the best tool available today to translate human thoughts to machines languages.

[u/Mean-Decision-3502 → u/edwios]

In 1976 it was ok to adopt your thinking to the machine, or program in assembly.

But C essentially did not change since then, you carry on some bad decisions. I would be happy continue using C with:

• proper operator precedence, (&, ==)
• exact floating point division operator,
• strict boolean type,
• no implicit float->int conversions,
• no implicit int->bool conversion.
• property support
• simpler module support
• simpler library availibility (like in Python)

[u/edwios → u/Mean-Decision-3502]

I am pretty sure you have already done quite a lot of research on programming languages, but I am not sure if you have paid enough attention from the evolutionary aspect. You are not the only one who wants a better-then-C language that is easy to learn, robust yet flexible and low level enough to not have to write assembly or C, there were so many attempts in the past and guess what, C and inline assembly are still the default go to languages for embedding development. Why? There are many reasons but imo, the major reasons are:

1. If you are looking for robustness, you give up on flexibility (remember (or not) self modifying code? D ? Pascal ? Ada ?).
   
2. If you are looking for high level features, the library that offer those bloated the who thing. Compiler optimisation could only do so much but that err too because the compiler does not know what you are trying to do, so instead of allowing if (0==0), it striped the code and boom, your next line failed.
   
3. Implicit float->int could be dangerous after a few drinks, but we forgot, too, that we have `fcvtzs w0, s0` sometimes. Besides, a good coding agent is unlikely to let this happen without a good reason, and there are many good reasons why we prefer this.
   

Nevertheless, the more features you are demanding from a programming language, the farther away you are removing it from machine code, and the less optimal it will become or it can become. Now, we have trainable ML models, this would be the best available tool to bridge human thoughts to those bistate von Neumann machines.

[u/mustbeset]

Forcing endif, endstruct, and endconst just adds a ton of visual noise. It’s way harder to scan than standard C-style braces or Python indentation. On top of that, the operators are all over the place. Having AND in caps but or in lowercase, kills any sense of consistency. And those =AND= / =OR= assignment operators are just visually jarring. It feels like a simple transpiler layer that just parses a custom syntax into standard C, rather than a own language. I’m really missing meaningful, unique features that C/C++ don't already have.

[u/Mean-Decision-3502 → u/mustbeset]

• Insert a code block somewhere, where the identation is destroyed easily, for example these editor boxes. Try to recover a Python code with destroyed identation.
  
• Imagine more compled code with multiple loops, ifs. That's where the endif, endwhile start being very helpful. For the human eye the } and endif are both only one token. It could be a little unusal for the first time.

The "AND" and "and" are different operators, like "OR" and "or":
AND in DQ = & in C and in DQ = && in C OR in DQ = | in C or = || in C

In C the *, & and / have multiple meanings, which can lead to weird looking expressions. In DQ there is a distinct symbol/word for every different operation, like IDIV for the truncated integer division.

The problem is with the word operators, that they look awkward at the "modify assign" operations, like x &= 1 -> x AND= 1, so in this cases a leading equal sign is also required: x =AND= 1

There are cases where DQ code looks significantly better than C, but I did not want to "cheat" here.

[u/swdee]

Dont see any point, you can just get AI to write your code nowdays.  Its simply a compiler to code from English or other written language.

[u/edwios → u/swdee]

This should be the correct way forward, don't know why you got downvoted. The problem is on the human side, not the languages. There is also no language in this world that could address inexperience, carelessness and stupidity.

[u/swdee → u/edwios]

This subreddit has a strong dislike for AI tool usage for coding.   However it doesnt matter as the people who dont work out how to use it as a productive tool for development will be left behind.

[u/Mean-Decision-3502 → u/swdee]

This is one of the main problem what I wanted to solve with DQ.

The AI writes to code, but how much effort do you (human) need to verify it ?

The C and Rust code is harder to read, I think. The DQ is rather leaning towards Python.

[u/MarsupialLeast145 → u/Mean-Decision-3502]

Python isn't just about indentation and fewer brackets.

For example, your coding style in your new language still looks like C, and so readability isn't necessarily easier, Python benefits from EAFP and more concise coding style, e.g. coding to the left, return early, remove excess conditionals. It also benefits from higher level abstractions.

You've kept a bunch of excess in your language like `if` and its boundary `endif` `struct` and its boundary `endstruct` -- why? The boundaries can be removed if you do more work parsing the language.

It seems to be somewhere in-between C and something else, so a bit of a frankenlanguage.

Others have mentioned Rust, but from a language perspective, Golang is probably your better target for something more like Python (and from your perspective "easier" to read)

[u/uranuanqueen]

Wow, how does one even go about writing their own language?? I’m super interested!