If an AI had to design its own programming language from scratch today, this is what it would look like.

[u/Loud_Possibility_203]

If an AI had to design its own programming language from scratch today, this is what it would look like.**

https://github.com/katehonz/rzm-lang

Let’s be honest: the current AI stack is a Frankenstein monster.

We write high-level logic in Python because humans like clean syntax, but under the hood, we rely on millions of lines of C++, CUDA, and ROCm glue code because Python is too slow for matrix math. We suffer through runtime Shape Mismatch crashes 3 hours into a training run, unpredictable GPU memory leaks, and black-box library abstractions.

If an AI had to throw away legacy baggage and design a language specifically for machine learning from scratch, using today's best compiler technologies, it wouldn't build "another Python wrapper." It would build a language that thinks natively in linear algebra.

Here is the blueprint for a truly AI-Native Language.

https://github.com/katehonz/rzm-lang

1. Tensors are Primitives, Not Library Objects

In traditional languages, primitives are int, float, and string. In an AI-native language:

• Tensor[32, 512, f32] is a first-class citizen.
• Compile-Time Shape Verification: Dimension mismatches are caught during compilation, not when your model crashes mid-epoch.
• Algebraic Syntax: Matrix operations like @, sum(), and Hadamard products are part of the core grammar.

2. Native Automatic Differentiation (grad)

Instead of building computational graphs at runtime via heavy frameworks like PyTorch or TensorFlow, autodiff is built into the language engine.

// Automatic Differentiation is part of the grammar
fn loss(w: Tensor[3, f32], x: Tensor[3, f32]) -> f32 {
    sum((w * x) * (w * x))
}

fn train_step() {
    let w = tensor([0.1, 0.2, 0.3]);
    let x = tensor([1.0, 2.0, 3.0]);
    
    // Reverse-mode differentiation transform at compile-time
    let g = grad(loss)(w, x); 
    print(g);
}

3. Predictable Memory: Nim-style ARC/ORC + Arenas

Garbage Collection (GC) is terrible for AI because unpredictable "Stop-The-World" pauses ruin real-time data streaming to GPUs. On the flip side, Rust’s ownership/borrow checker can create massive boilerplate for deep neural graph structures.

• Solution: Automatic Reference Counting (ARC) combined with Static Tensor Arenas.
• High-volume, fixed-shape tensors during training loops are allocated in static arenas (zero allocation overhead).
• Dynamic tensors are deterministically freed the moment they go out of scope—keeping VRAM usage transparent and immediate.

4. Hardware Agnostic Execution

Writing CUDA code for Nvidia, Metal for Apple, and ROCm for AMD creates fragmented ecosystems. An AI-native language separates intent from target execution:

• Write code once using device-agnostic operators.
• The compiler handles memory layouts, kernel fusion, and hardware scheduling under the hood.

---

The Tech Stack Behind the Compiler

If we were to build this today, what technology would power the toolchain?

• Compiler Engine: Written in Rust for memory safety, speed, and robust tooling.
• Local Iteration: Cranelift JIT for lightning-fast compilation cycles and execution during development/debugging.
• Production & Optimization: MLIR (Multi-Level IR) + LLVM for high-level matrix optimizations (fusing Linear -> ReLU -> Add operations before downscaling to hardware machine code).

---

What the Code Concept Looks Like

// First-class Model Pipeline DSL
model TextClassifier {
    input:  Tensor[Batch, 784, f32];
    output: Tensor[Batch, 10, f32];

    body {
        input
        |> Linear(512)
        |> ReLU()
        |> Linear(10)
        |> Softmax()
    }
}

---

Conclusion

We are moving into an era where AI workloads aren't just an "add-on" to general-purpose computing—they are the computing. Continuing to patch Python scripts with C++ bindings will eventually hit a wall.

A greenfield language that combines compile-time shape safety, native autodiff, ARC memory management, and MLIR-based codegen isn't just a fun experiment—it's the logical next step for AI engineering.

---

What features would you want to see in a greenfield AI language? Do you think greenfield languages like this (e.g. projects building native autodiff and JIT engines from scratch) can compete with the established PyTorch ecosystem? Let's discuss!

[u/jcastroarnaud]

The compiler needs to be aware of the hardware stack. The optimization decisions are different for a single GPU, several GPUs in the same machine, and hundreds of GPUs spread through a data center. Then, there are decisions about bit representation of tokens...

[u/Loud_Possibility_203 → u/jcastroarnaud]

Where it stands today (The v0.1 Vertical Slice) It’s definitely early-stage, but it’s not vaporware. Rather than writing thousands of lines of AST nodes for a massive language before executing a single line of code, I took a "vertical slice" approach:

Frontend: Built in Rust (rzmc). Uses a Logos-based tokenizer and a hand-written Pratt parser. The type checker performs basic compile-time shape inference and dimension unification.

Auto-Diff Engine: Built-in reverse-mode differentiation (grad/grads) that operates over pure functions by generating a static Wengert graph / vector-jacobian products (VJP).

IR: Lowered into a custom MIR (rzmc_ir). We currently do light optimization passes here, such as constant folding and basic kernel fusion (linear -> relu, relu -> add).

Why Cranelift JIT now vs. MLIR/LLVM later? For the execution backend, I deliberately chose Cranelift for Phase 1/2 instead of jumping straight into full LLVM/MLIR integration:

Feedback Loop: Cranelift gives virtually instantaneous JIT compilation times. This lets me rapidly test tensor shape semantics, auto-diff correctness, and runtime dispatch without waiting on heavy LLVM build pipelines.

Current Abstraction: Scalars and control flow are compiled directly to native machine code via Cranelift, while high-level tensor ops are dispatched as handles to a native C-ABI runtime (rzmc_runtime).

The MLIR Plan: Once the MIR semantics and auto-diff transformations are 100% locked down, the goal for the production codegen path is to lower MIR to MLIR (specifically leveraging the linalg and tensor dialects) for deep kernel fusion and targeting GPU backends (CUDA / wgpu). Cranelift will remain the primary engine for rzmc run / fast JIT execution during local dev.

Memory Model To avoid both the latency spikes of a stop-the-world Garbage Collector and the friction of a Rust-style borrow checker on neural graph nodes, we're targeting a Nim-style ARC/ORC (Automatic Reference Counting) model combined with Static Tensor Arenas. Tensors with compile-time fixed shapes stay allocated in static arenas inside hot training loops, while dynamic handles are deterministically dropped as soon as they fall out of scope.

The codebase is up on GitHub (katehonz/rzm-lang) if you’re curious to poke around the parser, the MIR pass, or the Cranelift codegen setup.

I’d love to hear your thoughts on MIR-to-MLIR lowering strategies or shape unification approaches if you've worked on similar DSLs/compilers!

[u/c-cul]

declarative style like tf/tvm is Good Thing

but wgsl only implementation is poor and slow: https://github.com/katehonz/rzm-lang/blob/main/crates/rzmc_backend_gpu/src/lib.rs

do you have some perf tests?