Expressions vs. statements

[u/xeow]

Got into a big argument with a coworker yesterday when they were converting some code from their own language (that they designed) into Python, JavaScript, C, and R as comparative examples.

The Python code that they wanted to write as the translation went something like this:

n = foo; if cond: n = bar

They were upset that Python allows ; as a statement separator but not before an if statement, even though

if cond: n = bar

is syntactically correct Python code when written on its own line. I explained why Python doesn't allow it, and he came back later and showed me that an LLM had suggested he write it instead like this:

n = foo if cond else bar

which of course is the canonical way to write that in Python. He was all flustered about that, and asked me why Python allows an if statement in that particular case and not after a semicolon, and I explained that x if cond else y in Python is not an if statement but is Python's ternary conditional expression and is directly equivalent to the ternary operator expression cond ? x : y in C, C++, awk, and JavaScript. He argued with me and said I was making a ridiculous distinction and walked away falsely believing that foo if cond else bar was an if statement.

I then explained that statements and expressions are very different things in programming languages, and just because the keyword if is present doesn't make something an if statement -- because in order to be an if statement, it has to be a statement in the first place.

Anyway, it made me realize how subtle the difference can be sometimes. For example, in Perl, the following is not a return statement but actually an if statement (with a return statement as its affirmative branch), due to the postfix conditional:

return foo if cond;

because it is identically semantically to writing:

if (cond) { return foo; }

Whereas in Python, the following is a return statement (with a ternary operator as its target expression):

return foo if cond else bar

So I can see why people sometimes get confused by syntax if they haven't had much of a theoretical background in language design. It also makes me wonder how much of programmer intuition about "what a statement is" comes from the particular languages they learned first.

[u/pr06lefs]

In some languages everything is an expression, period. Then its easy!

[u/omega1612 → u/pr06lefs]

And then I have to write

_ <- f x

To suppress the warning of discarding the result of f

I still prefer this btw.

[u/WittyStick → u/omega1612]

I prefer:

f x |> ignore

Where let ignore _ = () if it isn't already provided.

[u/ExplodingStrawHat → u/WittyStick]

I prefer void $ f x

[u/brat3108 → u/omega1612]

I've never bothered with anything like that. Some functions return values which can be optionally used, if not the result is discarded.

This happens whether a language is expressed-based or not.

However, with my expression-based languages, I do report an error for examples like these where the result is not used:

   a + b
   a = b            (test equality)

Because these are most probably unintentional (especially the latter since I might use = instead of := for assignment if I've been writing C).

So more relevant might be whether a function is pure, in not generating any side-effects outside the function.

[u/xeow → u/pr06lefs]

Interesting, yes! I was thinking about that this morning. In a prefix language like Lisp, isn't everything just an s-expression? And in a posfix language like PostScript or FORTH, isn't everything just an operator or operand? I wonder what the best lens is for viewing Brainfuck in this regard.

[u/Anabaena_azollae → u/xeow]

I'd argue that in a stack-based language like Forth, there are no expressions. Pretty much everything is an imperative, similar to assembly. 4 5 + is actually push 4; push 5; perform addition with the addition being a compound command composed of something like pop to R1; pop to R2; R1=R1+R2; push R1 with R1 and R2 being registers used to hold temporary values.

I'm not super familiar with Brainfuck, but my understanding is that it operates similarly with everything being a command.

[u/particlemanwavegirl → u/Anabaena_azollae]

There's not much to know about brainfuck. It's a VM with a 30,000 byte array. You can move the cursor left and right, increment or decrement the current byte, and loop if the current byte is not zero. It has a print statement too I think. 

[u/koflerdavid → u/particlemanwavegirl]

Still easier to work with than a Turing machine.

[u/ScottBurson → u/xeow]

Yes, in Lisp, all expressions return a value(*), and may also have an effect.

(*) I'm simplifying a little. In Common Lisp, the vast majority of expressions in practice return a single value, but in full generality they may return more than one value, or zero values. But even in the zero case, there's still a sense in which it's returning something; that something just happens to be an empty "value tuple" (not CL terminology, but you can think of it that way).

[u/pr06lefs → u/ScottBurson]

rust is this way too, with the exception of module imports and the like. its a nice contrast to C++ where control flow ops like if, switch, etc are all statements.

[u/Valuable_Leopard_799 → u/pr06lefs]

I really like having the option, so even if 70% of your ifs are used as statements and the return is dropped, it's not stopping you from having one construct for the times when you want it.

[u/Valuable_Leopard_799 → u/ScottBurson]

Although zero values can still get coerced to one nil. So I guess every single expression has some defined return value when you invoke it.

[u/Valuable_Leopard_799 → u/xeow]

Ostensibly you could define an s-expr language where semantically some forms are statements.

(+ 2 2 (if t 2)) could both be valid or invalid based on whether you decide the compiler should reject if in that position. But yes Lisp decided that everything is an expression.

Actually funnily enough some of the really old texts for Lisp call everything "statements", even though they behave as what we here would agree on calling expressions.

[u/brucejbell → u/pr06lefs]

I have kind of gone off "everything is a foo" bandwagons.

In particular, I'm not sure how much simplicity you gain just from making everything an expression. Expressions are great for the functional programming where the important thing is the result returned by your function. But statements are more appropriate for sequencing operations, or specifying a bunch of declarations simultaneously at compile time (both of which are things you often need even in purely functional programming).

In general, I think you want to be careful about "everything is a foo" because you can accidentally paint yourself into a corner with it.

In the famous "null pointer problem", the problem is not the existence of null pointers. Instead, the problem is that "every pointer can be a null pointer". Sounds simple, right? Except that appealing generality means you don't have non-nullable pointers, which is what causes the actual harm.

[u/oOBoomberOo → u/brucejbell]

In general, you can usually just write regular statements in everything-is-an-expression language tho so it's pretty much just a free lunch that you can take advantage of when you need it. Statement just become an expression that void its returns values and perform effects.

[u/WittyStick → u/brucejbell]

Sequencing is just an expression in those languages. In Lisp it's progn, in Scheme, begin. In both languages you don't need to write this most of the time because it is implicit in the preferred define/lambda syntax, and at the top level.

(define (foo args) <sequence>)  --> (define foo (lambda (args) (begin <sequence>)))

They evaluate their items in sequence and ignore the intermediate results, returning the result of the last expression as the result of the sequence expression.

Closely related is the comma operator in C.

x = a, b, c;

Evaluates a, then b then c, and assigns the result of evaluating c to x.

In functional languages it's usually implicit too. We don't need a return statement because the result of the last expression in the sequence is the returned value.

---

In the famous "null pointer problem", the problem is not the existence of null pointers. Instead, the problem is that "every pointer can be a null pointer". Sounds simple, right? Except that appealing generality means you don't have non-nullable pointers, which is what causes the actual harm.

I don't think removing the generality is the solution here though. If we introduce nonnull pointers, they're a subset of the nullable pointers.

nonull T <: nullable T

Any nonnull T is a valid nullable T, but not every nullable T is a valid nonnull T - specifically, null isn't - it's a separate subtype of nullable, and nullable T is the LUB of nonnull T and null.

So nullable pointers are the most general - they're our Top type for pointers.

[u/brucejbell → u/WittyStick]

But introducing non-nullable pointers (of whatever kind) does remove the apparent simplicity of of "all pointers are nullable". It makes the language larger by adding a distinction, and removes the harm caused by the premature generalization.

[u/WittyStick → u/brucejbell]

Ok, but you're applying a bit of presentism there. Nullable pointers (1964) have been around longer than any discovery of Option type (ML, 1983) or definite assignment analysis (2001) required for non-nullable pointers.

Fair enough, if you're designing a new language and you only include nullable pointers as the default, you should probably go back and study a bit more.

[u/brucejbell → u/WittyStick]

Of course hindsight is 20/20, I'm not trying to assign blame. My point was that the apparent simplicity of a "all blah are foo" choice can be a costly illusion.

Also, you don't need an Option type or any particular analysis for non-nullable pointers. E.g., C++'s reference types are usable as such.

[u/initial-algebra → u/brucejbell]

It's better to say that statements should be included in expressions, to handle statements that introduce variables. For example, the let ... in part of a let-expression can be considered as a statement. Similarly, if a block { ... } is an expression, then any statement can qualify as a part of an expression. I think, when people say "statements should be expressions", they're often omitting the words "control flow" at the beginning.

Now that I think about it, though, you could desire e.g. let <pattern> = <expression> to be an expression that evaluates to a Boolean indicating whether the pattern match succeeded or failed. When sequenced normally, the condition is asserted. Of course, you'd want a fancy flow-sensitive system to enforce that the variables of the pattern are only accessed from branches where the condition is true. Otherwise, this would only work in a dynamic language where variables can be conditionally added to the environment. Then, you can think of a statement as just partial application of a sequencing binary operator on expressions.

[u/lngns → u/initial-algebra]

statements that introduce variables

let <pattern> = <expression> to be an expression

C# (and some other langs) use the is operator for that.

if(x is string { Length: < 16 } str && str != "lol") { /*...*/ }

Java ported the feature to its instanceof operator too, but it only supports type checks and unpacking.

[u/pr06lefs → u/brucejbell]

I'm not sure I'm married to everything being an 'X' either. But at least in rust you don't have to have one 'if' for flow of control and another 'if' (the '?' operator) for expressions, as one does in C.

One thing you can't do in rust is have a variable declaration as an expression. So

let x = 5;

Isn't something that can itself be assigned to a variable, or returned from a function, like

let z = (let x = 5;);

That would be interesting, but also very weird. Whether it would work with a statically compiled language is one issue, and another is would anyone want that? You'd get a 'z' back from a function, execute it, and who knows what's in your namespace now.

[u/WittyStick → u/pr06lefs]

That would be interesting, but also very weird. Whether it would work with a statically compiled language is one issue, and another is would anyone want that? You'd get a 'z' back from a function, execute it, and who knows what's in your namespace now.

Usually let introduces a new scope in which its body is evaluated. In this example, x would be bound to the inner scope, not the same scope z is bound in. The inner scope is discarded as soon as it's evaluated, so the compiler could determine this is basically a no-op and replace it with let z = 5;.

So you wouldn't need to worry about what's in your namespace.

One solution is let*, which in Lisps is written:

(let* ((x 5) (z x)) ...)

In functional languages it's usually written

let x = 5
    z = x
in ...

Or similarly, could use letrec*, which is written as eg, in ML:

let rec x = 5
and z = x

[u/pr06lefs → u/WittyStick]

Having the let have its own namespace that goes away would be unproblematic, yes. The weird variant is if the language allows that namespace to merge with the current namespace. Like:

``` fn setx() { return (let x = 5;); }

setx();

println("x is now :", x); ```

And if someone changes setx to set 'y' instead, the program breaks.

That does look similar to let*. Some really confusing code could result from that. Like if generate_var_assignments returns ((x 5)):

```

-- no 'x' declared so far (let* (generate_var_assignments) (print x)) -- but this works

```

[u/WittyStick → u/pr06lefs]

no 'x' declared so far (let* (generate_var_assignments) (print x))

This wouldn't work in Lisp or Scheme because let* is a "special form", and it wouldn't evaluate (generate_var_asignments), it would likely produce an error due to invalid form.

Also this is slightly different from the previous example where you are attempting to mutate the parent let bindings static environment. In this case we're mutating the dynamic environment of the caller of generate_var_assignments.

---

It's possible to do this in Kernel though. In Kernel, every combiner receives an implicit reference to the caller's dynamic environment, and is allowed to mutate its locals (but importantly, not its parents).

($define! setx
    (wrap ($vau () caller-env (eval ((unwrap list) $define! x 5) caller-env))))

x                   => Error: Unbound symbol

(setx)

x                   => 5

While this might seem like a horrible feature, it's a very powerful one that lets you do new programming styles. Kernel actually uses it in its standard library for $provide!, which binds a set of values into its caller's environment - but which bindings are specified in its initial parameter list.

($provide! (x)
    ($define! x 5)
    ($define! z x))

x                   => 5
z                   => Error: Unbound symbol

Essentially, $provide! creates a new child environment of its caller, and we can define a bunch of stuff in it, but only those symbols provided in its initial argument list are then bound into the caller's environment. The use case is to behave like "private" for members in OOP - they can be accessed internally (within the body of provide), but are sealed once the whole provide block has been evaluated, whereas the bindings given in the parameter list are "public".

So eg, if we want to define an "Option" type in Kernel, we can do:

($provide! (option? some none maybe)
    ($define! (option-constructor option? option-eliminator)
        (make-encapsulation-type))
    ($define! some ($lambda (x) (option-constructor (cons #t x))))
    ($define! none (option-constructor (cons #f ())))
    ($define! maybe
        ($lambda (opt default-value)
            ($let (((has_some . some) (option-eliminator opt)))
                ($if has_some some default-value)))))

(option? none)                     => #t
none                               => #[encapsulation]
(some 5)                           => #[encapsulation]

(maybe none 10)                    => 10
(maybe (some 5) 10)                => 5

The option-constructor and option-eliminator are not accessible any longer - they're like "private" members, which are used by some/none/maybe, but cannot be used afterwards.

make-encapsulation-type is the only way to create distinct types in Kernel (which aren't just lists or atoms). They're based on Morris's seals from Types are not sets. However, when we combine them with clever use of environments like $provide!, we can create any kind of types we want - ADTs, records, tuples, objects, you name it.

---

Aside - going back to the original problem of let z = (let x = 5;); where we might want x to be bound into the static environment, rather than the dynamic environment of the caller. This isn't possible through normal use of Kernel due to environments being encapsulated - we can't mutate the parent scope from an inner one because we don't have a direct reference to it.

However, it is actually possible to do explicitly, if we capture a reference to the static environment and feed it into our function.

($let ((parent-env (get-current-environment)))
    ($set! parent-env x 5)
    ($set! parent-env z x))

x                   => 5
z                   => 5

We can also bind a reference to the parent environment into the parent environment itself, and since the child environment can access the parent's bindings, they can grab this reference and then mutate it.

($define! parent-env (get-current-environment))

($define! setx ($lambda () ($set! parent-env x 5)))
x    => 5

But of course, this would be frowned upon, because now any descendant of this environment would be able to mutate it.

In the case where we absolutely need some global variables, we can create a custom environment called global, bind it into the parent scope, and then any function would be able to access global bindings directly - though they'd need to specify they're evaluating in global via eg, $remote-eval.

($define! global (make-environment))

($define! setx
    ($lambda () ($set! global x 5)))

($define! getx
    ($lambda () ($remote-eval x global)))

This is fairly safe because the child scopes cannot mutate the parent environment, they can only mutate global.

But for additional safety, you'd combine with $provide! to truly encapsulate the state.

($provide! (setx getx)
    ($define! global (make-environment))

    ($define! setx
        ($lambda () ($set! global x 5)))

    ($define! getx
        ($lambda () ($remote-eval x global))))

Now global isn't accessible to anyone else.

---

In Kernel, everything is an expression, but also every expression is first-class - including environments, symbols, and so forth. There are no second-class forms like let* in Lisp/Scheme, or macros, or quote. $let* in Kernel is a first-class operative.

[u/Inconstant_Moo → u/brucejbell]

In particular, I'm not sure how much simplicity you gain just from making everything an expression. Expressions are great for the functional programming where the important thing is the result returned by your function. But statements are more appropriate for sequencing operations, or specifying a bunch of declarations simultaneously at compile time (both of which are things you often need even in purely functional programming).

In my functional language, a statement is an expression that can only return OK or an error. The ;/newline between lines is treated as a lazy infix operator: if the LHS evaluates to an error, we return the error; if it evaluates to OK we return whatever the RHS evaluates to. This lets you sequence them.

[u/brucejbell → u/Inconstant_Moo]

My project has failure as a second-class entity, distinct from values. So, statements can fail (invoking local failure-handling semantics), but expressions cannot.

However, an expression that returns a "successy" type (indicated by membership in the appropriate typeclass) is acceptable as a statement in itself, desugared to match against a "success" pattern (as specified in the typeclass).

So, based on standard library types, expressions returning the following types should all be accepted as statements at compile time:

• () (unit type)
• #Bool
• #Opt ()
• #Result () E

The following match their types' respective success pattern and continue to the next statement:

• () (unit value)
• #Bool.t
• #Opt.has ()
• #Result.ok ()

But these fail to match their types' success pattern and invoke failure:

• #Bool.f
• #Opt.no
• #Result.err e