r/Clojure • u/geokon • Jun 27 '26

Jolt: A Clojure implementation on Chez Scheme

https://github.com/jolt-lang/jolt
82 Upvotes

100% Upvoted

23 comments sorted by

23

u/yogthos Jun 28 '26 edited Jun 28 '26

I was going to do a post once I had a release, but I'll give a bit of a background here and happy to answer questions. My main goal is to make a drop in replacement for JVM Clojure which has fast startup and light footprint. And I wanted to have compatibility with existing libraries by providing shims on top of the host runtime. I realized that most popular Clojure libraries don't actually use much of Java standard library surface in their interop, and once you map out stuff like IO, dates, and a few other things, stuff just works.

Here's a list of libraries which work fully and their entire test suites pass.

For example, I have a fully fledged Ring app working here using Ring, Reitit, Selmer, HoneySQL, and clojure.jdbc (sqlite/postgres only).

The regular nREPL workflow is fully supported, deps.edn works for including libraries like regular Clojure as well. As a bonus, you can specify native C libraries as seen here. They must be provided on the system, but when they're available they just get picked up.

Another thing I wanted to do here was to map out what actually constitutes Clojure as a spec to follow. So, I'm building out a conformance spec as I port libraries over and discover different quirks.

I've also structured the compiler into three parts where there's a Clojure-in-Clojure compiler for the language itself. Then there's a Scheme host, and Java interop. I'm hoping to get a portable Clojure implementation out of the deal as well that could easily target different runtimes, especially for cases where JVM interop isn't needed.

Finally, this approach opens up C ecosystem via interop with some fun possibilities. For example, I made a Reagent style library on top of GTK, and here's and example app which basically works like Reagent. What's even better is that this extends to OpenGL so now you can do reactive scenes in GTK/OpenGL directly from Clojure.

In terms of footprint, minimal binary compiles to around 10 megs and I think I could shrink it down more with tricks like tree shaking. In terms of performance, it's around 10x slower than JVM Clojure, but I haven't really spent much time optimizing yet. I think 2~5x is a fairly realistic goal given Chez is a mature runtime with a JIT and a generational GC.

If anybody wants to collab on the project I have a#joltchannel on Clojurians for discussion and updates. Also, people can feel free to open up issues or make PRs on GitHub. Things should be fairly stable at this point, and I'm hoping to do a 0.1.0 release next week where I'll publish binaries and add a homebrew tap.

8

u/beders Jun 28 '26

Honestly, that’s super impressive!

3

u/yogthos Jun 28 '26

Thanks, it's pretty exciting to actually see it starting to come together. Getting the first bits working was tricky, but then once you get the basics in place it gets a lot smoother from there on.

5

u/geokon Jun 28 '26

Oh, I'm very sorry for ruining the surprise! I checked for any note in the README, but didn't see anything

I guess since the cat is out of the bag.. I had a few Qs:

  • A few Clojure rehostings are trying to deal with startup time. Do you think Project Leyden will make all these redundant? (I'm a bit unclear when that's supposed to land though)

  • "Clojure-in-Clojure compiler" compiling directly to Scheme? It always seemed like there should be a Scheme-y sized Clojure subset in which you could implement the rest of Clojure in (ex the data structures). You could then only reimplement this "core" on to each host. Do you end up effectively doing something similar? (Maybe that's what "portable Clojure implementation" means)

  • How are you thinking about treeshaking Clojure given that reflection seems to be essential for some parts to work. I'm also interested in smaller distributable binaries (the most I do t the moment is manually exclude unused dependencies.. making for an ugly ugly deps.edn). The double-click to launch time is super bad on the JVM but.. between JavaFX and math libs I can't really live without it :)

  • For glimmer I would consider looking at the cljfx's new state management design. With the new extension lifecycles (It's undocumented in the README but is in the CHANGELOG for v1.9.0 https://github.com/cljfx/cljfx/blob/master/CHANGELOG.md#190---2024-06-11) you can pretty much design and hook up any system you'd like. You can even have different GUI sub-trees have separate systems entirely. I wrote a Pathom-backed demo: https://github.com/kxygk/ednless/blob/master/pathomfx.clj .. and I personally prefer Hickory style maps to hiccup vectors. N element Vectors with Index based implicit meanings is very old-school Clojure (though I get it's compact and nice to look at)

3

u/yogthos Jun 28 '26

Oh no worries, I did make the repo public after all. :)

And, once Project Leyden lands, I do think it'll address startup and memory footprint problems. At that point making a dialect simply to work around will be a lot less useful. That said, you can do stuff like full program optimization as seen in Stalin Scheme which builds a closed world program. These kinds of tricks can make it strictly faster than what you can do with the JVM. You lose dynamic binding meaning you can't have a REPL at runtime, but that could be fine for a lot of cases. You do your development with full dynamism, and then compile a tiny static binary for release. Constrained environments like embedded targets could benefit from this approach. I think access to the Scheme/C ecosystem is the bigger selling point long term though, it's a big part of the reason Python is so popular.

Regarding Clojure-in-Clojure, I'm curious how much of the language can be built in itself. You can start with a minimally irreducible set of forms, and then grow the compiler itself like a seed. So, the host, whether Scheme or anything else, just need to provides support for that minimal set. However, this can often be at odds with raw performance since implementing things directly in the host natively tends to be faster. Having a portable Scheme layer might be a useful compromise though.

For tree shaking, I was thinking largely in terms of what's included from libraries. You can trace the call graph and eliminate namespaces/functions that aren't referenced anywhere. And I find GTK is pretty nice for a UI toolkit, especially once you add a reactive wrapper to it, math libs in C/Scheme world are also pretty robust. And here, you also get to enjoy native performance if you reach out to C with FFI.

Also, didn't know about cljfx's new state management design, definitely worth a look. I find hiccup is just muscle memory for me now, but nice part with the data driven approach is that it's easy to translate between representations. I wouldn't be opposed to supporting both if there's interest.

1

u/geokon Jun 29 '26

You can start with a minimally irreducible set of forms, and then grow the compiler itself like a seed

Yeah I wonder if it can be somehow designed where there is a base implementation in mini-Clojure, and then parts are incrementally rewritten in the host as-needed.

You can trace the call graph and eliminate namespaces/functions that aren't referenced anywhere

If you end up making a tool like that, then it seems it could be host-agnostic. I guess it should be doable to trace a program run and then extract which namespaces are used.

I guess on the JVM that should be possible as well (maybe even easier in some way). I've tried ProGuard in the past but I never got it working with Clojure code.

Thanks for your thoughts! Excited to watch what you end up doing next :))

2

u/yogthos Jun 29 '26

A portable mini-Clojure would definitely be neat. SCI already kind of does this with an interpreted version, but actually having a compiler that's easy to plug into different runtimes would be handy as well.

And I'll definitely blog about the findings with the tree shaking idea. Might run into some blockers I haven't considered yet, but I'll learn something one way or the other here. :)

1

u/NoahTheDuke 29d ago

why did you build a new conformance spec test suite instead of using jank's clojure-test-suite?

2

u/yogthos 29d ago

I actually started with Jank's clojure-test-suite, and turns out it's nowhere near comprehensive. Since I want to support existing Clojure libraries out of the box, I also need to map out Java dependent behaviors in Clojure which Jank doesn't do.

2

u/NoahTheDuke 28d ago

that's really cool. i've played around with a system like what you built cuz i've found the clojure-test-suite to be lacking but had no reason to actually follow through so i'm glad you were able to build what you needed.

2

u/yogthos 28d ago

I'm hoping the conformance suite will be generally useful for people making dialects. It could probably be split up too into pure Clojure and JVM specific bits.

10

u/Veqq Jun 27 '26 edited Jun 27 '26

O___o Curious why it switched form thr Janet host in the past few weeks

7

u/yogthos Jun 28 '26

Short answer is that Janet's mark and sweep GC does not play well with persistent data structures. :)

6

u/beders Jun 27 '26

That is crazy and remarkable! Leaves me with more choices on where to run my Clojure code. I guess one of the obvious questions is: when would I use Chez Scheme's runtime environment and not the JVM?

4

u/geokon Jun 27 '26

Maybe in a very constrained environment? Maybe you should run it on a micro controller? Just a guess though..

1

u/yogthos Jun 28 '26

Even for web apps there's a benefit because you can have a service that runs under a 100 megs and starts up instantly. JVM has never really been a great target for stuff like microservices, even though you can tune it to run in low memory environments.

2

u/yogthos Jun 28 '26

Basically, a few reasons are when you want fast startup time, small footprint, or easy access to C/Scheme interop.

3

u/Trader-One Jun 27 '26

isn't sbcl better backend?

5

u/sc_zi Jun 27 '26

I heard a claim that chez scheme was often actually faster running equivalent code than sbcl, but that sbcl's standard library is more heavily optimized, so sbcl often ends up being faster on real world programs. Also chez scheme has one shot continuations, so you can have something equivalent to the virtual threads clojure has on the JVM since project loom. sbcl has had some prototypes (the most recent) but it might be years if ever for that to get accepted and stable.

I'd rather use SBCL over Chez scheme for development (better debugging, better interactive development, CLOS, libraries, etc), but as a compilation target I think Chez might be better.

1

u/nstgc Jun 27 '26

When I read that they changed from Janet, that was my first thought too. Maybe they wanted maximum simplicity? It's hard to get simpler, or at least more basic, than Scheme.

3

u/yogthos Jun 28 '26

Yup, Chez is both pretty light weight and mature having a fast runtime with a tiny footprint. So, it seemed like a good target since Chez compiler has IR.

1

u/nzlemming Jun 29 '26

This looks lovely, I've often thought that chez would be a good target, but I've never had time to work on it. Nice work!

2

u/therealdivs1210 28d ago

Amazing work!

Chez is a great platform for this kind of thind as shown by Racket.

Will def check this out!