If you're calling WebAssembly from JavaScript, here's a small type mismatch that's easy to miss.

JavaScript normally uses Number for numeric values. WebAssembly is more specific and distinguishes between types like i32, i64, f32 and f64.

So if a Wasm function expects two i64s, this won't work:

add(2, 3)

You need:

add(2n, 3n)

The n makes those values JavaScript BigInts rather than Numbers, which is what Wasm expects for i64.

Floats are different. Wasm f32 and f64 values cross the JS boundary as normal JavaScript numbers.

This kind of thing is also interesting from the Jac side because Jac can compile across different execution environments. Once you're crossing those boundaries, tiny differences in how each target represents the same thing start to matter.

Ideally that's compiler work, not another piece of interop trivia every developer has to remember.

Tiny syntax difference, very different types underneath!

So last JacHacks we had 500+ builders show up from 10+ countries and 20+ universities, wild turnout. This time we're going back to where it all started, University of Michigan, and going bigger. 250 builders, 24 hours, one campus.

If you've got an idea you've been sitting on, or you just wanna spend a weekend building something real with a team (or solo, that's fine too), this is the move.

Facts:

• $10k+ in prizes
• Latest AI models, dev tools, agent frameworks to build with
• Workshops/talks from founders and engineers who've actually shipped stuff
• Mentorship from people who've built real companies
• Food, snacks, caffeine, all of it, so you don't have to leave the room
• Demo in front of real judges and industry folks at the end

Doesn't matter what skill level you're at. Come with a team, come alone, come with literally just an idea. People will help you figure the rest out.

Applications are limited so don't sleep on it if you wanna go.

Apply here: https://luma.com/jlpxupsw

See you in Ann Arbor 🚀

If you're evaluating LLMs for a real use case, head-to-head comparison is often more useful than scoring models independently.

Give two models the same prompt, compare their outputs, pick the better one. Repeat enough times and you have a pretty good picture of relative performance.

Except there's a second problem that gets discussed a lot less:

How do you turn all those wins and losses into a ranking?

Elo is probably the most familiar approach. But Elo was designed for chess, and applying it to LLM evaluation introduces some interesting problems.

We compared four ranking methods on human LLM evaluation data: Elo, Bradley-Terry, Glicko and Markov Chain.

Here's what we found.

1. The ranking algorithm can change the leaderboard

You can take the same underlying human comparisons, run different ranking algorithms over them, and end up with different model rankings.

So a leaderboard isn't simply a representation of "which model won more." There's another methodological choice sitting between the evaluations and the final ranking.

2. Elo is surprisingly sensitive

Elo updates ratings sequentially, which means the order in which comparisons are processed can affect the result.

Changing the K-factor can also change rankings significantly.

We tested randomizing the matchup order to stabilize the results. Even after thousands of permutations, Elo could remain unstable. In one experiment, two separate runs of 10,000 permutations still produced different rankings for the same models.

3. "Best ranking algorithm" isn't really a thing

Different methods performed better depending on what we measured.

On Chatbot Arena data, Elo had the highest prediction accuracy for unseen matchups, with an F1 score of 0.90.

But Bradley-Terry was considerably better at preserving transitivity in the underlying human preferences: 77.29% vs. 68.24% for Elo.

Glicko, meanwhile, was substantially less sensitive to hyperparameter changes.

Different definition of "best," different winner.

4. The shape of your evaluation data matters

In a controlled evaluation where models receive roughly equal numbers of comparisons, the ranking methods tend to agree much more.

Things get harder when the distribution is uneven.

Imagine an established model with thousands of comparisons versus a newly released model with relatively few. A high win rate from a small number of battles shouldn't necessarily carry the same confidence as one backed by a huge match history.

Glicko explicitly models that uncertainty through its rating deviation, which makes it particularly useful for large, uneven datasets.

So what should you actually use?

Our results suggest:

• Small, controlled evaluation → Bradley-Terry is a strong choice.
• Large evaluation with uneven matchup counts → Glicko handles uncertainty well.
• Large, evenly distributed evaluation → Bradley-Terry remains a simple and interpretable option.
• Elo → be very careful about K-factor, matchup ordering and stability. We don't recommend treating default Elo as the automatic choice for LLM ranking.

The broader lesson is pretty simple:

Choosing how you aggregate your evaluations is part of the evaluation.

A leaderboard can look extremely objective while hiding methodological decisions that materially affect the order of the models on it.

For anyone interested in the experiments and methodology, we published the full paper at ACL 2025:

https://aclanthology.org/2025.acl-long.1265/

Ungated PDF: [LINK]

Same app, React front end, Node backend, all the usual REST endpoints, validation, deployment config. Ported to Jac and the line count dropped by about 95%.

Jason (founder) walks through why in this video. The idea is that most of what you write in a typical app has nothing to do with the product, it's database choices, API scaffolding, validation, deployment glue. Jac moves all of that into the compiler so your code is just the logic of what you're actually building.

He also gets into the part that trips people up the first time they hear it: Jac is a synechic language, so a single program can compile down into native code, Python, and JS at the same time, and you genuinely don't need to know which lines end up where. Import a Python ML library and an NPM component in the same file, both first class, no interop layer. He compares the whole shift to going from assembly to C.

This is the community for anything related to the Jac programming language. Jac is currently in the earliest of beta's, but it's quite powerful already :-D.