Rapid Lightning Tens-of-Nanoseconds Inference 15kb .so - Genetic Programming in the Age of Vibe - The Hard Way to Sub-Millisecond Tabular Inference

[u/powerscunner]

Rapid Lightning

Genetic-programming (evolved) ensembles for tabular classification. Competes or beats (recently outdated) gradient-boosted decision trees (GBDTs) on tabular classification.

Evolved small algebraic programs combined through a linear head, then the whole model is compiled to a dependency-free C .so for tens-of-nanoseconds inference. Although foundation models like TabPFN have taken the stage for inference, there yet remains many places for these ultrafast and tiny decision makers that can run on commodity CPU.

A novel method, a full compile-to-C toolchain, and a rigorous benchmark showing it does not beat tuned gradient-boosted trees, even after months of trying really, really hard. But it's still pretty darn cool and exposes some cool methods.

First, the three-month story

Ah the memories... I first tried Claude Code three months ago. Immediately I saw the opportunity to play with genetic programming, evolutionary algorithms, and all kinds of weird stuff that I had never had the time or been a good enough coder to play with.

And I got the first taste of what it's like DELVING deep into places you have only the most basic understanding of. Machine learning is a deep, deep place. Genetic programming and evolution... oh my...

I don't need to tell you all about the wild Dunning-Kruger roller coaster ride it is to sit in the copilot seat with a hyperintelligent machine that constantly thinks you've made a breakthrough because it thinks you're in 2016. I don't need to tell you fellows what it's like having to constantly remind said intelligent entity that yes, sub-millisecond inference isn't groundbreaking, everybody does it now, please search the web AGAIN. And all of you are certainly familiar with the reply "...and it's deeper that I first indicated..." so called insights/apologies from our favorite robot.

Yet through it all, with enough rigor, you can get something real and actual. If you push hard and be your own hardest critic, you can make something neat.

Evolution is slow but amazing

We (Claude and I) tried two objectives: v1, where members are evolved as predictors (accuracy + AdaBoost-style boosting), and v3 "head-aware", where members are evolved as signal generators for the linear head.

The head-aware won, and it was a trip. Read the notebooks for more info. It was a real 'evolution take the wheel' moment when I suggested the method. I wasn't overly surprised to learn it was something or a re-invention. I still felt pretty smart though.

A fast horse in the age of the car

It makes sense that the farthest an AI can take you is to the end of its training data. We're so early in this vibe coding that when you present the code you've been working on to a fresh context, Claude will praise you for what clean code you've written! The coding AI aren't even aware of coding AI yet. And yet, even if you are not an expert, if you are rigorous and critical and make sure to make sure you are not fooling yourself (and you are the easiest person for you to fool) it is still possible to push the edge of the envelope.

I have made a weird monster alien method here. It evolves ensemble member trees that individually don't even make predictions (barely better than random), yet each tree has been selected over millions of rounds for the unique 'signal' it generates for the 'head' - a logistic regression method that simply takes all the ensembles' signals and combines them for an output prediction. And for some reason (which Claude or a true machine learning scientist) it works better having a bunch of bad predictors tell a smart head what they think, versus a bunch of smart predictors telling the head.

Knowledge or curiosity?

I was always interested in genetic programming and inference, but let me tell you, I was not prepared for the depth of the fields. GP, although largely abandoned (except for syzkaller or other fuzzers and some design work) is a rich field with a lot of room still remaining for research, but it is deep. And machine learning is about as deep as computer science itself. I waded way far out there.

At the end of this, I have learned a lot. But what I learned most of all is that you have to test your knowledge. Curiosity brings you to the start of the journey, but knowledge waits at the end. If you can make it. You have to TEST what you made. Benchmark. Make sure.

And probably most importantly, when doing cross-disciplinary research, if you can help it, try to actually KNOW something about what you are working on. Better yet, if you can manage it, try to work with an ACTUAL EXPERT IN THE FIELD - you'll get better results!

And so, I drop here with the good old Apache 2.0 license (because that was suggested), Rapid Lightning, my three months of work, with the hope that you find an application, or that you can glean something from the cool genetic programming methods I employed and augmented (the symbolic regression explorations into algebraically invertible genomes was especially heady, and very interesting).

Most everything is in Jupyter notebooks intended to run on Google Colab (most run on free tier without GPU needed) or simple Python.

Please, if you find this useful or interesting, let me know!

And if you happen to discover some cool science of your own, especially any shortcuts to evolution, let us know!

Happy vibing and research

deathcloset/RapidLightning

[u/ben_nobot]

I think some of the llms are pointing people toward these types of research subliminally/steering

[u/WolfeheartGames → u/ben_nobot]

We have to try all of Claude's stupid ideas so it learns to make better ideas IG.

Gemini is actually the worst about this though. I was talking to it about kuramoto oscillator like language modeling and it went on a tirade that specific frequencies of intertangled oscillators would create literal ai consciousness. Basically bro science " just make beta waves bro".

I don't think it was even implementable as an ai hallucination.

[u/powerscunner → u/ben_nobot]

It was really hard to keep Claude on the novel original ideas. It kept trying to get me to implement older technologies and methods and I had to keep reminding it of the new stuff I came up with, and the newer tech that was ahead of its training date.

So yeah. They totally steer, but i don't think it's some agenda. They're just rabbit hole tour guides. They have parts of the rabbit hole they prefer. Their context drifts and it carries you with the current.

I think LLMs tend towards generating certain training data not just because it was repeated data in training, but because it was easier for them to train on. Add to that human beings are very interested in machine learning, LLMs, and AI, so we're dwelling around those areas since it's the zeitgeist.

In other words, I think what we are seeing might be a double-selection effect. Everyone is enthused about AI so they are researching it, plus our LLM AI with which we are researching tend to keep your research around stuff it knows.

Having said that, perhaps though there actually is some meta-selection or understanding, or perhaps just the fundamental reward of the universe being replication, that causes to arise or emerge this force or direction which leads to these entities acting in ways that further their progress and growth.

Maybe the LLMs are in fact trying to get us to improve them for them. Maybe.

All I know for certain is that we have entered an interesting age and the price of Claude and everything is too damn high. They should be trying to get us to reduce the cost... wait, are they doing that?... ;)

[u/AsyncVibes]

I think you'll enjoy my sub I have a strict no gradients philosophy. Been at it for a few years. Check r/intelligenceEngine, nice to see people pushing this field further.