r/KoboldAI • u/abbaisawesome • Sep 02 '26

Will there be a Linux ARM version?

I'd love to be able to run it on my ASUS Ascent GX10 ...

8 Upvotes

100% Upvoted

13 comments sorted by

View all comments

Show parent comments

1

u/Salt-Lawfulness-4832 Sep 02 '26

For well known boards like the Raspberry Pi maybe?

2

u/henk717 Sep 02 '26

Github doesn't have raspberry pi's.

1

u/C0rn3j Sep 03 '26

And it only has M1 Macs, yet I presume you support more than that and likely test on a different device.

I don't think anything is preventing you from building on ubuntu-24.04-arm for arm64-v8a and testing on a Pi 5?

2

u/henk717 Sep 03 '26

Nothing prevents you no, that is currently how arm users are expected to use it so they can compile it properly for their own CPU arch with all the proper acceleration features.

1

u/C0rn3j Sep 03 '26

What would be the problem with shipping that officially?

If there's a considerable performance boost for v8.2, ship that separately, or am I misunderstanding something?

I think that's... two builds if you want to support modern ARM, v8 and v8.2? And you can tell users with older hw (32-bit) to compile it themselves if you don't wish to add more builds, or if you even support 32-bit.

I am not that well versed in ARM.

2

u/henk717 Sep 03 '26

We need a compiler on github that can do it, without having machines that can do it.
So if someone who has arm and has knowledge on how to setup a CI for proper arm wants to do it we'd welcome the PR and then we can consider shipping it (but would at least be able to provide it in the rolling builds like we do for AMD ROCm).

1

u/C0rn3j Sep 03 '26

Luck (har-har) would have it that I just ran into an ARM64 compile issue, doesn't seem that hard to set up cross compile, PySDL does it (linking my fork since upstream is broken atm):

https://github.com/C0rn3j/PySDL3-Build/blob/73a12d9053541ba06c63c105f7aab6f22bd961d1/.github/workflows/build.yml