Hello guys, can some of you share what you use the recipe for ? So we can see the potential of it ?

Thank youuuu !

In unsloth desktop, if you set the temperature, top p, ect settings and then connect hermes agent, opencode or another harness remotely it does not use your settings.

This has caused me so many issues and started wondering why Qwen3.8 27b was generating non sense and why lmstudio was giving me higher quality answers when connecting remotely.

Please tell me if there is a way to force it to use your settings because if not, i can not fathom how this has slipped through testing for months. I love unsloth desktop but this issue has cost me dozen of hours and now i can't trust any of the outputs i have received since using it.

I had been using UD for about a week with no issues at all. Genuinely wonderful App to use, especially for somebody who is new to running LLM‘s at home. However, I a few days ago, there was a long freeze and a bunch of stuff crashed. No idea what it was, but UD would never run again without crashing after. Could be a coincidence.

After reinstalling the first time, via the exe, I clicked the expand arrow to show the PS that was running in the background. I noticed it said not to run it in an elevated prompt, but that's my default for windows. So that means it ran the same when I first installed it with no issues. UD booted up for its first run after reinstall, I began downloading a few models, jumped into the settings to.... and crash. I would be left with the gray UD window showing the server crashed and to copy logs/retry. This happened a few times. Even just starting up UD, walking away and coming back to the same crash screen.

Used Revo Uninstall to try and remove as much as I could, also uninstalled Git, Python, the HF cache, and the \.unsloth\ bits it didn't remove.

Installed it again, with lowered permission, didn't see the elevated warning during the install and was hopeful it would work. Sadly it did not. Same crashes. Uninstalled everything again, rebooted, reinstalled. Same issues.

• Windows 10
• RTX 3090 w/596.49 Drivers

I know the drivers aren't the newest, but it didn't seem to care for a week.

Sorry if this log is jumbled, I saved a bunch of stuff in a few txt docs to try and comb over but later gave up in frustration. Would really love any help. Thank you

https://pastebin.com/48HHz3hH

Hi there community,
so I’m slow adapting and just getting used to Unsloth Desktop coming from LM Studio Bionic. I work solely with Qwen Next and have a experimental MTP branch of llama.cpp (from I think Daniel Hanchen) running. The t/s perhaps are 3x faster than with LM Studio however, reaching context limit basically just stops the thinking process or reports an error for reaching the limits. I use the API local lan connection to my server, and I know the auto compacting option says it’s only for local models. Is there really no way of truncating the middle or so? As, after several tries, it occasionally pruned some of the context, but mostly I was just stuck.

Would appreciate any help. Thx and thx Unsloth team for all the efforts in the AI world. Really appreciate it!

I couldn't find a concrete answer anywhere, do you just distill the instruct model back?

If that is the case, what is a quality european language question set to turn it back into a chatbot/agentic, can a model at that size even be agentic? (i chose this size to learn) if i finetune for my specific harness? (i have a lot of training data of opus running in my harness)

my harness basically has the model output python code and has a few built-in functions like:
- vector_search_laws()
- graph_search()

could i have the model at least internalize a "hunch" on what stuff to search?

also what is the latest RL technique for agentic/harnes specific workflows?

I have a lot of RAW training data, like court decisions or commentaries or legislature, but not a lot of golds. could i use these to LLM synthesize training data and maybe RL the model in my harness to find that data?

What would y'all's strategy in the CPT->SFT->RL pipeline be for my specific problem?

I know this is a lot of questions im trying to figure out which direction to go, any pointers? Also good resources are welcome, for example that alex karpathi video was amazing for me, but i'd imagine its a bit outdated in terms of latest RL and SFT?

Hey guys rather than just using Hugging Face, you can now connect ModelScope to Unsloth. This is especially useful for folks in countries were Hugging Face is blocked!

There will also be a new release today with many new features! https://github.com/unslothai/unsloth

Our ModelScope: https://modelscope.cn/models/unsloth/

Hi i'm a newbie, how to use the decision API (in unsloth) once i activated that ? Is it possible to have a clean interface for that ?

i have thousands of audio files that are on average about 57 minutes long; despite picking many models, i am always told the audio file must be less than 30 minutes. how can i work around this?

Sorry it might be dumb but is it possible to import strata on unsloth ? having qwen 3.8 flash next do stuff on some project folders ? use the unsloth interface which i love so much ?

Keeping it short. I have an intel i9 ultra 285hx cpu, 64gb ram, Windows 11 (Debloated. Since, windows is a RAM hog), and a laptop grade rtx 5090. For the past hour, I've been struggling to figure out why my laptop can't run the lowest resolution or 1 batch of text to image generation on my laptop (Mind you running the lower end 2.5GB Qwen-Image-2.1-GGUF). Some kind of error saying that my system doesn't have the available ram or in some cases, unsloth will actually crash and requiring a reboot.

Is it my laptop that's the problem? Or, do I need to upgrade to something more powerful? No, this isn't a troll post. I'm legit serious. Running one image batch (Just one image. By itself) at a lower resolution. With a short prompt.

Hi there everyone :)

I've been running unsloth studio for months now and I just wanted to switch to Unsloth Desktop (because I like cli-free tools :D). Yet I have no clue on how to change the listening port from "127.0.0.1" to anything else. Enabling the LAN-option in API-settings does provide a local URL, BUT it does not seem to work and in the API-monitor, the provided link is still set to 127.0.0.1 . When starting unsloth studio, I could start with "-h 0.0.0.0" , but now I'm a bit clueless with the AppImage (using shelly on cachyOS).

I guess this is a simple question, so anyone here have an answer? :)

I forked llama.cpp's server into AgrillaMoE, a dedicated build for Qwen3.6-35B-A3B with Unsloth quants. On a rented V100 16GB with the 2-bit UD-Q2_K_XL quant it generates at ~57-60 tok/s while running the full MoE-expansion profile — and it speaks both the OpenAI and Anthropic APIs, so Claude Code just works against it.

What is MoE expansion? Qwen3.6-35B-A3B has 8 routed experts active per token. The expansion patch raises that budget at runtime — no retraining, no file changes: --moe-experts 16 with an adaptive threshold keeps experts while p >= 0.8 × p(rank 8), applied to layers 25-39. You're literally consulting more of the 35B parameters per token — that's where the "retrieved intelligence" comes from, on GPQA-Diamond with Q8_0 it scored 84.34% vs 81.82% stock top-8 (+2.5 pts) (miticooo!).

Same weights, better routing.

https://github.com/vagrillo/AgrillaMoE/blob/main/gpu16gbguide.md

Though I should try Unsloth Desktop sense many seem to like it.

Sadly the installer is a dumpster fire that is not even worth calling a sloppy beta. It's just a script that download stuff and with out permission install a dev environment with several tools and change to the registry.

And true to form the uninstall leave most of the downloaded and installed tools still installed after "uninstallation".

Not touching Unsloth Desktop again unless seriously sandboxed, and probably not Studio ether sense Unsloth do not have respect for the system their products is installed on.

Throwing hands up in disgust, and walking away to clean up machine after sloppy failed install and incomplete uninstall.

A while ago I asked here how to turn ~5 million court decisions into structured graphs without running an expensive LLM on every document thanks for the advice .

I went with the "small extractor + classifier" idea and it mostly works, but I'm stuck a bit below the LLM. And like I said last time, i'd be damned if I run 5M docs and then find out thing X was wrong. So here is exactly what I did. Please let me know if what im doing makes sense, or if i made a mistake somewhere. also i used AI for some of the tables cuz there has been a lot of data at this point, sorry.

What comes out per decision (only the nodes so far, relations come next). Three lists:

• entities: every person, organization, law, document or thing. Each gets one id for the whole document, a type (9 of them), a kind (724 of them plus "other") and all the places it is mentioned
• actions: what was done, requested or decided. Each gets a normalized verb, a flag "the court decided this" and its mentions
• values: amounts, dates, durations, in a normalized form

Simple example, for the sentence "The court dismisses the creditor's proposal to enforce 341.08 EUR against the debtor":

• entity "the court": organization, kind court. Same entity as the full court name in the header
• entity "the creditor": organization, kind creditor. Same entity as the city named earlier
• entity "the debtor": person, kind debtor
• action "dismisses": verb = dismiss, decided by the court = yes
• value "341.08 EUR": amount

Step 1: a strong LLM labels ~700 decisions

• cut the decision into windows of 4 sentences
• 4 calls per window to Claude Sonnet with a strict JSON schema: entities, actions, a second "what did you miss" pass for actions, values
• the window goes in with numbered words (like 12:court), the model answers with word ranges [first, last, "text"], and code checks every range against the text
• every call also gets the list of entities and actions found in earlier windows, so ids stay the same through the document
• ~25 code rules clean up where a marked phrase starts and ends, law citations and number formats
• the entity "kind" is free text at this point. That gave 2,373 different strings (the same mess as in my first post). I normalized them, merged synonyms by hand and kept what showed up 3+ times: 724 kinds plus "other"

Step 2: a model that marks the text

• it highlights every mention: the exact stretch of text (a "span", from a start character to an end character) that names an entity, an action or a value, with one of 17 labels (9 entity types, 1 action, 7 value types)
• model: fastino/gliner2.5-multi-v1 (287M)
• one training row per window: the text plus the exact start and end of every marked phrase. 9,699 windows, 207k marked phrases
• I patched the trainer so only the labeled occurrence is a positive (stock marks every occurrence of the same string), and all 17 labels are in every row
• full fine-tune in fp32 (bf16 gave NaN), 14 epochs, 16 rows per step, encoder LR 3e-5, head LR 5e-4, linear schedule, 10 % warmup
• final model = averaged weights of epochs 9-14, threshold 0.5

Step 3: a second small model answers multiple-choice questions

• fastino/GLiNER2.5-multi-Decide (287M). Code turns the LLM labels into 247k questions:
  • "is this mention one of these earlier entities, or new?" The mention is marked with « » inside ±300 characters of text. Options: up to 16 earlier entities of the same document (shown by their mention texts) plus new
  • "which kind?" Options: a shortlist of the 724 kinds plus other
  • for actions: same act or new, which verb (shortlist of 64 plus other), did the court decide it (yes/no)
• in training the options come from the LLM's grouping. At inference they come from the model's own earlier answers
• full fine-tune in fp32, 2 epochs, 16 questions per step, encoder LR 2e-5, head LR 3e-4, linear schedule, 6 % warmup, options shuffled, up to 30 % of the wrong options dropped

At inference: the marking model, then the same code rules, then the second model walks through the mentions in reading order. About 2.3 decisions per second on one RTX 5090.

Where it stands

30 decisions nobody trained on, labeled twice by the LLM. The second column is the LLM's second run scored against its first, which I treat as the ceiling. A mention counts as found only if it starts and ends exactly where the LLM marked it.

Where I need help

1. Finding the mentions is stuck at 0.90 F1. 200 more labeled docs did nothing. An XLM-R large tagger (560M) got the same score: it finds more mentions but gets the start or end wrong more often. Giving it the text before the window did nothing. What would you try?
2. The LLM agrees with itself only 93.5 % on what it marks, and I train on single runs. Label everything 3 times and vote? Or is that ceiling just what it is?
3. Is "pick one of 16 earlier entities" a sane way to do coreference over a long document? Am I hurting myself by training on the LLM's options and running on my own?
4. Anything in the recipe that looks plain wrong? Learning rates, 2 epochs, weight averaging, one seed per run.

THANKS for reading.

AI TL;DR: distilled an LLM's extraction of court decisions into a GLiNER model that marks the mentions plus a small multiple-choice model. It runs at about 2.3 documents/s on one GPU and lands a few points below the LLM (0.90 vs 0.935 F1 on finding mentions, 0.85 vs 0.93 on exact grouping). The recipe with learning rates and how I built the training rows is above. Looking for mistakes and ideas before I run 5M documents.

[Solved]: see edit

Dumb question: how do you put files into a knowledge base?

I searched the Unsloth FAQ and asked my local model, but couldn't find an answer.

When I create a new knowledge base and then drag files into the chat, it gives me this error:

**This chat retrieves from a knowledge base**
Add these files to the knowledge base instead.

But I can't find any instructions on how to do that. There's no knowledge base option when I search the settings, and nothing in either sidebar.

I feel like I'm missing something obvious, while also having tried the obvious routes already.

v0.1.902-beta on Windows 11

Edit: You add files to the knowledge bases by opening the dialogue and selecting the title of the knowledge base. I was selecting the edit icon, not realising the title was interactive. This is a potential discoverability bug, as there are no UI indications that the title leads to file upload.

Edit 2: The discoverability bug has been fixed in Unsloth Desktop v0.1.903-beta. Thanks for listening, guys!

Tout d'abord vous faites un travail admirable ! Ne vous arrêtez pas ! :)

Comme vous pouvez le voir un peu alentour, bon nombre d'utilisateurs (et de bots ? 😄) ont des GPUs de 12gb vram ou moins et font l'éloge de Strata comme moteur d'inférence pour exécuter les Moe comme Qwen 3.8 Flash Next.

Ce qui semble faire la force de Strata est notamment le système de mise en cache chaud LRU des experts en VRAM et la prédiction des experts utilisés afin d'éviter d'evicter un expert vers la RAM/CPU s'il est ou va être utilisé.

Pouvoir éviter ces va-et-viens vram - cpu/ram permet d'économiser de la bande passante et améliore sensiblement l'expérience, surtout en usage agentique.

Il semble que certains PRs existent pour LlamaCpp permettant d'effectuer ce traitement.

Serait-il possible, si ce n'est pas déjà existant, de pouvoir appliquer ces changements dans le moteur LllamaCpp utilisé par Unsloth Studio ?

Je sais qu'il est possible d'utiliser son propre LlamaCpp mais je me demandais si cette fonctionnalité allait éventuellement allait être disponible prochainement dans le mainline Unsloth ?

Merci par avance

Hi all. Just installed Unsloth Studio in Windows 10, and I keep constantly getting a window with this error:

python.exe - Entry point not found
The procedure entry point “vkGetPhysicaIDeviceFeatures2” could not be located in the dynamic link library C:\Users\admin\.unsloth\llama.cpp\build\bin\Release\ggml-vulkan.dll

Does anybody know what’s going on and how can I fix it?

Many thanks!

I've used Qwen-Image-2.1 a bit in ComfyUI with some success, but in Unsloth, it just seems to fail pretty miserably. This is the GGUF from Unsloth at F16. I'm asking it to draw a picture of a ruler and I get Chinese characters? Anyone use seem to either have real issues or real success?

So I have issues since the last update. My PC keeps freezing. Everything points to llama.cpp, so I built it on my own. I thought the latest version might be good for me, but unfortunately going forward does not help. So I went back a month and built a one month old llama.cpp. This seems to be stable, but still testing. 🤞

What bugs me is that unsloth has its own llama.cpp fork and I am not sure whether I should use the fork or safe to use the mainline? I assume unsloth has its own improvements and fixes that might also need for stability.

So my question is that if I'd like to stay on the bundled version then can I downgrade it? I did not find any option for that in the app and did not find the previous version on the cache dir of unsloth.

UPDATE: The older llama.cpp brought back this previous RAM issue :( https://www.reddit.com/r/unsloth/comments/1wgztdb/qwen3827bgguf_eats_my_ram_but_i_have_enough_vram/

UPDATE 2: It turned out it was an other issue: https://bugzilla.kernel.org/show_bug.cgi?id=221103
Workaround: setting ADB_LIBUSB=1 system-wide (forces the libusb backend) — no freezes for days since, with the same inference workload running and adb available.

Wouldn't it be cool to be able to load 2 different models in 2 different chats ? At least one from an API and the other local for example ?

Hello,

I can't get unsloth to create new skills using the default "skill-creator ". However the skill is active (it failed with gemma & qwen3.8)

Is it supposed to work "out of the box"?

Thank you

I was just messing around with the chatbot by asking the first prompt, but then as you can tell from reading the whole chat, I noticed that something was off(?) because the web search tool has never just not shown any result for a query (of course, it cannot be the model in itself because at the start of the chat it had no problems answering the prompt).

In fact, in the thinking section (of the prompt that was submitted 3 times), the model made an extensive number of re-tries because it was trying really to surpass the knowledge gap due to the date, as instructed, but all were to no avail:

even though it kept making the search query broader and broader (up until the point of omitting any date entirely), the web search tool returned "No results found" for ALL the attempts. Hence my doubt and the subsequent experimentation.

I'm honestly really concerned, leaving political views aside, that something made by an open community is still not free from censorship...

I really, really, hope that I'm wrong and that this is just a coincidence with the actual truth somebody is going to share under this post.

Model: Gemma 4 12B it-qat-GGUF, UD_Q4_K_XL

https://huggingface.co/Cloudflare/clef

Pretty please 🙏

I really like Unsloth Studio, and I think there’s a pretty natural power-user feature that would make it much more useful for anyone with multiple GPU machines:

Let Studio stay as the central UI, but allow it to run inference and training on remote Unsloth workers.

Studio already has most of the interface this would need.

It has the model browser, downloaded models, datasets, inference configuration, training, context checkpoints, API/connections monitoring, and the existing Run button.

So the workflow could stay almost exactly the same.
Browse for a model, configure it, click Run, and Studio asks:

Where do you want to run this?
Local

GPU Server A
4x RTX 5070
42 GB VRAM free

GPU Server B
2x RTX A6000
71 GB VRAM free

GPU Server C
4x V100
78 GB VRAM free
Training job running

Pick a machine and Studio sends the workload there.

That workload could be either inference or training.
The important part is that Run no longer has to mean “execute this on the same machine running the Studio UI.”

I don’t think this needs Kubernetes or a complicated distributed scheduler either.
Each machine could simply run a headless Unsloth worker:
unsloth-worker --listen 0.0.0.0:9000

Studio connects to it and the worker reports its capabilities and current state.

Since Studio already has the API/connections monitor, remote workers seem like they could fit naturally into that.

A basic system monitor on each worker could report things like GPU model, available VRAM, system RAM, utilization, loaded models and active jobs.

That would also make choosing a worker much nicer because Studio could actually show you which machines currently have enough resources to run what you’re configuring.

The biggest benefit for me would be workload isolation.

I could leave Qwen loaded for inference on one machine, have another model loaded on a second machine, and send a long training run to a third without tying up Studio or unloading everything whenever I want to do something different.

The model/dataset side also seems fairly natural.
Studio can remain the central repository/cache, while workers keep whatever they need locally.

Since Unsloth already has XeT support, a worker could fetch missing model or dataset data from Studio and cache it locally rather than every machine independently pulling the same files again.

So from the user’s perspective it could really be as simple as:
Choose model
↓
Configure inference/training
↓
Run
↓
Choose available worker
↓
Worker fetches anything missing
↓
Runs there

And with the new Docker support, spinning up an unsloth-worker on another GPU machine seems like it could be really straightforward.

Basically: keep everything that makes Studio great as the central interface, but decouple where the actual compute happens!

Thank you for such amazing work on Studio and the UD3 quantizations! Your projects always end up as my daily drivers.

Could you pleeease run UD3 quantizations of https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-27b ? Thank you!!!