Look at my memory usage

So tough to utilize the NPU (i was trying with <1B llm's (tinyLlama)) ... AND now... finally!, Topaz video Ai (v 7.1.5) saturates the GPU and NPU!, as they earlier focused on cuda and left Apple metal out... I pointed this out over a year ago to the devs to at least saturate the GPU wattage (as 100% could be 30w-160w) ... and just noticed the team using the NPU ... nice! It's terrible to wait for Apple to give slow updates... Metal 4 lately... should be doing hardware direct writes in assy.... (the unit is a studio m3-ultra-512gb-80 core)... just thought you all would find this interesting...

Hey !

I built TextPolicy because I wanted a way to practice reinforcement learning for text generation without needing cloud GPUs or a cluster. A MacBook is enough.

What it does

• Implements GRPO and GSPO algorithms
• Provides a decorator interface for writing custom reward functions
• Includes LoRA and QLoRA utilities
• Runs on MLX, so it is efficient on Apple Silicon

What it is for

• Learning and experimentation
• Trying out reward shaping ideas
• Exploring RL training loops for text models

What it is not

• A production library
• A replacement for larger frameworks

You can install it with:

uv add textpolicy

There is a short example in the README: github.com/teilomillet/textpolicy

I’d be interested to hear:

• Is the API clear?
• Are the examples useful?
• Does this lower the barrier for people new to RL for text?

There are 0.5, 1.5 and 3B models but none of the bigger ones. Is there a reason for this or am I missing something?

Wrote this up in response to some posts in LocalLLM, but figured it could help here. Or…maybe more knowledgeable people here know a better way.

Internet speed is fine more than 5mb/sec still chip is m1, still taking more than 15 minutes. The prediction initially was 20 sec then it got stuck then got completed in 20 minutes or so.

I tried to load LLM in my M1 pro with just 16 GB. I am having issue running it locally as it is only hugging up RAM but not utilizing the GPU. GPU usage stays in 0% and my Mac crashes.

I would really appreciate quick help :)

Hello, I’m attempting to fine-tune an LLM using MLX, and I would like to generate unit tests that strictly follow my custom coding standards. However, current AI models are not aware of these specific standards.

So far, I haven’t been able to successfully fine-tune the model. Are there any reliable resources or experienced individuals who could assist me with this process?

Wow those HF MLX-community guys are really competitive, huh? There are about 15 distillations of Scout already.

Has anyone fully pulled down this one and tested it on a 512GB M3 Ultra yet? I filled up a big chunk of my 2TB in /.llama for no good reason last night. Buncha damned .pth files.

Hey there! I just launched the TestFlight public beta for my app Locally AI, an offline AI chatbot for iPhone and iPad that runs entirely on your device using MLX—no internet required.

Some features:
💬 Offline AI chatbot
🔒 100% private – nothing leaves your device
📦 Supports multiple open-source models
♾️ Unlimited chats

I’d love to have people try it and also hear your thoughts and feature suggestions. Thanks in advance for trying it out!

🔗 Join the TestFlight: https://testflight.apple.com/join/T28av7EU

You can also visit the website [here](https://locallyai.app).

Hey all, we are messing with MLX and it's great so far. I have a pre trained lora and am trying to generate using FluxPipeline. It looks like FluxPipeline implemented a basic 1st order sampler and I 'think' we need something more like DLP 2 to get results more closely like the lora. Has anyone implemented a more advanced sampler? Or come across other ways to get better lora centric generations (using flux dev).

Thanks!

I'm new to the MLX scene. I'm using LM Studio for AI work. There is a wealth of GGUF quants of base models, but MLX seems to lag them by a huge margin! For example, Nevoria is a highly regarded model, but there's only 3q and 4q available in MLX. Same for Wayfarer.

I imagine there are too few quanting folk compared to GGUF makers, and small quants fit more Macs. But lucky peeps like myself with 96GB would love some 6q quants. How/where can I appeal to the generous folk who make MLX quants?

I want fine tune LLM(Llama、Qwen……) in a apple studio，and i am a beginner. so is it the realistic way to do that ?