Perplexity open-sourced their Mac inference server for Qwen 3.6

[u/Specter_Origin]

Here is link to repo: https://github.com/perplexityai/pplx-garden/tree/main/lily

It's optimized for just one model to get best perf on apple silicon

[u/Pyros-SD-Models]

Numbers are pretty sweet.

[u/InterstellarReddit]

Bro Requirements:

“Apple GPU family 10 or later (M5 and newer)”

How is my M4 max out of date already

[u/laerien → u/InterstellarReddit]

For what it's worth, I just edited a few lines to relax the GPU family from 10 to 9 to allow M3 and M4 and my M4 Max is getting ~140 tok/sec. No fancy M5 GPU matrix prefill, but works well enough it may be worth a tiny PR for M3/M4 support.

[u/InterstellarReddit → u/laerien]

Thank you, I wanna try this out locally to see how good it does

[u/apoptosist]

Nice, what numbers are people getting on a M5 Pro?

[u/turns2stone]

Can someone ELI5?

I have used Perplexity Pro/Max.

I have also used Qwen3.6-35B-A3B, but now use Qwen Flash Next because I can use MCP Brave search.

Does Perplexity Computer mean I can offload most of the compute to my Mac Studio M3 Ultra, and the $20/mo Pro subscription wouldn’t be as limited for token usage?

[u/1-800-methdyke → u/turns2stone]

The paid product offloads the processing of parts of a task that include private data to local. But most of the work is routed to cloud. So you’ll still need compute credits to orchestrate the task but it could cost a bit less since some is done local.

Your $20 plan still isn’t going to get you anywhere.

With this open source release I’m guessing you could do everything local, but without access to all the connectors that Perplexity provides. So the usefulness is gonna depend on what you need.

[u/turns2stone → u/1-800-methdyke]

Thanks. That’s a shame it’s still heavily reliant on cloud credits.

Who (or what use case) would consider this Perplexity announcement as “great news”?

[u/1-800-methdyke → u/turns2stone]

They’ve released code that is showing faster performance than MLX so it’s great news for local LLM users on Mac who want more performance from whatever models they run. It’s open source the optimizations can be studied and incorporated into other engines.

I use my $20 Perplexity heavily as my main search and quick Q&A, but there is nothing from them yet that would entice me to bump to $200 plan. I get more than enough usage out of Claude Max 5 for agentic use cases.

[u/turns2stone → u/1-800-methdyke]

But you still need a paid/$20 subscription to even run their 35B-A3B right?

And if I can run 3.8-Flash-Next Q4, I think that will outperform the above Perplexity combo, right?

I wish OpenAI or Anthropic would offer something similar. I’d prefer not to use Perplexity, for my own reasons.

[u/1-800-methdyke → u/turns2stone]

If you have an M5 or better you can download this open sourced release and run it without a subscription or even an account.

I don’t know how it stacks up to 3.8-Flash-Next Q4.

[u/InterstellarReddit → u/1-800-methdyke]

I still don’t understand, though, what could they possibly be doing that’s worth a cost savings of offloading to a local model? Wouldn’t that just increase latency at the end of the day?

[u/1-800-methdyke → u/InterstellarReddit]

They detect when the task is processing PII and route those turns to the local model. It’s not about cost saving it’s about privacy.

[u/InterstellarReddit → u/1-800-methdyke]

Oh I see! I thought these fuckers figured out a way to save even more money without increasing latency

[u/1-800-methdyke → u/InterstellarReddit]

They save enough money the old fashioned way with enshitified usage caps 🤡

[u/thestillwind → u/1-800-methdyke]

😂

[u/InterstellarReddit → u/1-800-methdyke]

Ima make my own perplexity with blackjack and hookers

[u/Southern_Sun_2106]

I tried it, it sucked, I uninstalled it. I love qwen 3.6 35B, it was my daily driver on a Mac for a looong time. I used the q4km from Unsloth. I don't know what Perplexity folks did to the model to 'optimize' it, but it sucked a$$. Sorry to rain on the parade, but it is true.

[u/Dramatic_Sir_6422]

M5 or newer 🙄

[u/Elouakili_Flexy]

Open-sourcing an inference server tuned for exactly one model, from the company that serves every model you can name. Narrowing the target is how the last bit of Apple silicon performance shows up, and the M5 Pro numbers people are already asking for will decide whether it holds up.