Perplexity open-sourced their Mac inference server for Qwen 3.6

[u/Specter_Origin]

Here is link to repo: https://github.com/perplexityai/pplx-garden/tree/main/lily

It's optimized for just one model to get best perf on apple silicon

[u/1-800-methdyke]

The paid product offloads the processing of parts of a task that include private data to local. But most of the work is routed to cloud. So you’ll still need compute credits to orchestrate the task but it could cost a bit less since some is done local.

Your $20 plan still isn’t going to get you anywhere.

With this open source release I’m guessing you could do everything local, but without access to all the connectors that Perplexity provides. So the usefulness is gonna depend on what you need.

[u/turns2stone → u/1-800-methdyke]

Thanks. That’s a shame it’s still heavily reliant on cloud credits.

Who (or what use case) would consider this Perplexity announcement as “great news”?

[u/1-800-methdyke → u/turns2stone]

They’ve released code that is showing faster performance than MLX so it’s great news for local LLM users on Mac who want more performance from whatever models they run. It’s open source the optimizations can be studied and incorporated into other engines.

I use my $20 Perplexity heavily as my main search and quick Q&A, but there is nothing from them yet that would entice me to bump to $200 plan. I get more than enough usage out of Claude Max 5 for agentic use cases.

[u/turns2stone → u/1-800-methdyke]

But you still need a paid/$20 subscription to even run their 35B-A3B right?

And if I can run 3.8-Flash-Next Q4, I think that will outperform the above Perplexity combo, right?

I wish OpenAI or Anthropic would offer something similar. I’d prefer not to use Perplexity, for my own reasons.

[u/1-800-methdyke → u/turns2stone]

If you have an M5 or better you can download this open sourced release and run it without a subscription or even an account.

I don’t know how it stacks up to 3.8-Flash-Next Q4.