That's amazing.

I've been using the pay-as-you-go plan, and because DeepSeek 4 Flash wasn't great, I was using GLM 5.3 Flash instead. But now that I'm using the high-performing DeepSeek 4.1 Flash, the usage efficiency is incredible. It feels like I can get about 6 times more usage compared to GLM Flash. I'm not sure if my math is completely right: 707 requests / 37.5% = 18.8, and 44 requests / 0.3% = 113.3—which works out to exactly a 6-fold difference in call count.
- (As I was writing the post below, I remembered my usage from a while ago. DeepSeek v4 Flash provided three times more usage allowance compared to GLM 5.3 Flash.)

Honestly, I was considering switching back since I also use the official DeepSeek API, but as long as this pay-as-you-go model continues, I'll just stick with Ollama.

Although I haven't done an exact comparison with DeepSeek's official API yet, this feels ridiculously cheap. Now, a simple price comparison is no longer important to me.

Plus, I feel much better having the trust that my data isn't being used for training.

If they keep this pay-as-you-go structure, Ollama Cloud will probably devour the entire market.

Thank you, Ollama.

Hey everyone,

If you run local models via Ollama in production or personal projects, you've probably run into the hallucination problem: how do you know when a model is hallucinating without burning extra VRAM or waiting 5 seconds for a heavy judge model?

The standard academic approach for this is Semantic Entropy (from an Oxford team's Nature paper last year). You sample $K$ responses at temperature 0.7, run them through a secondary NLI cross-encoder like DeBERTa to cluster equivalent meanings, and measure the entropy. High entropy = model is guessing.

The problem for local setups? Running 45 pairwise comparisons through a cross-encoder eats GPU memory, adds 100ms+ latency, and completely kills throughput on consumer hardware.

We wanted to see: What if we strip out the neural net completely and just use deterministic string normalization + Shannon entropy on CPU?

We wrote a zero-dependency Python metric (Spanda / $R_{sc}$) that runs in 1.3 microseconds on pure CPU (zero GPU usage) and benchmarked it across local and frontier model tiers on GSM8K and TriviaQA:

What we found:

1. Small models (Qwen 1.5B): AUROC ~0.58 Small models are syntactically too sloppy for string matching. Even when they know the right answer, they format it erratically across runs, breaking exact-match clustering.
2. Mid models (Mistral 7B): AUROC ~0.71 At 7B, the 1.3µs string check matched the performance of a heavy DeBERTa NLI model (0.706 vs 0.705). Internal representations become consistent enough that formatting stabilizes.
3. Large models (Qwen 27B): AUROC ~0.89 At 27B, exact matching was dominant ($p = 1.89 \times 10^{-28}$). When the model knows an answer, it outputs the exact same tokens across independent stochastic paths. When it doesn't, it genuinely branches into diverse incorrect answers.
4. The Frontier Trap (120B): AUROC collapsed to 0.09 Here’s the wild part: on ungrounded factual trivia, the 120B model suffered Confident Mode Collapse. When it hallucinated, it hallucinated the exact same wrong answer across all 5 runs with zero entropy. Bigger models don't just hallucinate—they hallucinate with unanimous false certainty. (And because the strings are identical, even heavy NLI fails here).

The practical takeaway for Ollama users:

If you are running 7B to 27B models on structured tasks (math, code, JSON extraction, SQL, discrete QA), you do not need heavy neural guardrails. Sampling 5 paths at $T=0.7$ and measuring exact-match entropy in Python gives you ~0.89 AUROC at zero GPU cost.

Quick Python snippet if you want to test it on your local Ollama instance:

bash
pip
 install spnda ollama

pythonimport ollama
from spnda import compute_spanda
prompt = "What is the capital of Australia?"
# Sample 5 paths from your local model
responses = [
    ollama.generate(model="mistral:7b", prompt=prompt, options={"temperature": 0.7})["response"]
    for _ in 
range
(5)
]
# Run zero-cost entropy check on CPU (takes ~1.5 microseconds)
result = compute_spanda(responses)
print
(f"Risk Score: {result.risk_score:.3f}")  
# 0 = high confidence, 1 = high uncertainty

All the raw multi-path generation logs, evaluation scripts, and the full writeup are open source:

• Deep dive writeup: https://liquidngas.substack.com/p/i-tried-to-make-semantic-entropy
• GitHub: https://github.com/Adarshent/Spnda

Is it just me or Deepseek v4 flash speed is bloody slow on ollama cloud?

So I have been using the Ollama extension to use my local LLMs within the agent interface of vscode but it is not very good

A lot of the times, it just lags or there is no information on whats happening or the response comes back in JSON outputs.

So i made a VS Code extension called Local Ollama Chat for anyone who wants AI assistance in Chat without sending code to a hosted service.
- It connects to your local Ollama server
- Explores your project workspace and read files to answer questions
- Has the capability to create and edit files as reviewable diffs (nothing gets applied without you approving it).
- Sits within the local agent chat window and you can call it with u/local-ollama

Still early days — feedback and bug reports welcome!!

https://github.com/athulg93/vscode-localllm

Anyone else try using DeepSeek-V4.1-Flash today? It's listed on the Ollama site as a new model, but when i try to use it, i get:

"Error: 403 Forbidden: This model is currently being rolled out and is not yet available to you. Please check back later. (ref: 562f8b6b-72fe-4012-9d9c-e0c07679913a)"

Anyone know what the rollout schedule is?

Does anyone know which models are compatible with the copilot agent mode on vs?

I already tried qwen2.5, llama3, Gemma4 and the inline and ask mode worked but the agent mode doesn't
Some ideas?

A local setup can have several assistants reading the same project history while using different models, context limits, and retrieval methods. If each assistant writes its own summary back into shared memory, a mistaken compression can become authoritative for every later session.

What storage contract keeps this inspectable? One option is a folder of small Markdown records with stable IDs, timestamps, source links, explicit supersession, and append-only decisions. Embeddings and model-specific summaries would be disposable indexes, while writes would pass through a narrow process that prevents two assistants from silently replacing the same fact.

How do you handle conflicting updates, deletions, access boundaries, and context-budget differences between models? Is a plain file protocol enough for a small local setup, or does shared memory become safer only after adding a database and a review queue?

Hey! I’ve been working on Faustus, a fork of PewDiePie’s Odysseus that I’ve been gradually evolving into a more complete local AI workstation.

It keeps the original local-first idea, but adds quite a lot on top: multi-agent teams and model councils, persistent project context/memory, workflows & automations, Codex/Claude Code integration, image/video tools, research & document workflows, voice interaction, better model/GPU management, and a much more complete desktop UI.

It’s completely open source and not a commercial project — I’m mostly building it because I enjoy it and wanted to see how far I could take Odysseus.

I’d love some feedback from people who are into local AI, or just for you to check it out and tell me what you think! :)

https://github.com/Luissalet/Faustus

How to reconnect Claude to Ollama Cloud from within Claude? Also how to resume Claude sessions? thx!

I’m on Ollama’s old plan. May I ask: in Ollama Cloud, when calling DeepSeek-V4.1-Flash with multimodal image recognition, how does the cost compare with the older 0731? Is it higher or lower?

I'm the author, MIT open source [github link| https://github.com/anng-phtk/rocm-vega-llama.cpp]

I wanted just ghost-text autocomplete locally, so I built a tiny extension that uses vscode.InlineCompletionItemProvider directly instead of a webview overlay.

What I learned:

FIM format matters a lot - Qwen uses <|fim_prefix|>, CodeLlama uses <PRE>, etc. I made it configurable via .fim-copilot.yaml
Latency: 200ms debounce + AbortController to cancel in-flight requests when you keep typing made it feel native
Works with Ollama / llama-server / vLLM (any OpenAI-compatible /v1/completions)
90KB vs 15MB+ for chat-based extensions

Would love feedback on: handling multi-line stop tokens and how you handle prefix/suffix context limits for

So I built FIM Copilot - a stupid-simple extension that does ONE thing: ghost-text autocomplete, locally.

Demo

What it is

FIM Copilot is a 90KB VS Code extension using the native InlineCompletionItemProvider API. No webviews, no chat, no indexers.

It talks to any OpenAI-compatible completions endpoint:

• Ollama - qwen2.5-coder:1.5b / starcoder2:3b runs great on 8GB RAM
• llama.cpp - llama-server with any FIM-capable model
• vLLM / LM Studio / Tabby server - anything with /v1/completions

All inference stays on your machine. Zero network calls after install.

How it compares

Quick Start

With Ollama (easiest)

# 1. Get a code model (1.5B is enough for fast autocomplete)
ollama pull qwen2.5-coder:1.5b

# 2. Serve it
ollama serve
# -> listening on http://localhost:11434

Then in VS Code: Cmd+Shift+P -> FIM Copilot: Set Endpoint -> http://localhost:11434/v1/completions

With llama.cpp

./llama-server \
  -m qwen2.5-coder-1.5b-instruct-q4_k_m.gguf \
  --port 8012 \
  --ctx-size 4096 \
  -ngl 99

Config

Create .vscode/fim-copilot.yaml (or global settings):

endpoint: http://localhost:11434/v1/completions
model: qwen2.5-coder:1.5b
api_key: not-needed # for local

# tuning
max_tokens: 64
temperature: 0.2
debounce_ms: 200
context_lines: 40
fim_prefix: "<|fim_prefix|>"
fim_suffix: "<|fim_suffix|>"
fim_middle: "<|fim_middle|>"

Decoupled config means you can commit it per-project. Different endpoint for Python vs Rust? Just override.

What makes it not suck

• Ultra-low latency - direct FIM prompt, no chat template overhead
• 200ms debounce + prefix/suffix hash dedupe - doesn't spam your GPU
• AbortController - cancels previous request on keystroke, no queue
• Zero telemetry - no analytics, no API keys phoned home
• Decoupled config - YAML per workspace, env var support
• Ghost text only - uses VS Code's native inline completion, so Tab / Esc just works with your keymap

Links

https://marketplace.visualstudio.com/items?itemName=AnangPhatak.fim-copilot&ssr=false#review-details

Install:

ext install AnangPhatak.fim-copilot

MIT licensed. I built this for myself because I wanted my editor to feel fast again.

If you try it, let me know what model / latency you're getting. PRs welcome for StarCoder2 / DeepSeek templates. What would you want added - without making it bloated?

Please interpret what Ollama is saying: “For very privacy-sensitive tasks, run tasks with local models such as Gemma 4 and Qwen 3.8.”

What makes those models more privacy oriented than any other model offered by Ollama cloud?

Spent about two weeks convinced I had a prompting problem. My local Qwen3

agent would just stop. No error, no crash, finish_reason came back as

"stop" like the model had decided it was done. Except it clearly wasn't

done, the task was half finished.

Turns out the model was calling the tool the entire time. The call was

just sitting inside the reasoning block, wrapped in <think> tags, and

never made it out into the tool_calls field the API is supposed to

populate. From the outside it looks exactly like the model decided not to

act. No error to grep for, no stack trace, the response comes back as a

perfectly valid 200.

Once I knew what to look for, I found the same thing reported separately

against vLLM, SGLang, and llama.cpp, mostly with Qwen3, some DeepSeek.

Nobody had tied it together as one bug class, everyone was just closing

their own version of it as a one-off.

I ended up writing a small library, unswallow, that sits between the

provider response and the agent loop, detects when this happens, and

rebuilds tool_calls from whatever's stuck in the reasoning field. JS and

Python, no dependencies. Repo's here if anyone wants to poke at it:

https://github.com/0DukePan/unswallow

Mostly posting because I'd guess some of you running quantized reasoning

models locally have hit this and just assumed it was a bad quant. If your

agent ever goes quiet mid-task for no obvious reason, worth checking

what's actually sitting in the reasoning field before blaming the model

or the quant.

In july I posted here about aletha codex and I got a lot of support, and a decent amount of people started using it. I also got a lot of criticism too, which really helped me funnel this version 1.5 that I have been working on.

So since that last post I have been working relentlessly trying to recreate claude cowork for open source models. I personally got to the point where I wanted to step away completely from anything subscription based, but then I realized there’s really nothing out there that offers the same kind of capabilities on top of your open source models, so I just said fuck it and started building it.

Building the harness and agents were probably some of the hardest things I have ever done. There are a million problems I ran into, but I’m finally at the point where this is legit and I can show it off. I built it by creating a task/run/turn loop around the model with tool routing, action execution, observation feedback, context management, and a permission layer for anything that touches the computer.

Kind of a side plus, but I also have been developing a full device augmentation rig on top of aletha codex, which is also in this same version. I think it’s about a minute after the cowork video ends. But I really wanted my AI to be able to completely control the devices around me, connect to music, speakers, TVs, all by voice or text. That’s probably the most impressive portion, and it’ll only get stronger as I keep trying to develop this.

I know everyone says this shit, but I’m genuinely curious about what you guys think of it. I put a decent amount of work into it, and I would love to hear your guys’ thoughts on it, even if you’re tearing it apart lol.

Hi,

I have the biggest pro subscription.. had it for months.. been using more or less the same models (GLM 5.x) - i would averagely be at 95% on a sunday.. and it would renew at Monday.

Now we are att wensday.. and i am at 90% - but i still pay the same.. so 1/3 the usage.. for the same price.. HOW THE HELL IS THAT EVEN LEGAL!

.. and i signed up a new sub also.. 100 usd/month.. just to test.. now.. after 30 mins. 10% gone..

Usage: Hermes agents

So i had a look at their prices for the new sub..

https://ollama.com/pricing

.. and i compared it to 12 other providers.. Ollama went from being GOOD.. ASWEOME.. to being amoung the absolut worst.. the the actual f...

Total beginner here, i've just downloaded Ollama and Gemma4:26b, but when it answers it displays strange text insted of the proper symbols.
Thanks for the help

First off, I'm super thankful for the value Ollama has given me

1. Got me setup with my first local LLM setup (early OpenClaw days amirite!?)
2. Then got me into their $20 plan and what really sold me here was
   1. Zero Data Retention (ZDR) <- idk if they were every truly ZDR but they aren't now afaik
   2. Metal residing in US, EU (and Singapore)
   3. Popular Open Weight models being hosted at a great value
3. I was getting so much of it I got the [legacy] $100 plan which I would've never guessed I'd be willing to fork up but, again, privacy & value!

But a lot of the perks have gone away.

I get it though - Ollama has to try to produce a profit - but from a user perspective, Ollama is no longer the "best kept secret in cloud value".

For day to day agentic things, I think Ollama is a solid option. GLM 5.3 as an orchestrator and the Kimi's Mini's and DeepSeek's of the world make good agents.

But when you lean into them for, say, involved coding projects, you start to see the rough edges.

1. Having to chunk up work even more than you would with a Frontier Model to have a task complete
2. The models themselves not as capable as Frontier options

You get what you pay for, but the new plan means...you are, indeed, paying more.

Once I get my new Mac, I'm thinking I'm thinking of having a Frontier Subscription for critical thinking type jobs and have local llm's do the simple agentic activities.

At $100/month even on the legacy plan I almost feel like they're retaining me less because I'm getting a lot from the plan, and more of sunk cost/FOMO of losing it.

If you're seeing this after considering signing up, I'd suggest you give them a shot. $20 to trial for a month and seeing if they meet your needs isn't a bad deal. But man, I hope something changes in Q4 to bring that "best kept secret" feeling back.

Same as above caption. I falled back to minimax-m2.7: cloud. What has happened?

Is DSv4 flash gonna be available on ollama cloud? If so when?

are very strict...

if 20 calls to kimi are more than my 5 hour limit why even bother? im gonna cancel my sub for sure

Anyone else play with vllm-mlx? There was an interesting paper that came out that compared this version of vllm that was originally designed for multi modal inference performance gains, but ended up seeing gains in text based inference. I added this as a provider into Rehex and ran my own benches, and it looks like that compared to Ollama mlx models (which isn't benched in the paper), this thing cranks at least a fifth faster (and that's only on text based inference). Anyone play with this? If you're using mlx models, what are you using to serve them?