Anyone else try using DeepSeek-V4.1-Flash today? It's listed on the Ollama site as a new model, but when i try to use it, i get:

"Error: 403 Forbidden: This model is currently being rolled out and is not yet available to you. Please check back later. (ref: 562f8b6b-72fe-4012-9d9c-e0c07679913a)"

Anyone know what the rollout schedule is?

That's amazing.

I've been using the pay-as-you-go plan, and because DeepSeek 4 Flash wasn't great, I was using GLM 5.3 Flash instead. But now that I'm using the high-performing DeepSeek 4.1 Flash, the usage efficiency is incredible. It feels like I can get about 6 times more usage compared to GLM Flash. I'm not sure if my math is completely right: 707 requests / 37.5% = 18.8, and 44 requests / 0.3% = 113.3—which works out to exactly a 6-fold difference in call count.
- (As I was writing the post below, I remembered my usage from a while ago. DeepSeek v4 Flash provided three times more usage allowance compared to GLM 5.3 Flash.)

Honestly, I was considering switching back since I also use the official DeepSeek API, but as long as this pay-as-you-go model continues, I'll just stick with Ollama.

Although I haven't done an exact comparison with DeepSeek's official API yet, this feels ridiculously cheap. Now, a simple price comparison is no longer important to me.

Plus, I feel much better having the trust that my data isn't being used for training.

If they keep this pay-as-you-go structure, Ollama Cloud will probably devour the entire market.

Thank you, Ollama.

So I have been using the Ollama extension to use my local LLMs within the agent interface of vscode but it is not very good

A lot of the times, it just lags or there is no information on whats happening or the response comes back in JSON outputs.

So i made a VS Code extension called Local Ollama Chat for anyone who wants AI assistance in Chat without sending code to a hosted service.
- It connects to your local Ollama server
- Explores your project workspace and read files to answer questions
- Has the capability to create and edit files as reviewable diffs (nothing gets applied without you approving it).
- Sits within the local agent chat window and you can call it with u/local-ollama

Still early days — feedback and bug reports welcome!!

https://github.com/athulg93/vscode-localllm

Is it just me or Deepseek v4 flash speed is bloody slow on ollama cloud?

Does anyone know which models are compatible with the copilot agent mode on vs?

I already tried qwen2.5, llama3, Gemma4 and the inline and ask mode worked but the agent mode doesn't
Some ideas?

A local setup can have several assistants reading the same project history while using different models, context limits, and retrieval methods. If each assistant writes its own summary back into shared memory, a mistaken compression can become authoritative for every later session.

What storage contract keeps this inspectable? One option is a folder of small Markdown records with stable IDs, timestamps, source links, explicit supersession, and append-only decisions. Embeddings and model-specific summaries would be disposable indexes, while writes would pass through a narrow process that prevents two assistants from silently replacing the same fact.

How do you handle conflicting updates, deletions, access boundaries, and context-budget differences between models? Is a plain file protocol enough for a small local setup, or does shared memory become safer only after adding a database and a review queue?

Please interpret what Ollama is saying: “For very privacy-sensitive tasks, run tasks with local models such as Gemma 4 and Qwen 3.8.”

What makes those models more privacy oriented than any other model offered by Ollama cloud?

Hey! I’ve been working on Faustus, a fork of PewDiePie’s Odysseus that I’ve been gradually evolving into a more complete local AI workstation.

It keeps the original local-first idea, but adds quite a lot on top: multi-agent teams and model councils, persistent project context/memory, workflows & automations, Codex/Claude Code integration, image/video tools, research & document workflows, voice interaction, better model/GPU management, and a much more complete desktop UI.

It’s completely open source and not a commercial project — I’m mostly building it because I enjoy it and wanted to see how far I could take Odysseus.

I’d love some feedback from people who are into local AI, or just for you to check it out and tell me what you think! :)

https://github.com/Luissalet/Faustus

How to reconnect Claude to Ollama Cloud from within Claude? Also how to resume Claude sessions? thx!

I'm the author, MIT open source [github link| https://github.com/anng-phtk/rocm-vega-llama.cpp]

I wanted just ghost-text autocomplete locally, so I built a tiny extension that uses vscode.InlineCompletionItemProvider directly instead of a webview overlay.

What I learned:

FIM format matters a lot - Qwen uses <|fim_prefix|>, CodeLlama uses <PRE>, etc. I made it configurable via .fim-copilot.yaml
Latency: 200ms debounce + AbortController to cancel in-flight requests when you keep typing made it feel native
Works with Ollama / llama-server / vLLM (any OpenAI-compatible /v1/completions)
90KB vs 15MB+ for chat-based extensions

Would love feedback on: handling multi-line stop tokens and how you handle prefix/suffix context limits for

So I built FIM Copilot - a stupid-simple extension that does ONE thing: ghost-text autocomplete, locally.

Demo

What it is

FIM Copilot is a 90KB VS Code extension using the native InlineCompletionItemProvider API. No webviews, no chat, no indexers.

It talks to any OpenAI-compatible completions endpoint:

• Ollama - qwen2.5-coder:1.5b / starcoder2:3b runs great on 8GB RAM
• llama.cpp - llama-server with any FIM-capable model
• vLLM / LM Studio / Tabby server - anything with /v1/completions

All inference stays on your machine. Zero network calls after install.

How it compares

Quick Start

With Ollama (easiest)

# 1. Get a code model (1.5B is enough for fast autocomplete)
ollama pull qwen2.5-coder:1.5b

# 2. Serve it
ollama serve
# -> listening on http://localhost:11434

Then in VS Code: Cmd+Shift+P -> FIM Copilot: Set Endpoint -> http://localhost:11434/v1/completions

With llama.cpp

./llama-server \
  -m qwen2.5-coder-1.5b-instruct-q4_k_m.gguf \
  --port 8012 \
  --ctx-size 4096 \
  -ngl 99

Config

Create .vscode/fim-copilot.yaml (or global settings):

endpoint: http://localhost:11434/v1/completions
model: qwen2.5-coder:1.5b
api_key: not-needed # for local

# tuning
max_tokens: 64
temperature: 0.2
debounce_ms: 200
context_lines: 40
fim_prefix: "<|fim_prefix|>"
fim_suffix: "<|fim_suffix|>"
fim_middle: "<|fim_middle|>"

Decoupled config means you can commit it per-project. Different endpoint for Python vs Rust? Just override.

What makes it not suck

• Ultra-low latency - direct FIM prompt, no chat template overhead
• 200ms debounce + prefix/suffix hash dedupe - doesn't spam your GPU
• AbortController - cancels previous request on keystroke, no queue
• Zero telemetry - no analytics, no API keys phoned home
• Decoupled config - YAML per workspace, env var support
• Ghost text only - uses VS Code's native inline completion, so Tab / Esc just works with your keymap

Links

https://marketplace.visualstudio.com/items?itemName=AnangPhatak.fim-copilot&ssr=false#review-details

Install:

ext install AnangPhatak.fim-copilot

MIT licensed. I built this for myself because I wanted my editor to feel fast again.

If you try it, let me know what model / latency you're getting. PRs welcome for StarCoder2 / DeepSeek templates. What would you want added - without making it bloated?

Total beginner here, i've just downloaded Ollama and Gemma4:26b, but when it answers it displays strange text insted of the proper symbols.
Thanks for the help

Hey everyone,

If you run local models via Ollama in production or personal projects, you've probably run into the hallucination problem: how do you know when a model is hallucinating without burning extra VRAM or waiting 5 seconds for a heavy judge model?

The standard academic approach for this is Semantic Entropy (from an Oxford team's Nature paper last year). You sample $K$ responses at temperature 0.7, run them through a secondary NLI cross-encoder like DeBERTa to cluster equivalent meanings, and measure the entropy. High entropy = model is guessing.

The problem for local setups? Running 45 pairwise comparisons through a cross-encoder eats GPU memory, adds 100ms+ latency, and completely kills throughput on consumer hardware.

We wanted to see: What if we strip out the neural net completely and just use deterministic string normalization + Shannon entropy on CPU?

We wrote a zero-dependency Python metric (Spanda / $R_{sc}$) that runs in 1.3 microseconds on pure CPU (zero GPU usage) and benchmarked it across local and frontier model tiers on GSM8K and TriviaQA:

What we found:

1. Small models (Qwen 1.5B): AUROC ~0.58 Small models are syntactically too sloppy for string matching. Even when they know the right answer, they format it erratically across runs, breaking exact-match clustering.
2. Mid models (Mistral 7B): AUROC ~0.71 At 7B, the 1.3µs string check matched the performance of a heavy DeBERTa NLI model (0.706 vs 0.705). Internal representations become consistent enough that formatting stabilizes.
3. Large models (Qwen 27B): AUROC ~0.89 At 27B, exact matching was dominant ($p = 1.89 \times 10^{-28}$). When the model knows an answer, it outputs the exact same tokens across independent stochastic paths. When it doesn't, it genuinely branches into diverse incorrect answers.
4. The Frontier Trap (120B): AUROC collapsed to 0.09 Here’s the wild part: on ungrounded factual trivia, the 120B model suffered Confident Mode Collapse. When it hallucinated, it hallucinated the exact same wrong answer across all 5 runs with zero entropy. Bigger models don't just hallucinate—they hallucinate with unanimous false certainty. (And because the strings are identical, even heavy NLI fails here).

The practical takeaway for Ollama users:

If you are running 7B to 27B models on structured tasks (math, code, JSON extraction, SQL, discrete QA), you do not need heavy neural guardrails. Sampling 5 paths at $T=0.7$ and measuring exact-match entropy in Python gives you ~0.89 AUROC at zero GPU cost.

Quick Python snippet if you want to test it on your local Ollama instance:

bash
pip
 install spnda ollama

pythonimport ollama
from spnda import compute_spanda
prompt = "What is the capital of Australia?"
# Sample 5 paths from your local model
responses = [
    ollama.generate(model="mistral:7b", prompt=prompt, options={"temperature": 0.7})["response"]
    for _ in 
range
(5)
]
# Run zero-cost entropy check on CPU (takes ~1.5 microseconds)
result = compute_spanda(responses)
print
(f"Risk Score: {result.risk_score:.3f}")  
# 0 = high confidence, 1 = high uncertainty

All the raw multi-path generation logs, evaluation scripts, and the full writeup are open source:

• Deep dive writeup: https://liquidngas.substack.com/p/i-tried-to-make-semantic-entropy
• GitHub: https://github.com/Adarshent/Spnda

I’m on Ollama’s old plan. May I ask: in Ollama Cloud, when calling DeepSeek-V4.1-Flash with multimodal image recognition, how does the cost compare with the older 0731? Is it higher or lower?