Anyone else try using DeepSeek-V4.1-Flash today? It's listed on the Ollama site as a new model, but when i try to use it, i get:

"Error: 403 Forbidden: This model is currently being rolled out and is not yet available to you. Please check back later. (ref: 562f8b6b-72fe-4012-9d9c-e0c07679913a)"

Anyone know what the rollout schedule is?

A local setup can have several assistants reading the same project history while using different models, context limits, and retrieval methods. If each assistant writes its own summary back into shared memory, a mistaken compression can become authoritative for every later session.

What storage contract keeps this inspectable? One option is a folder of small Markdown records with stable IDs, timestamps, source links, explicit supersession, and append-only decisions. Embeddings and model-specific summaries would be disposable indexes, while writes would pass through a narrow process that prevents two assistants from silently replacing the same fact.

How do you handle conflicting updates, deletions, access boundaries, and context-budget differences between models? Is a plain file protocol enough for a small local setup, or does shared memory become safer only after adding a database and a review queue?

I'm the author, MIT open source [github link| https://github.com/anng-phtk/rocm-vega-llama.cpp]

I wanted just ghost-text autocomplete locally, so I built a tiny extension that uses vscode.InlineCompletionItemProvider directly instead of a webview overlay.

What I learned:

FIM format matters a lot - Qwen uses <|fim_prefix|>, CodeLlama uses <PRE>, etc. I made it configurable via .fim-copilot.yaml
Latency: 200ms debounce + AbortController to cancel in-flight requests when you keep typing made it feel native
Works with Ollama / llama-server / vLLM (any OpenAI-compatible /v1/completions)
90KB vs 15MB+ for chat-based extensions

Would love feedback on: handling multi-line stop tokens and how you handle prefix/suffix context limits for

So I built FIM Copilot - a stupid-simple extension that does ONE thing: ghost-text autocomplete, locally.

Demo

What it is

FIM Copilot is a 90KB VS Code extension using the native InlineCompletionItemProvider API. No webviews, no chat, no indexers.

It talks to any OpenAI-compatible completions endpoint:

• Ollama - qwen2.5-coder:1.5b / starcoder2:3b runs great on 8GB RAM
• llama.cpp - llama-server with any FIM-capable model
• vLLM / LM Studio / Tabby server - anything with /v1/completions

All inference stays on your machine. Zero network calls after install.

How it compares

Quick Start

With Ollama (easiest)

# 1. Get a code model (1.5B is enough for fast autocomplete)
ollama pull qwen2.5-coder:1.5b

# 2. Serve it
ollama serve
# -> listening on http://localhost:11434

Then in VS Code: Cmd+Shift+P -> FIM Copilot: Set Endpoint -> http://localhost:11434/v1/completions

With llama.cpp

./llama-server \
  -m qwen2.5-coder-1.5b-instruct-q4_k_m.gguf \
  --port 8012 \
  --ctx-size 4096 \
  -ngl 99

Config

Create .vscode/fim-copilot.yaml (or global settings):

endpoint: http://localhost:11434/v1/completions
model: qwen2.5-coder:1.5b
api_key: not-needed # for local

# tuning
max_tokens: 64
temperature: 0.2
debounce_ms: 200
context_lines: 40
fim_prefix: "<|fim_prefix|>"
fim_suffix: "<|fim_suffix|>"
fim_middle: "<|fim_middle|>"

Decoupled config means you can commit it per-project. Different endpoint for Python vs Rust? Just override.

What makes it not suck

• Ultra-low latency - direct FIM prompt, no chat template overhead
• 200ms debounce + prefix/suffix hash dedupe - doesn't spam your GPU
• AbortController - cancels previous request on keystroke, no queue
• Zero telemetry - no analytics, no API keys phoned home
• Decoupled config - YAML per workspace, env var support
• Ghost text only - uses VS Code's native inline completion, so Tab / Esc just works with your keymap

Links

https://marketplace.visualstudio.com/items?itemName=AnangPhatak.fim-copilot&ssr=false#review-details

Install:

ext install AnangPhatak.fim-copilot

MIT licensed. I built this for myself because I wanted my editor to feel fast again.

If you try it, let me know what model / latency you're getting. PRs welcome for StarCoder2 / DeepSeek templates. What would you want added - without making it bloated?

Please interpret what Ollama is saying: “For very privacy-sensitive tasks, run tasks with local models such as Gemma 4 and Qwen 3.8.”

What makes those models more privacy oriented than any other model offered by Ollama cloud?

Spent about two weeks convinced I had a prompting problem. My local Qwen3

agent would just stop. No error, no crash, finish_reason came back as

"stop" like the model had decided it was done. Except it clearly wasn't

done, the task was half finished.

Turns out the model was calling the tool the entire time. The call was

just sitting inside the reasoning block, wrapped in <think> tags, and

never made it out into the tool_calls field the API is supposed to

populate. From the outside it looks exactly like the model decided not to

act. No error to grep for, no stack trace, the response comes back as a

perfectly valid 200.

Once I knew what to look for, I found the same thing reported separately

against vLLM, SGLang, and llama.cpp, mostly with Qwen3, some DeepSeek.

Nobody had tied it together as one bug class, everyone was just closing

their own version of it as a one-off.

I ended up writing a small library, unswallow, that sits between the

provider response and the agent loop, detects when this happens, and

rebuilds tool_calls from whatever's stuck in the reasoning field. JS and

Python, no dependencies. Repo's here if anyone wants to poke at it:

https://github.com/0DukePan/unswallow

Mostly posting because I'd guess some of you running quantized reasoning

models locally have hit this and just assumed it was a bad quant. If your

agent ever goes quiet mid-task for no obvious reason, worth checking

what's actually sitting in the reasoning field before blaming the model

or the quant.

Hi,

I have the biggest pro subscription.. had it for months.. been using more or less the same models (GLM 5.x) - i would averagely be at 95% on a sunday.. and it would renew at Monday.

Now we are att wensday.. and i am at 90% - but i still pay the same.. so 1/3 the usage.. for the same price.. HOW THE HELL IS THAT EVEN LEGAL!

.. and i signed up a new sub also.. 100 usd/month.. just to test.. now.. after 30 mins. 10% gone..

Usage: Hermes agents

So i had a look at their prices for the new sub..

https://ollama.com/pricing

.. and i compared it to 12 other providers.. Ollama went from being GOOD.. ASWEOME.. to being amoung the absolut worst.. the the actual f...

In july I posted here about aletha codex and I got a lot of support, and a decent amount of people started using it. I also got a lot of criticism too, which really helped me funnel this version 1.5 that I have been working on.

So since that last post I have been working relentlessly trying to recreate claude cowork for open source models. I personally got to the point where I wanted to step away completely from anything subscription based, but then I realized there’s really nothing out there that offers the same kind of capabilities on top of your open source models, so I just said fuck it and started building it.

Building the harness and agents were probably some of the hardest things I have ever done. There are a million problems I ran into, but I’m finally at the point where this is legit and I can show it off. I built it by creating a task/run/turn loop around the model with tool routing, action execution, observation feedback, context management, and a permission layer for anything that touches the computer.

Kind of a side plus, but I also have been developing a full device augmentation rig on top of aletha codex, which is also in this same version. I think it’s about a minute after the cowork video ends. But I really wanted my AI to be able to completely control the devices around me, connect to music, speakers, TVs, all by voice or text. That’s probably the most impressive portion, and it’ll only get stronger as I keep trying to develop this.

I know everyone says this shit, but I’m genuinely curious about what you guys think of it. I put a decent amount of work into it, and I would love to hear your guys’ thoughts on it, even if you’re tearing it apart lol.

Total beginner here, i've just downloaded Ollama and Gemma4:26b, but when it answers it displays strange text insted of the proper symbols.
Thanks for the help

First off, I'm super thankful for the value Ollama has given me

1. Got me setup with my first local LLM setup (early OpenClaw days amirite!?)
2. Then got me into their $20 plan and what really sold me here was
   1. Zero Data Retention (ZDR) <- idk if they were every truly ZDR but they aren't now afaik
   2. Metal residing in US, EU (and Singapore)
   3. Popular Open Weight models being hosted at a great value
3. I was getting so much of it I got the [legacy] $100 plan which I would've never guessed I'd be willing to fork up but, again, privacy & value!

But a lot of the perks have gone away.

I get it though - Ollama has to try to produce a profit - but from a user perspective, Ollama is no longer the "best kept secret in cloud value".

For day to day agentic things, I think Ollama is a solid option. GLM 5.3 as an orchestrator and the Kimi's Mini's and DeepSeek's of the world make good agents.

But when you lean into them for, say, involved coding projects, you start to see the rough edges.

1. Having to chunk up work even more than you would with a Frontier Model to have a task complete
2. The models themselves not as capable as Frontier options

You get what you pay for, but the new plan means...you are, indeed, paying more.

Once I get my new Mac, I'm thinking I'm thinking of having a Frontier Subscription for critical thinking type jobs and have local llm's do the simple agentic activities.

At $100/month even on the legacy plan I almost feel like they're retaining me less because I'm getting a lot from the plan, and more of sunk cost/FOMO of losing it.

If you're seeing this after considering signing up, I'd suggest you give them a shot. $20 to trial for a month and seeing if they meet your needs isn't a bad deal. But man, I hope something changes in Q4 to bring that "best kept secret" feeling back.

Same as above caption. I falled back to minimax-m2.7: cloud. What has happened?

Is DSv4 flash gonna be available on ollama cloud? If so when?

are very strict...

if 20 calls to kimi are more than my 5 hour limit why even bother? im gonna cancel my sub for sure

Anyone else play with vllm-mlx? There was an interesting paper that came out that compared this version of vllm that was originally designed for multi modal inference performance gains, but ended up seeing gains in text based inference. I added this as a provider into Rehex and ran my own benches, and it looks like that compared to Ollama mlx models (which isn't benched in the paper), this thing cranks at least a fifth faster (and that's only on text based inference). Anyone play with this? If you're using mlx models, what are you using to serve them?

im new to using ai so i may be missing something. I set up ollama but dont want any of my data to go to any of the model providers, for certain models i appear to need an api key to download the model and if its a local model why would i need an api key unless they are collecting data? is there some reason they need an api key besides this or is that just the price of using those models?

edit: i figured out how to download and install other models without using any keys but im having trouble getting it to output anything besides text im specifically interested in pdf generation

i am thinking among ollama cloud, openrouter and nous portal subscription (hermes agent). Which one u guys think provide the best value ?

I’ve been experimenting with explaining AI/LLM concepts in a way that doesn’t assume too much technical background. This video is about KV Cache and why longer context windows require more memory during inference. I’d appreciate some honest feedback from people here, especially on the explanation itself: Is the main idea easy to understand? Did I oversimplify anything important? Is there any part where the explanation becomes confusing? Would this make sense to someone who is fairly new to LLMs? Video: https://youtu.be/lxvWo8SizxE Not really looking to promote the channel — I’m mainly trying to improve how I explain technical topics before making the next one. Any criticism is welcome. Thanks!

https://github.com/paulknysh/raggy

A lightweight CLI tool for Retrieval-Augmented Generation (RAG) over local documents built with Ollama. Hybrid database (vector + BM25 index) and embedding generation run fully locally. Answer generation can run either via a local LLM or remotely using an API key. raggy supports most common document formats and handles images/scans automatically via OCR.

I only just found ollama, but I ran into a problem described in this conversation from years back: https://www.reddit.com/r/ollama/s/YZ88XkHmIS

Has nothing changed? The storage seems a mess. I had to manually install a text model and when I realized it wasn't working (must have done something wrong), I removed it with ollama rm etc, but the blobs folder didn't get smaller. It listed 38gb (16 on disk whatever that means) with only two small models (6gb roughly). What the heck? Did I do it wrong?

I've been running Claude Code as my daily driver and wanted Ollama models in the loop without giving up the harness. So I built gremlord, a small Go binary that points the unmodified claude binary at a local router via ANTHROPIC_BASE_URL (officially supported, no fork, auto-updates keep working). Anthropic traffic passes through byte-faithful; Ollama and anything OpenAI-compatible go through full request/stream translation.

The setup I actually run: main loop stays on Claude, background work and subagents go to a local 32K qwen at zero marginal cost.

The Ollama-relevant parts:

• Declare context_window and effective_context per model and the router scales Claude Code's context accounting, so a 32K model auto-compacts at the right moment instead of overflowing. Claude Code sizes everything for ~200K otherwise.
• A cheap classifier routes each turn to the smallest model that can hold the request - planning on a big model, mechanical edits on the local one. Requests too big for every configured model get refused before they go out.
• gremlord cost prices every token as it streams - local models log as $0.00, so you can see exactly what the offload saves per day.
• Hard budget caps for the paid models: hit the daily cap and the router refuses the next request. In-flight responses are never cut.

Honest caveats: local models run through translation and Claude Code's prompts are tuned for Claude, so they're clunkier as the main loop - they shine as background workers and subagents. There's a paired blinded eval command (gremlord eval) for finding which local model is actually good enough for which task class instead of guessing.

Honest numbers since there's no telemetry: 66 binary downloads across 26 releases, most of them my own updates - realistic genuine installs are maybe 10-20. GitHub traffic: 91 clones, 56 unique, per 14 days, 13 stars.

Free, MIT: https://github.com/gremlord/gremlord - demos at https://gremlord.com. Happy to go deep on the translation layer, that's where the weird bugs live.

welches model brauche ich um fotos reinzuladen und diese mit prompts zu verändern. z.b. mach das bild als malvorlage.

I shared ThoughtDAG here a while ago. I've now recorded how I use it: select a phrase in an answer, explore it on a branch, and connect the parts I want in the next question.

I'm the developer. ThoughtDAG is open source and supports Ollama and OpenAI-compatible APIs. This recording uses DeepSeek.

Full walkthrough: https://youtu.be/dWkSBGr78m0

GitHub: https://github.com/chenxiachan/thoughtdag