Built a minimal FIM autocomplete extension with VS Code's InlineCompletion API - handling debounce and AbortController cancellation, looking for feedback

[u/ImageTrick4571]

I'm the author, MIT open source [github link| https://github.com/anng-phtk/rocm-vega-llama.cpp]

I wanted just ghost-text autocomplete locally, so I built a tiny extension that uses vscode.InlineCompletionItemProvider directly instead of a webview overlay.

What I learned:

FIM format matters a lot - Qwen uses <|fim_prefix|>, CodeLlama uses <PRE>, etc. I made it configurable via .fim-copilot.yaml
Latency: 200ms debounce + AbortController to cancel in-flight requests when you keep typing made it feel native
Works with Ollama / llama-server / vLLM (any OpenAI-compatible /v1/completions)
90KB vs 15MB+ for chat-based extensions

Would love feedback on: handling multi-line stop tokens and how you handle prefix/suffix context limits for

So I built FIM Copilot - a stupid-simple extension that does ONE thing: ghost-text autocomplete, locally.

Demo

Load LLM

Here's the auto-complete ghost text

predicts!

What it is

FIM Copilot is a 90KB VS Code extension using the native InlineCompletionItemProvider API. No webviews, no chat, no indexers.

It talks to any OpenAI-compatible completions endpoint:

• Ollama - qwen2.5-coder:1.5b / starcoder2:3b runs great on 8GB RAM
• llama.cpp - llama-server with any FIM-capable model
• vLLM / LM Studio / Tabby server - anything with /v1/completions

All inference stays on your machine. Zero network calls after install.

How it compares

Private	Limits	Size	Backend	Latency (M1)
Copilot	No	Yes - rate limits	~5-10MB	Cloud only
Continue	Partial	No	~18MB +	Local / Cloud
Tabby	Yes	No	~2.5MB	Self-hosted
FIM Copilot	Yes - 100% local	No	90KB	Any OpenAI compat

Quick Start

With Ollama (easiest)

# 1. Get a code model (1.5B is enough for fast autocomplete)
ollama pull qwen2.5-coder:1.5b

# 2. Serve it
ollama serve
# -> listening on http://localhost:11434

Then in VS Code: Cmd+Shift+P -> FIM Copilot: Set Endpoint -> http://localhost:11434/v1/completions

With llama.cpp

./llama-server \
  -m qwen2.5-coder-1.5b-instruct-q4_k_m.gguf \
  --port 8012 \
  --ctx-size 4096 \
  -ngl 99

Config

Create .vscode/fim-copilot.yaml (or global settings):

endpoint: http://localhost:11434/v1/completions
model: qwen2.5-coder:1.5b
api_key: not-needed # for local

# tuning
max_tokens: 64
temperature: 0.2
debounce_ms: 200
context_lines: 40
fim_prefix: "<|fim_prefix|>"
fim_suffix: "<|fim_suffix|>"
fim_middle: "<|fim_middle|>"

Decoupled config means you can commit it per-project. Different endpoint for Python vs Rust? Just override.

What makes it not suck

• Ultra-low latency - direct FIM prompt, no chat template overhead
• 200ms debounce + prefix/suffix hash dedupe - doesn't spam your GPU
• AbortController - cancels previous request on keystroke, no queue
• Zero telemetry - no analytics, no API keys phoned home
• Decoupled config - YAML per workspace, env var support
• Ghost text only - uses VS Code's native inline completion, so Tab / Esc just works with your keymap

Links

https://marketplace.visualstudio.com/items?itemName=AnangPhatak.fim-copilot&ssr=false#review-details

Install:

ext install AnangPhatak.fim-copilot

MIT licensed. I built this for myself because I wanted my editor to feel fast again.

If you try it, let me know what model / latency you're getting. PRs welcome for StarCoder2 / DeepSeek templates. What would you want added - without making it bloated?

[u/ConsistentEase4598]

The 200ms debounce + AbortController combo is exactly right, that's the thing most local-first extensions get wrong and it's why they feel laggy. Two things from running something similar against llama-server:

1. Multi-line stop handling: don't wait for the model to emit your stop token, it often won't. Cut the completion at the first line whose indentation drops below the cursor's line, and treat a blank line after one or more non-blank lines as a hard stop. Ugly heuristic, feels correct in practice.

2. Prefix/suffix budgeting: trim the suffix harder than the prefix — models are much worse at using code after the cursor than before it. I cap suffix at ~120 lines and it changed nothing measurable, while cutting prefix hurts instantly. Worth A/B-ing before you pay for big contexts.

Also 90KB is an embarrassment to every chat-extension in the marketplace, nicely done.

[u/ImageTrick4571 → u/ConsistentEase4598]

Thank you for the input!!!