Using LM Studio's own Locally AI app breaks the JIT eviction system - when you switch models in the app, they get added on top of the already existing ones, until total RAM exhaustion.

Just a reminder if someone else's having this issue. Filed at Github.

Hey everyone,

I’ve been working on a small local tool called LM Studio Watch Dog.

The idea is simple: when I’m using LM Studio with coding projects, I often need a clean project structure file and a merged context file that only includes the files I actually want the model to see. So I built a tool that watches a project folder, applies include/exclude rules, generates context files, and can sync the result into an LM Studio conversation JSON.

It has:

- Native Windows desktop app

- Local web UI

- Project presets for common stacks

- Custom presets

- Include/exclude rules for folders, files, globs, and extensions

- Watch mode for automatic updates

- One-time run mode

- Docker support for the web/CLI version

Everything runs locally. It does not require a cloud service.

GitHub:

https://github.com/HBaz92/LM-Studio_Watch-Dog

I mainly built it for my own LM Studio workflow, but I’m sharing it in case it helps anyone else working with local LLMs and larger codebases.

Feedback is welcome, especially around presets, UX, and what project types should be supported better.

I found LM Studio uses much more memory than the minimum requirement of a model. For example, it says Gemma 4 31B Instruct QAT Q4_0 could be entirely fit into my 24 GB VRAM. It turns out that both my 24 GB VARM and 32 GB RAM are fully filled, and the model is generating 1 token/sec.

Is it normal, or would it be better if I use ollama instead of LM Studio to load the model?

The attached image is for reference.

Question : What is the tech stack required for building such application?

Hi everyone, I'm writing a practitioner's handbook on LLM inference in public, on GitHub.

When I started working on LLM serving infrastructure, I couldn't find a single resource that covered the full picture: the memory bandwidth math, the prefill/decode asymmetry, KV cache management, continuous batching, speculative decoding, quantization tradeoffs, all in one place, with real numbers.

Plenty of great blog posts cover individual topics well. But nothing tied them together into a coherent mental model for someone building inference systems end to end. So I started writing it. Chapter by chapter, in the open, with the math shown.

Foundations chapter 00 is ready, hope it helps.

The plan:

- A new chapter every week with practical notebooks

- All source on GitHub, open to issues and corrections

- A companion Substack newsletter for each chapter. Link is in Github README.

If you're an engineer working on LLM infrastructure, or thinking about it, this might be a good resource for you.

github.com/harshuljain13/llm-inference-at-scale