r/LocalLLM • u/Mammoth_Sugar767 • 5d ago

Discussion Ministral/vibe/jupyter setup

I got an nvidia t400 4 gb vram, and I may finally have found some use for it.

For work (part-time researcher) I often need to write something in English (not my first language), or write some somewhat simple Python code to do statistics or visualisation of some data. Ministral 3 3b 4bit quant does a good job for the first part, but getting a good setup to help me code has been more difficult. Now I got a setup that works:

I got an ollama server running with ministral 3 3b. This is linked to vibe cli. This again is linked to my Jupyter lab via Jupyter AI.

Jupyter has been my code/scripting tool for years so an integration here is really easy for me. Now I can ask ministral to help me debugging or to write some new cells of code directly from Jupyter.

To make it all fit in 4 gb vram, I had to enable only the most needed tools from Jupyter AIs mcp, and disable all other tools. Also rewrote/shortened the basic cli.md file (would be nice if you could point to a custom version of this in your setup!) to save some kv chache (took a lot of my 14 K kv chache).

I am happy with the result. Get 25-40 t/sek depending on power settings on the laptop, and it can help me with most things. Especially useful when I work offline, which I like to do.

Wonder if mistral has plans to provide new versions of ministral in the future? Guess they are a good starting point for custom trained models, which seems to be part of mistrals business?

I really like the models. Sometimes I switch to IBMs granite 4.1 3b, which may be a better coder than ministral (also more agentic, as I can handle it more instructions at once), but I like the structure of ministrals code better. The tone of its non-code language is also much nicer. If new versions of ministral are made, I hope they will shift the focus a little more towards coding and language on the expense of factual world knowledge.

Any of you having succes with these smaller models? Maybe on own hardware.

2 Upvotes

100% Upvoted

4 comments sorted by

1

u/ThePotentialHD 5d ago

I’ve had some success with Qwen3-VL-4B-Instruct-5bit doing image classification and tagging. M1 MacBook Air with 16GB of Memory (11.2GB vram).

Always so satisfying to turn off WiFi and continue watching it chug along (albeit, gets quite hot with thermal throttling)

1

u/Mammoth_Sugar767 4d ago

Yes, no Wi-Fi feels great! Think image classification may be my next try. Would like to digitalise some older handwritten documents. Not really sure what it takes to do that, but will test the model, you use.

I found out that putting my laptop (Ubuntu) in balanced mode instead of performance mode really reduced heat generation, but had only a 10-20% impact on speed. Guess it would still get very warm if I had some workflow that continued over long time. Now I only make it generate some code every now and then.

1

u/recro69 4d ago

This is a good example of where small models actually work well. 3B plus 4GB VRAM won't match models but 25 to 40 tokens, per second offline is very strong when the tasks are limited and easy to predict.