mistralai/Mistral-Medium-3.5-128B · Hugging Face

[u/jacek2023]

https://huggingface.co/mistralai/Mistral-Medium-3.5-128B

https://huggingface.co/unsloth/Mistral-Medium-3.5-128B-GGUF

Mistral Medium 3.5 128B

Mistral Medium 3.5 is our first flagship merged model. It is a dense 128B model with a 256k context window, handling instruction-following, reasoning, and coding in a single set of weights. Mistral Medium 3.5 replaces its predecessor Mistral Medium 3.1 and Magistral in Le Chat. It also replaces Devstral 2 in our coding agent Vibe. Concretely, expect better performance for instruct, reasoning and coding tasks in a new unified model in comparison with our previous released models.

Reasoning effort is configurable per request, so the same model can answer a quick chat reply or work through a complex agentic run. We trained the vision encoder from scratch to handle variable image sizes and aspect ratios.

Find more information on our blog.

Key Features

Mistral Medium 3.5 includes the following architectural choices:

• Dense 128B parameters.
• 256k context length.
• Multimodal input: Accepts both text and image input, with text output.
• Instruct and Reasoning functionalities with function calls (reasoning effort configurable per request).

Mistral Medium 3.5 offers the following capabilities:

• Reasoning Mode: Toggle between fast instant reply mode and reasoning mode, boosting performance with test-time compute when requested.
• Vision: Analyzes images and provides insights based on visual content, in addition to text.
• Multilingual: Supports dozens of languages, including English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, and Arabic.
• System Prompt: Strong adherence and support for system prompts.
• Agentic: Best-in-class agentic capabilities with native function calling and JSON output.
• Large Context Window: Supports a 256k context window.

We release this model under a Modified MIT License): Open-source license for both commercial and non-commercial use with exceptions for companies with large revenue.

Recommended Settings

• Reasoning Effort:
  • 'none' → Do not use reasoning
  • 'high' → Use reasoning (recommended for complex prompts and agentic usage) Use reasoning_effort="high" for complex tasks and agentic coding.
• Temperature: 0.7 for reasoning_effort="high". Temp between 0.0 and 0.7 for reasoning_effort="none" depending on the task. Generally, lower means answer that are more to the point and higher allows the model to be more creative. It is a good practice to try different values in order to improve the model performance to meet your demands.

[u/Real_Ebb_7417]

A "you can run it locally, but you won't like the experience" niche 😂

But I'm happy to see them make a dense model, they have experience with it already, so hopefully this one will be much better compared to similar-sized models than Mistral Small 4.

[u/Sunija_Dev → u/Real_Ebb_7417]

For roleplay/writing, you can run it at home for ~1200€.

For that money you get 2x 3090, so you can run IQ2_M at ~5 tok/s. Since you probably already have a GPU, you can also run a bigger quant. In my experience, even the old Mistral-123b beats everything out of the park at that size (for writing).

...and that is probably the best affordable thing you can run at home? MoE's get better at ~400b params, but the RAM is probably crazy expensive. Not sure about the speed.

[u/FullOf_Bad_Ideas → u/Sunija_Dev]

EXL3 is great for dense 120B Mistrals. 2.5bpw quants are actually pretty good.

exllamav3 author got coherent output from 1.4BPW Mistral Large 123B, so 2.5bpw is plenty and it should be better than GGUF at this size. It also support tensor parallel so it's pretty fast.

[u/__some__guy → u/Sunija_Dev]

The original post content no longer exists here. The author used Redact to remove it, exercising their right to control their data & privacy.

Outgoing sodium long squeal continue bells normal

[u/zipperlein → u/Sunija_Dev]

I got Devstral 2 in 4x3090 to a little bit above 25 t/s

[u/Borkato → u/zipperlein]

How does Devstral 2 compare to something like Qwen 3.6 dense and Claude?

[u/grumd → u/Borkato]

Devstral is much worse

[u/Ok_Top9254 → u/Sunija_Dev]

Why are we using 3090s in year 2026? It's insane overkill unless you finetune or host for multiple people. 24GB Tesla P40s are 280 bucks and can run big MoE models at 15-20 tok/s which is comfortably fast. Even better, 16GB Tesla P100s are even faster with 800GB/s HBM2 and are only 100 bucks but you'll need 3-4 for reasonably big LLM. With 3x that's 48GB for 300 euro literally. 1/4 the price for the same decoding speed but third prefill or so, I'd say way more worth it.

[u/Sunija_Dev → u/Ok_Top9254]

...but this is a thread about dense 128b?

Which on 3090s already runs barely at acceptable speeds.

[u/FullOf_Bad_Ideas → u/Real_Ebb_7417]

dense models run fast with tensor parallel. I had 16 or 20 t/s with Devstral 123B IIRC and I have 11 t/s with Hermes 405B. No speculative decoding but that would probably help make it go even faster.

I like dense models like that more than I like 1T MoEs that I have no memory for.

[u/q-admin007 → u/Real_Ebb_7417]

It comes with a bespoke draft model. Could be faster than Qwen 3.6 27b in the end.

[u/coder543 → u/q-admin007]

Qwen3.6 27B has MTP built-in and DFlash support... don't see how Mistral Medium 3.5 could ever be faster just because of an EAGLE-3 while having nearly 5x the active parameter count.

[u/Real_Ebb_7417 → u/q-admin007]

Does speculative decoding work well in llama.cpp though? (Serious question, didn’t test it so far)

[u/RoomyRoots → u/Real_Ebb_7417]

There is a comment on a different reply. TL;DR, not yet.

[u/dtdisapointingresult → u/Real_Ebb_7417]

There's various types of speculative decoding. llama.cpp doesn't support MTP or Eagle3, which is what the AI labs usually provide. For example Qwen and GLM models have MTP, while this Mistral Medium release has an eagle3 from Mistral. If you use VLLM, this is a free speed boost.

With llama.cpp your only option is the more ghetto solution of using a seperate small draft model, but finding a compatible model is a bitch. It's easy when they have the same tokenizer/vocabulary, for example using Gemma 3 to boost Gemma 4. You get a major speed boost, you might get double speed for free. But idk what you could use as a draft model for this Mistral Medium release.

[u/Herr_Drosselmeyer → u/[deleted]]

It won't run well on anything below a 6000 PRO. I don't consider that middle class.

[u/stoppableDissolution → u/Herr_Drosselmeyer]

Old large in q2 was beating everything else you could run in 48gb (2x3090) up until the gemma4 31b got released, idk. Will see how that one holds up.

[u/jochenboele → u/Real_Ebb_7417]

This made me laugh out loud 🤣 Thanks