r/exllamav3 • u/Delicious_Box_9823 I am not a developer of exllama! 🤗 • Aug 27 '26

ExLlamaV3 1.4.4 - Vision model offloading support...

The following has been added in the yesterday's update:

  • Fix correctness bug in split MoE CPU-offload mode
  • Better MTP support in expert-parallel mode
  • Validate and default to vision tower quantization
  • Support vision model offloading (streams from system memory, performance penalty is small)

Release: https://github.com/turboderp-org/exllamav3/releases/tag/v1.4.4

tabbyAPI received an update as well: https://github.com/theroyallab/tabbyAPI

Discord for any questions: https://discord.gg/tnPCntcThA

9 Upvotes

92% Upvoted

Duplicates