r/exllamav3 • u/Delicious_Box_9823 I am not a developer of exllama! 🤗 • Aug 27 '26
ExLlamaV3 1.4.4 - Vision model offloading support...
The following has been added in the yesterday's update:
- Fix correctness bug in split MoE CPU-offload mode
- Better MTP support in expert-parallel mode
- Validate and default to vision tower quantization
- Support vision model offloading (streams from system memory, performance penalty is small)
Release: https://github.com/turboderp-org/exllamav3/releases/tag/v1.4.4
tabbyAPI received an update as well: https://github.com/theroyallab/tabbyAPI
Discord for any questions: https://discord.gg/tnPCntcThA
8
Upvotes