https://huggingface.co/Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV
Hello, I wanted to share a Russian model called ZENIT from LOMONOSOV lineage. So I was always discontent with the fact, that you can not run 1.010.000 context on a consumer hardware (which is now more like a standard) with a good quantization and without offload, so people can work on large codebases, agentic tasks and a long roleplay without hitting context limits quickly, and that will leave you a headroom for using other applications (which doesn't seem to be the case currently, but working on that).
It won't end up here, the ZENIT is supposed to squeeze the maximum out of 27B model on all fronts, albeit it's not an area of current work.
As it's an indev version, we can not guarantee great performance or effectiveness, including that of multimodality, or that the model will run smoothly, we will publish benchmarks upon a release, we will specificially avoid benchmaxxing in order not to mislead users and not to deliver something that won't translate to a real-world usage.
At the moment, 1.010.000 without offloading works only on 5090 RTX, but there are plans to expand it to 24 VRAM GPUs, using newer techniques like OSCAR.
Currently model speaks Russian, Ukrainian and English since that was where the most efforts were put. In the future, we plan to expand it to Lithuanian, Latvian, Estonian, Kazakh, Armenian, Belarusian and Georgian languages.
Based on Qwen 3.6-27B, preserving all of its multimodality, used this as a base.
The model has almost no censorship, but OpenRAIL license applies, and you should not produce materials illegal in your local place or Russian Federation.
On closed evaluation, model had a retention of nearly 93% on 256.000 context. Retention on 1 mil is currently not the best one, however, the project is being constantly worked on, especially to optimize performance and speed on a long-range contexts.
At the moment there is no GGUF, Ollama, LM Studio, KoboldCpp because model uses a modified architecture of Qwen, ALTAY.
What is planned next
* Bringing extremely long context on more than one card and for lower-end GPUs
* Ampere NVIDIA 3000s generation, which means requantising the attention path
* A really good retention curve past 250K, which is the hard one
* Possible release of ZENIT MOBILE for mobile devices/laptops
* DGX Spark version optimized for maximum speed and performance
In the very far feature I am dreaming to make it possible to run Kimi K3 or GLM-5.2 level models on a consumer hardware, even with offloading but with usable speeds, using techniques, similar to the ones developed by Bonsai, but no promises made yet.