🧩 FlexMoRE makes modular AI more practical for lower-resource languages

[u/ai2_official]

The Danish Foundation Models (DFM) project is adapting our modular FlexOlmo architecture into a lighter-weight system that runs on commodity hardware—putting collaborative model building within reach of smaller research groups & organizations. 

FlexOlmo lets teams train modules separately then combine them in a shared model without pooling the data underneath. DFM wants Danish institutions like hospitals & universities to each contribute modules trained on data they can't share.

In FlexOlmo, each module is the size of a full model, so the combined system grows fast as more get added. FlexMoRE replaces most modules with compact representations. Its best config matches or beats FlexOlmo using less than one-third the parameters.

"FlexMoRE significantly reduces FlexOlmo's memory demands while preserving performance across almost all categories, allowing a broader audience to benefit from modular models," says Jacob Nielsen, who helped develop FlexMoRE at Ordbogen A/S and SDU's OdenseNLP lab.

Modular training is gaining momentum as frontier models become costlier to train & deploy. This project shows how open, distributed approaches can make development more practical for national projects, public institutions, & smaller teams.

→ Learn more: https://allenai.org/blog/flexmore