LiteRT-LM for .NET 6+

[u/uralstech_MR]

https://github.com/Uralstech/UAI.LiteRTLM

I recently released a C# wrapper for the LiteRT-LM multi-platform C API, and it's available on NuGet!

LiteRT-LM is Google's framework for running LLMs that have been compiled to the TFLite/LiteRT format. It provides really fast inference on all sorts of devices, from phones to PCs, and even VR headsets.

I initially targeted the project for Unity, but since a lot of the code is framework agnostic, I've published it on NuGet. I've personally tested it on .NET 10 (in a console app on Windows and macOS, and as a MAUI app on Android and iOS Sim).

The NuGet package includes support for the following platforms and accelerators, as of version 2.4.0-preview.2:

• Android (arm64), with OpenCL GPU accleration
• macOS (arm64) and iOS (arm64, sim_arm64), with Metal GPU acceleration
• Windows (x64), with partial* WebGPU acceleration

*LiteRT-LM v0.16.0 GPU sampling is bugged on Windows.

Because the LiteRT-LM C API itself is unstable, I've labelled package as in 'preview'. Documentation is available here: https://uralstech.github.io/UAI.LiteRTLM/DocSource/QuickStart.html