IBM Granite 4.0 Tiny Preview SLM Has Been Released

Posted on Edited on

Quick Report

IBM has announced its preliminary version of Granite 4.0 Tiny, which is a 1B parameter model that is 10x smaller than the previous version. The new model is designed to be more efficient and faster than the previous version, and is available for download on the IBM website.

The main highlights are longer context window size of 128K at FP8 precision and can run on many consumer hardware with minimal resources which is perfect for offline inference with decent accuracy thanks to its new hybrid approach with Mamba-2/Transformer model marrying speed and efficiency capable of running on wide variety of hardware.

Although the model is still in preview, it is already available for download on the Hugging Face and the best thing is that it is free to use with Apache-2 license. LMStudio and Ollama will add official support in summer later this year.

More info can be found here

Source(s)

  • TPU