🔥 What's New: v0.18.0
- New Model Release (EmbeddingGemma 2): Shipped multimodal EmbeddingGemma 2 supporting text, vision, and audio embeddings with Matryoshka dimension truncation across Python, Kotlin, Swift, Web (JavaScript), and C++. Check out our DevSite documentation for more details!
- CLI & Developer Experience: Added fast model imports (
litert-lm import) and an OpenAI-compatible/v1/embeddingsendpoint (litert-lm serve) for multimodal embedding models. - Model Info API: Added
ModelInfoandlitert-lm describefor full model introspection—covering metadata, capabilities, and runtime requirements before loading. - NPU & GPU Acceleration: Enabled dynamic on-demand KV cache growth on NPU and attention mask pruning optimizations on GPU.