v0.1.25

Summary

• Added AnyMoE
• Added the Starcoder 2 model
• Vision interactive mode
• X-LoRA and LoRA support for Gemma 2
• Optimized local loading speed
• Release using cargo dist

MSRV

cargo msrv

MSRV of mistral.rs is 1.75.0.

What's Changed

• Add a vision interactive mode by @EricLBuehler in #468
• Add support for X-LoRA/LoRA to Gemma 2 by @EricLBuehler in #499
• Add adapter activation support for Gemma 2 LoRA by @EricLBuehler in #500
• Decrease MSRV to 1.75.0 by @EricLBuehler in #501
• Fix docs with preprocessor config by @EricLBuehler in #502
• Allow for optional model field in OpenAI requests by @EricLBuehler in #504
• Support streaming for completion by @EricLBuehler in #508
• Select the best device in examples by @EricLBuehler in #507
• Handle Metal error from bf16 autodtype selection by @EricLBuehler in #509
• Use chat messages for the rust examples and show T/s by @EricLBuehler in #510
• Avoid f64 to f32 cast in phirope for dtype on metal by @EricLBuehler in #512
• Handle errors in rust examples by @EricLBuehler in #513
• AnyMoE: Build an MoE model from anything, quickly by @EricLBuehler in #476
• chore: update paths.rs by @eltociear in #516
• Add AnyMoE support for vision models by @EricLBuehler in #515
• Support images in AnyMoE dataset by @EricLBuehler in #517
• Handle metal case for nonzero and bitwise ops by @EricLBuehler in #518
• Support saving and loading AnyMoE gating layer by @EricLBuehler in #519
• Implement the Starcoder 2 model architecture by @EricLBuehler in #522
• Docs for Starcoder 2 by @EricLBuehler in #524
• Starcoder2 docs by @EricLBuehler in #525
• Export loader for starcoder2 by @EricLBuehler in #526
• Support the new phi3 models by @EricLBuehler in #530
• Source the AnyMoE dataset from a structured JSON file by @EricLBuehler in #531
• Create an AnyMoE loss graph by @EricLBuehler in #532
• Add AnyMoE demo video by @EricLBuehler in #533
• Add judgment for whether model_id is a local path to speed up local model loading by @chenwanqq in #523
• Update docs by @EricLBuehler in #534
• release using cargo-dist by @kranurag7 in #480
• Bump version to 0.1.25 by @EricLBuehler in #535

New Contributors

• @kranurag7 made their first contribution in #480

Full Changelog: v0.1.24...v0.1.25

Download mistralrs-server 0.1.25

File	Platform	Checksum
mistralrs-server-aarch64-apple-darwin.tar.xz	Apple Silicon macOS	checksum
mistralrs-server-x86_64-apple-darwin.tar.xz	Intel macOS	checksum
mistralrs-server-x86_64-unknown-linux-gnu.tar.xz	x64 Linux	checksum