Integrating Mistral.rs with Swiftide

[timonv]

Hey there,

Figured I'd give it another shot to integrate mistral.rs with Swiftide. Tested several text models and it works great. Kudos on the new v4 builder style api, it's very clean 👍

However, github actions is now running into limitations and kills the worker with a 143 (afaik this is usually due to high cpu).

I've been looking at cargo timings and tried some tuning, but no luck so far. Does mistral.rs have to do tuning to be able to run in GA? I couldn't find anything specific. The full PR is at bosun-ai/swiftide#386

Side note: Would it be possible to publish mistralrs as a crate?

[timonv]

@EricLBuehler With a bit of tuning builds are working reliably and the integration is pretty cool bosun-ai/swiftide#421. However, crates.io does not allow publishing crates that rely on unpublished crates. Is there a reason mistral.rs is not published on crates.io? release-plz makes the process rather painless.