feat(framework): unsloth

[aarnphm]

follow the https://colab.research.google.com/drive/1Ys44kVvmeZtnICzWz0xgpRnrIOjZAuxp?usp=sharing#scrollTo=QmUBVEnvCDJv

Then one can save with bentoml.unsloth.build_bento

bentoml.unsloth.build_bento(model, tokenizer, chat_template="<chat_template_that_trained_with>", quantization_method="bitsandbytes")

If model is continued from a fine-tune checkpoint, then model_name should be passed:

bentoml.unsloth.build_bento(model, tokenizer, model_name="llama-3-continued-from-checkpoint", chat_template="<chat_template_that_trained_with>", quantization_method="bitsandbytes")

supports llama3, gemma, gemma2, mistral and most chat templates that is supported with fine-tune. Based on BentoVLLM and OpenLLM logics for template.

[github-actions]

Package	Statements
bentoml	55.97% (9595 / 17141)
Summary	55.97% (9595 / 17141)

[ssheng]

Could you please also create a documentation similar to this section of Unsloth? https://docs.unsloth.ai/tutorials/how-to-finetune-llama-3-and-export-to-ollama#id-13.-exporting-to-ollama

[aarnphm]

Could you please also create a documentation similar to this section of Unsloth? https://docs.unsloth.ai/tutorials/how-to-finetune-llama-3-and-export-to-ollama#id-13.-exporting-to-ollama

Should we submit this to unsloth?

[ssheng]

Should we submit this to unsloth?

We should but let's create a draft to review first.