Using adapters in a bentofile

[QLutz]

I am following the tutorial for using OpenLLM with BentoML. I'd like to build a Bento using adapters. What should I put into my bentofile.yaml and in my service.py ? Should I consider them models in the Bento sense ?

[aarnphm]

We have plans to refactor the bentofile.yaml quite a bit. For the workflow using OpenLLM from BentoML I think you can probably just include the directory for the adapter under include.

include:
- ./adapter
- service.py

This will then include the adapter into $(b get bento-name:tag -o path)/src/adapter,

Then for openllm.LLM you can do:

LLM = openllm.LLM('model-id', adapter_map={'./adapter': 'default'})

note that openllm CLI --adapter-id (i.e: --adapter-id ./adapter:default --adapter-id hf-name/adapter-id:a1) will transpile to this adapter_map on the Python SDK

I haven't tested this integration from the BentoML POV yet, but will definitely include this into a test plan that I'm currently working on.