How to make batch inference in pipeline Transformers #3614
Unanswered
khongtrunght
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I'm new to BentoML. When I read this example : https://github.com/bentoml/BentoML/tree/main/examples/inference_graph . I see that the model is not batched.
When I follow the tutorial Adaptive Batching, I do somthing like that:
But it not work and yield error when run, can anybody with experiments helps me?
Beta Was this translation helpful? Give feedback.
All reactions