Help needed using Select Object #914
Unanswered
mbilalazeem
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi people, I was trying to use the select to structure my output from the OpenAI API, but I am having the error I mentioned below. I have googled Reddit and checked communities and reached to the conclusion that the Select object won't work with OpenAI API. My question is will it work with any API or with only the local models? I am not sure if my laptop is capable enough to have a hugging face transformer installed.
Error I am getting:
The model attempted to generate b'!\n\nThe' after the prompt
b'********: \nAnswer:'
, but that doesnot match the given grammar constraints! Since your model is a remote API that does not support full guidance
integration we cannot force the model to follow the grammar, only flag an error when it fails to match.
You can try to address this by improving the prompt, making your grammar more flexible, rerunning with
a non-zero temperature, or using a model that supports full guidance grammar constraints.
Beta Was this translation helpful? Give feedback.
All reactions