Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEATURE] Update InferenceClient to use chat.client_completion instead of apply_chat_template #772

Closed
plaguss opened this issue Jul 4, 2024 · 1 comment
Closed

[FEATURE] Update InferenceClient to use chat.client_completion instead of apply_chat_template #772

plaguss opened this issue Jul 4, 2024 · 1 comment
Assignees
@gabrielmbmb
Labels
good first issue Good for newcomers improvement refactor
Milestone
1.3.0

Comments

@plaguss
Copy link
Contributor

plaguss commented Jul 4, 2024

Is your feature request related to a problem? Please describe.
TGI and huggingface_hub added support for chat completion, removing the need of using the tokenizer to format the prompt correctly.

Describe the solution you'd like
We can use client.chat_completion instead of apply_chat_template as documented here

Describe alternatives you've considered
Maintain as it is.

Additional context
Thanks @osanseviero for the hint.
@gabrielmbmb gabrielmbmb self-assigned this Jul 10, 2024
@gabrielmbmb gabrielmbmb added this to the 1.3.0 milestone Jul 10, 2024
@plaguss
Copy link
Contributor Author

plaguss commented Jul 18, 2024

Closed with #778

@plaguss plaguss closed this as completed Jul 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gabrielmbmb gabrielmbmb

Labels
good first issue Good for newcomers improvement refactor
Projects
None yet
Milestone
1.3.0
Development

No branches or pull requests
2 participants
@gabrielmbmb @plaguss