[pull] main from vllm-project:main #32
Conversation
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
|
Hi @pull[bot]. Thanks for your PR. I'm waiting for a opendatahub-io member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Pass the CUDA stream into the CUTLASS GEMMs, to avoid future issues with CUDA graphs
The 2nd PR for #4532. This PR supports loading FP8 kv-cache scaling factors from a FP8 checkpoint (with .kv_scale parameter).
Signed-off-by: Muralidhar Andoorveedu <[email protected]>
Co-authored-by: Varun Sundar Rabindranath <[email protected]> Co-authored-by: Varun Sundar Rabindranath <[email protected]>
Co-authored-by: Elisei Smirnov <[email protected]>
Co-authored-by: Michael Goin <[email protected]>
Co-authored-by: Cody Yu <[email protected]>
Co-authored-by: Lei Wen <[email protected]>
…-Small model (#4799) Co-authored-by: beagleski <[email protected]> Co-authored-by: bapatra <[email protected]> Co-authored-by: Barun Patra <[email protected]> Co-authored-by: Michael Goin <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: Alexey Kondratiev <[email protected]> Co-authored-by: Alexei-V-Ivanov-AMD <[email protected]> Co-authored-by: Alexei V. Ivanov <[email protected]> Co-authored-by: omkarkakarparthi <okakarpa>
Co-authored-by: Breno Faria <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
…red_metadata modifier (introduced with PTX 8.5) (#5136)
Co-authored-by: Zhuohan Li <[email protected]>
…e ::ordered_metadata modifier (introduced with PTX 8.5)" (#5149)
Co-authored-by: xuhao <[email protected]>
|
[APPROVALNOTIFIER] This PR is APPROVED Approval requirements bypassed by manually added approval. This pull-request has been approved by: pull[bot] The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Adds support for multi-lora adapters. Passing tests added over in this PR: https://github.ibm.com/ai-foundation/tgis-deploy-tests/pull/25/files --------- Signed-off-by: Joe Runde <[email protected]>
* add gaudi installation readme * readme writeup * Create README_GAUDI.md * Update README.md * Update README_GAUDI.md * Update README.md * Update readmes
Update linear.py
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )