Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[do not merge] ibm-dev REbuild ibm-20241008 #184

Open
wants to merge 57 commits into
base: main
Choose a base branch
from
Open

[do not merge] ibm-dev REbuild ibm-20241008 #184

wants to merge 57 commits into from

Conversation

fialhocoelho
Copy link

PR to generate a compiled image

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Oct 8, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: fialhocoelho
Once this PR has been reviewed and has the lgtm label, please assign rpancham for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Xaenalt pushed a commit that referenced this pull request Oct 14, 2024
@afierka-intel
Enable llama-405b - w/a for memory allocation error (#184)
691255b 
Work around for allocation error while loading llama-405b.
tlrmchlsmth and others added 9 commits October 15, 2024 00:24
@tlrmchlsmth
[Bugfix] Clean up some cruft in mamba.py (vllm-project#9343)
169b530
@stevegrubb
[Frontend] Clarify model_type error messages (vllm-project#9345)
44eaa5a
@mgoin
[Doc] Fix code formatting in spec_decode.rst (vllm-project#9348)
8e836d9
@hhzhang16
[Bugfix] Update InternVL input mapper to support image embeds (vllm-p…
55e081f 
…roject#9351)
@njhill
[BugFix] Fix chat API continuous usage stats (vllm-project#9357)
e9d517f
@gracehonv
pass ignore_eos parameter to all benchmark_serving calls (vllm-projec…
5d264f4 
…t#9349)
@mgoin @DarkLight1337
[Misc] Directly use compressed-tensors for checkpoint definitions (vl…
22f8a69 
…lm-project#8909)

Co-authored-by: DarkLight1337 <[email protected]>
@CatherineSue @njhill
[Bugfix] Fix vLLM UsageInfo and logprobs None AssertionError with emp…
ba30942 
…ty token_ids (vllm-project#9034)

Co-authored-by: Nick Hill <[email protected]>
@LucasWilkinson
[Bugfix][CI/Build] Fix CUDA 11.8 Build (vllm-project#9386)
717a5f8
mrsalehi and others added 7 commits October 16, 2024 04:56
sasha0552 and others added 27 commits October 17, 2024 15:06
@sasha0552
[Bugfix] Allow prefill of assistant response when using `mistral_comm…
5e443b5 
…on` (vllm-project#9446)
@WoosukKwon
[TPU] Call torch._sync(param) during weight loading (vllm-project#9437)
8e1cddc
@bigPYJ1151
[Hardware][CPU] compressed-tensor INT8 W8A8 AZP support (vllm-project…
5eda21e 
…#9344)
@KuntaiDu
[Core] Deprecating block manager v1 and make block manager v2 default (…
81ede99 
…vllm-project#8704)

Removing the block manager v1. This is the initial piece of prefix-caching-centric design. In order to achieve prefix-caching-centric design, we need to simplify the code path so that we only use v2 block manager (which has much higher performance on prefix caching).
@dtrifiro
[CI/Build] remove .github from .dockerignore, add dirty repo check (v…
a2c71c5 
…llm-project#9375)
@DarkLight1337
[Misc] Remove commit id file (vllm-project#9470)
7871659
@ProExpertProg
[torch.compile] Fine-grained CustomOp enabling mechanism (vllm-projec…
0f41fbe 
…t#9300)
@bnellnm
[Bugfix] Fix support for dimension like integers and ScalarType (vllm…
eca2c5f 
…-project#9299)
@wukaixingxp @Isotr0py
[Bugfix] Add random_seed to sample_hf_requests in benchmark_serving s…
d65049d 
…cript (vllm-project#9013)

Co-authored-by: Isotr0py <[email protected]>
@sasha0552
[Bugfix] Print warnings related to mistral_common tokenizer only on…
d615b5c 
…ce (vllm-project#9468)
@sssrijan-amazon
[Hardwware][Neuron] Simplify model load for transformers-neuronx libr…
bb76538 
…ary (vllm-project#9380)
@robertgshaw2-neuralmagic @afeldman-nm
@WoosukKwon @laishzh @maxdebayser @DarkLight1337
Support BERTModel (first encoder-only embedding model) (vllm-proj…
343f8e0 
…ect#9056)

Signed-off-by: Max de Bayser <[email protected]>
Signed-off-by: Max de Bayser <[email protected]>
Co-authored-by: Andrew Feldman <[email protected]>
Co-authored-by: afeldman-nm <[email protected]>
Co-authored-by: Woosuk Kwon <[email protected]>
Co-authored-by: laishzh <[email protected]>
Co-authored-by: Max de Bayser <[email protected]>
Co-authored-by: Max de Bayser <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
@dsikka
[BugFix] Stop silent failures on compressed-tensors parsing (vllm-pro…
48138a8 
…ject#9381)
@joerunde
[Bugfix][Core] Use torch.cuda.memory_stats() to profile peak memory u…
de4008e 
…sage (vllm-project#9352)

Signed-off-by: Joe Runde <[email protected]>
@blueyo0
[Qwen2.5] Support bnb quant for Qwen2.5 (vllm-project#9467)
154a8ae
@russellb
[CI/Build] Use commit hash references for github actions (vllm-projec…
944dd8e 
…t#9430)
@njhill
[BugFix] Typing fixes to RequestOutput.prompt and beam search (vllm-p…
1ffc8a7 
…roject#9473)
@tomeras91
[Frontend][Feature] Add jamba tool parser (vllm-project#9154)
d2b1bf5
@njhill
[BugFix] Fix and simplify completion API usage streaming (vllm-projec…
25aeb7d 
…t#9475)
@DarkLight1337
[CI/Build] Fix lint errors in mistral tokenizer (vllm-project#9504)
1bbbcc0
@tlrmchlsmth
[Bugfix] Fix offline_inference_with_prefix.py (vllm-project#9505)
ae8b633
@fialhocoelho
Merge branch 'main' of https://github.com/vllm-project/vllm into ibm-…
5c074b3 
…20241018

Signed-off-by: Jefferson Fialho <[email protected]>
@fialhocoelho
Squash 5733
682e622 
Signed-off-by: Jefferson Fialho <[email protected]>
@fialhocoelho
Squash 6357
a823383 
Signed-off-by: Jefferson Fialho <[email protected]>
@fialhocoelho
Squash 9027
1105122 
Signed-off-by: Jefferson Fialho <[email protected]>
@fialhocoelho
Squash 9387
05fe6cb 
Signed-off-by: Jefferson Fialho <[email protected]>
@fialhocoelho
using cuda-pr-209 as intermediate-image 🚀
440bbfa 
Signed-off-by: Jefferson Fialho <[email protected]>
@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Oct 18, 2024 •
edited
Loading

@fialhocoelho: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/smoke-test 440bbfa link true /test smoke-test

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rpancham rpancham Awaiting requested review from rpancham

@vaibhavjainwiz vaibhavjainwiz Awaiting requested review from vaibhavjainwiz

@njhill njhill Awaiting requested review from njhill njhill is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.