-
Notifications
You must be signed in to change notification settings - Fork 151
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TypeError: LlamaRotaryEmbedding.forward() got an unexpected keyword argument 'seq_len' when running VILA model inference #126
Comments
same |
Pretty sure this broke in 54c9706, as there the |
Same here, previously I was able to run inference on V100 with no problem, but now it is broken |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello,
I am trying to run the VILA model for inference, but I have encountered a couple of issues that I need help with.
(1)FlashAttention Issue: Initially, I faced a problem related to FlashAttention. After going through all relevant issues on Github, I managed to resolve this issue by modifying the relevant code (lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py ).
(2)TypeError Encountered: After addressing the FlashAttention issue, I encountered the following error during model inference:
TypeError: LlamaRotaryEmbedding.forward() got an unexpected keyword argument 'seq_len'
Could you please provide guidance on how to resolve this issue? Any help would be greatly appreciated!
The text was updated successfully, but these errors were encountered: