-
Notifications
You must be signed in to change notification settings - Fork 57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
shrink the IN dim respect to the SIMD #36
base: main
Are you sure you want to change the base?
Conversation
change from |
This is a good change, does it impact speed? I was considering adding a rust gated feature that lets you do generic constant arithmetic, but this solution is probably better for now. |
i tried the |
sorry! I'll get this checked in. just made a bunch of other changes that I need to merge in. |
this proj is a great place to learn rust and llama and cuda(triton), very appreciated, hope to do something helpful to the proj |
Would love any contribution, I'm also learning Rust and Triton on the fly. What if we try this library? It seems pretty cool. |
Another idea would be to explore adding testing. Not sure how unit tests work in rust, but it would be nice to have these for small sizes. |
import of the |
I am happy to write some unit test after i carefully finish the reading source code |
the SIMD_8 is used in the method
matvec
ofQLinear
, so the input x with (B,IN) should transformed into[[Simd<f32, 8>; B]; IN/8]