Skip to content

0.1.0

Latest
Latest
Compare
Choose a tag to compare
@feifeibear feifeibear released this 19 Sep 08:07
· 34 commits to main since this release
0.1.0
c3e97f5
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Demonstrate Speculative Sampling using bloom 560m and 7b1 models.
Support KV Cache Optimization.
Only works for batch size as 1.

Assets 2
Loading