Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[libshortfin] Initial implementation of LLM inference server. #181

Merged
merged 14 commits into from
Sep 24, 2024
Merged

[libshortfin] Initial implementation of LLM inference server. #181

merged 14 commits into from
Sep 24, 2024

Commits on Sep 24, 2024

  1. [WIP] LLM Server.

    @stellaraccident
    stellaraccident committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    7a0eb7d View commit details
    Browse the repository at this point in the history

  2. Properly load prefill request.

    @stellaraccident
    stellaraccident committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    f61c835 View commit details
    Browse the repository at this point in the history

  3. Return logits

    @stellaraccident
    stellaraccident committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    3959b5c View commit details
    Browse the repository at this point in the history

  4. Use argmax to get token

    @stellaraccident
    stellaraccident committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    a03658a View commit details
    Browse the repository at this point in the history

  5. Adapt to scope->fiber rename

    @stellaraccident
    stellaraccident committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    05ddb89 View commit details
    Browse the repository at this point in the history

  6. Server working.

    @stellaraccident
    stellaraccident committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    ef786d8 View commit details
    Browse the repository at this point in the history

  7. Move to shortfin_apps

    @stellaraccident
    stellaraccident committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    b6e565b View commit details
    Browse the repository at this point in the history

  8. Fix import name

    @stellaraccident
    stellaraccident committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    deeaf2e View commit details
    Browse the repository at this point in the history

  9. Fix package

    @stellaraccident
    stellaraccident committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    e8ee7b3 View commit details
    Browse the repository at this point in the history

  10. Fix test

    @stellaraccident
    stellaraccident committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    287e4a8 View commit details
    Browse the repository at this point in the history

  11. Add dep

    @stellaraccident
    stellaraccident committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    e9337ca View commit details
    Browse the repository at this point in the history

  12. Add lsan suppression

    @stellaraccident
    stellaraccident committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    1c30f9e View commit details
    Browse the repository at this point in the history

  13. Merge branch 'main' into shortfin_llm

    @stellaraccident
    stellaraccident authored Sep 24, 2024
    Configuration menu
    Copy the full SHA
    3cba367 View commit details
    Browse the repository at this point in the history

  14. Comments

    @stellaraccident
    stellaraccident committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    ee45a9d View commit details
    Browse the repository at this point in the history