Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a Python script that runs LLM benchmark #406

Merged
merged 17 commits into from
Oct 5, 2023
Merged

Add a Python script that runs LLM benchmark #406

merged 17 commits into from
Oct 5, 2023

Commits on Oct 5, 2023

  1. Run PA using Python

    @nv-hwoo
    nv-hwoo committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    e944803 View commit details
    Browse the repository at this point in the history

  2. Formatting and clean up

    @nv-hwoo
    nv-hwoo committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    8339172 View commit details
    Browse the repository at this point in the history

  3. Run multiple inferences with different prompt lengths

    @nv-hwoo
    nv-hwoo committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    043e628 View commit details
    Browse the repository at this point in the history

  4. Address feedback

    @nv-hwoo
    nv-hwoo committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    f6568a6 View commit details
    Browse the repository at this point in the history

  5. Add prompt length command line option

    @nv-hwoo
    nv-hwoo committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    4f02034 View commit details
    Browse the repository at this point in the history

  6. Add max_tokens and ignore_eos parameters

    @nv-hwoo
    nv-hwoo committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    3ce918a View commit details
    Browse the repository at this point in the history

  7. Add token-token latency calculation

    @nv-hwoo
    nv-hwoo committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    8dcb1b0 View commit details
    Browse the repository at this point in the history

  8. Allow user to provide input data file

    @nv-hwoo
    nv-hwoo committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    c8e18ab View commit details
    Browse the repository at this point in the history

  9. Update llm.md

    @nv-hwoo
    nv-hwoo committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    bcf8dce View commit details
    Browse the repository at this point in the history

  10. Minor changes to llm.md

    @nv-hwoo
    nv-hwoo committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    6560780 View commit details
    Browse the repository at this point in the history

  11. Remove note about 23.09 release

    @nv-hwoo
    nv-hwoo committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    7d6aa59 View commit details
    Browse the repository at this point in the history

  12. Update sample output

    @nv-hwoo
    nv-hwoo committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    00e9484 View commit details
    Browse the repository at this point in the history

  13. Use python staistics.mean to compute avg

    @nv-hwoo
    nv-hwoo committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    88d8aab View commit details
    Browse the repository at this point in the history

  14. Address feedback

    @nv-hwoo
    nv-hwoo committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    b6d267d View commit details
    Browse the repository at this point in the history

  15. Address feedback

    @nv-hwoo
    nv-hwoo committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    02ba02d View commit details
    Browse the repository at this point in the history

  16. Excluded null response from T2T latency calculation

    @nv-hwoo
    nv-hwoo committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    c24edda View commit details
    Browse the repository at this point in the history

  17. Remove avg T2T latency sample output from prefill benchmark

    @nv-hwoo
    nv-hwoo committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    f3a69fc View commit details
    Browse the repository at this point in the history