Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support for optimizers which don't have "fused" parameter such as grokadamw and 8bit bnb #1744

Merged
merged 12 commits into from
Sep 26, 2024
Merged

Support for optimizers which don't have "fused" parameter such as grokadamw and 8bit bnb #1744

merged 12 commits into from
Sep 26, 2024

Commits on Sep 25, 2024

  1. check if optimizer class has "fused" parameter in litgpt.utils.instan… 

    …tiate_torch_optimizer
    @mtasic85
    mtasic85 committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    8e1a096 View commit details
    Browse the repository at this point in the history

  2. refactor instantiate_torch_optimizer

    @mtasic85
    mtasic85 committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    4938578 View commit details
    Browse the repository at this point in the history

  3. support for "AdamW" and "torch.optim.AdamW" as string values for "opt… 

    …imizer"
    @mtasic85
    mtasic85 committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    b63091a View commit details
    Browse the repository at this point in the history

  4. Update litgpt/utils.py 

    Co-authored-by: Sebastian Raschka <[email protected]>
    @mtasic85 @rasbt
    mtasic85 and rasbt authored Sep 25, 2024
    Configuration menu
    Copy the full SHA
    2269828 View commit details
    Browse the repository at this point in the history

  5. Update litgpt/utils.py 

    Co-authored-by: Sebastian Raschka <[email protected]>
    @mtasic85 @rasbt
    mtasic85 and rasbt authored Sep 25, 2024
    Configuration menu
    Copy the full SHA
    4c064a6 View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2024

  1. Merge branch 'Lightning-AI:main' into main

    @mtasic85
    mtasic85 authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    0654018 View commit details
    Browse the repository at this point in the history

  2. fix instantiate_torch_optimizer "dict(kwargs).items()"

    @mtasic85
    mtasic85 committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    e1bcf53 View commit details
    Browse the repository at this point in the history

  3. add optimizer arg test

    @rasbt
    rasbt committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    449b4fb View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into main

    @rasbt
    rasbt authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    5c7c662 View commit details
    Browse the repository at this point in the history

  5. Update test_pretrain.py

    @rasbt
    rasbt authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    5bf407a View commit details
    Browse the repository at this point in the history

  6. Update litgpt/utils.py

    @rasbt
    rasbt authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    70b0f4e View commit details
    Browse the repository at this point in the history

  7. Update litgpt/utils.py

    @rasbt
    rasbt authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    7bf3ad1 View commit details
    Browse the repository at this point in the history