v0.5.1

What's Changed

• v0.5.0 post release setup by @rasbt in #1774
• Be more specific about missign RoPE parameters by @rasbt in #1781
• Use correct Llama 3.1 and 3.2 context lengths by @rasbt in #1779
• Fixing Llama 3.1 and 3.2 Maximum Context Length by @rasbt in #1782
• Use more realistic RoPE tests by @rasbt in #1785
• AMD (MI250X) support by @TensorTemplar in #1775
• Tidy up RoPE by @rasbt in #1786
• Bump version for 0.5.1 bugfix release by @rasbt in #1787

New Contributors

• @TensorTemplar made their first contribution in #1775

Full Changelog: v0.5.0...v0.5.1

v0.5.0

What's Changed

• Post 0.4.13 release set up by @rasbt in #1755
• Add missing explanation on how to use a finetuned model by @rasbt in #1756
• Bump lightning version to latest stable release (2.4.0) by @rasbt in #1765
• Improve rope by @rasbt in #1745
• Add bnb.nn.StableEmbedding for quantized training by @rasbt in #1770
• [fix][1760] Added fix for the missing context key issue in dolly! by @pytholic in #1766
• Fix Llama 3.2 tokenizer by @rasbt in #1772

New Contributors

• @pytholic made their first contribution in #1766

Full Changelog: v0.4.13...v0.5.0

v0.4.13

What's Changed

• Make 0.4.13.dev1 version by @rasbt in #1722
• Enable MPS support for LitGPT by @rasbt in #1724
• Simplify MPS support by @rasbt in #1726
• Add Chainlit Studio by @rasbt in #1728
• Fixing the tokenizer for slimpajama data preparation by @tomaslaz in #1734
• Add pretrain conversion by @rasbt in #1735
• Typo fix and formatting improvements in API Trainer docs by @rasbt in #1736
• bump macos to m1 by @t-vi in #1725
• Improve filepath handling in unit tests by @rasbt in #1737
• Add a more informative message in case text exceeds context size by @rasbt in #1738
• Update Thunder README.md by @rasbt in #1740
• Add sliding window attention to Mistral and Phi 3 by @rasbt in #1741
• Extend context length for sliding window tests by @rasbt in #1742
• Fix jsonarparse version by @rasbt in #1748
• Update RoPE tests by @rasbt in #1746
• Make json parsing more robust by @rasbt in #1749
• Support for optimizers which don't have "fused" parameter such as grokadamw and 8bit bnb by @mtasic85 in #1744
• Increase rtol and atol in Gemma 2 for macOS by @rasbt in #1751
• Repair json files by @rasbt in #1752
• Llama 3.2 weights by @rasbt in #1750
• Bump version to 0.4.13 for new release by @rasbt in #1753
• Temporarily take out thunder dependency for deployment by @rasbt in #1754

New Contributors

• @tomaslaz made their first contribution in #1734
• @mtasic85 made their first contribution in #1744

Full Changelog: v0.4.12...v0.4.13

v0.4.12

What's Changed

• Preserve eos in encoding when max_seq_length = -1 by @sanderland in #1694
• Add batched_next_token() and batched_sample() by @apaz-cli in #1693
• bump thunder dependency to main by @t-vi in #1701
• add support for batched input_pos to model by @t-vi in #1700
• Add batched_generate_fn() by @apaz-cli in #1702
• minor Readme update/typos by @Borda in #1710
• Fix device Error in Decode Stream by @Motsepe-Jr in #1707
• Temporarily remove MPS device support by @rasbt in #1718
• Add uvloop as a dependency by @rasbt in #1719
• Change default precision on macOS by @rasbt in #1720
• Bump for v0.4.12 by @rasbt in #1721

New Contributors

• @Motsepe-Jr made their first contribution in #1707

Full Changelog: v0.4.11...v0.4.12

v0.4.11

What's Changed

• Add distribute=None to python-api.md by @rasbt in #1676
• Make LitGPT LLM API compatible with PyTorch Lightning Trainer 1/2 by @rasbt in #1667
• Auto device handling in LLM API by @rasbt in #1677
• Fix KV cache issue in LLM API by @rasbt in #1678
• Improved benchmark utils by @rasbt in #1679
• Add PR benchmark util for internal use by @rasbt in #1680
• Added git hash to benchmark utility. by @apaz-cli in #1681
• Spelling fix by @rasbt in #1685
• Add Microsoft Phi 3.5 checkpoint by @rasbt in #1687
• Update check_nvlink_connectivity by @sanderland in #1684
• Make number of generated tokens consistent with CLI by @rasbt in #1690
• Avoid error when executing benchmark util outside a git folder by @rasbt in #1691
• Combine generate() functions by @apaz-cli in #1675
• Bumb version to 0.4.11 by @rasbt in #1695
• Fix falcon prompt template by @rasbt in #1696

New Contributors

• @sanderland made their first contribution in #1684

Full Changelog: v0.4.10...v0.4.11

v0.4.10

What's Changed

• Support Tensor Parallel in Python API by @rasbt in #1661
• Swap old Llama model with Phi-3 by @rasbt in #1666
• Update azure-gpu-test.yml by @rasbt in #1669
• Support the refactored API in litgpt serve by @rasbt in #1668
• Multi-gpu serving by @rasbt in #1670
• Add Mistral Large 123B by @rasbt in #1673
• Bumb version to 0.4.10 for next release by @rasbt in #1674

Full Changelog: v0.4.9...v0.4.10

v0.4.9

What's Changed

• Update LitServe version and tests by @rasbt in #1654
• Support for using large models in the Python API via sequential generation by @rasbt in #1637
• Add a PyTorch Lightning example by @rasbt in #1656
• Refactor Python API to introduce new distribute method (part of a larger refactor for PTL support) by @rasbt in #1657
• Fix some issues with circular and relative imports by @rasbt in #1658
• Optionally return benchmark info in Python API by @rasbt in #1660
• Bumb version for 0.4.9 release by @rasbt in #1664

Full Changelog: v0.4.8...v0.4.9

v0.4.8

What's Changed

• Adds unit test to test for parity between streaming and non-streaming API by @rasbt in #1650
• Add Gemma 2 2B by @rasbt in #1651
• Pin litserve version by @rasbt in #1652
• Version bumb for Gemma 2 2B release by @rasbt in #1653

Full Changelog: v0.4.7...v0.4.8

v0.4.7

What's Changed

• Apply prompt style for tp.py and sequentially.py by @Andrei-Aksionov in #1629
• Fix prompt docstring in Python API by @rasbt in #1635
• Update windows cpu-tests.yml by @rasbt in #1630
• Remove NumPy < 2.0 pin by @rasbt in #1631
• Fix kv-cache issue in Python API streaming mode by @rasbt in #1633
• Updates installation requirements to install minimal required packages for basic use by @rasbt in #1634
• Faster safetensors conversion when downloading model by @awaelchli in #1624
• Add Sebastian as code owner by @awaelchli in #1641
• Add missing super() call in data modules by @awaelchli in #1639
• Update Lightning version to 2.4.0 pre by @awaelchli in #1640
• Add tunable kvcache with error handling for nonsense inputs. by @apaz-cli in #1636
• Use Python API in serve code by @rasbt in #1644
• Fix autodownload + conversion issue by @rasbt in #1645
• Properly clear kv-cache by @rasbt in #1647
• Fix error raising where max_returned_tokens > max_seq_length_setting by @rasbt in #1648
• Add quantization support to litgpt serve by @rasbt in #1646
• Bump for 0.4.7 release by @rasbt in #1649

Full Changelog: v0.4.6...v0.4.7

v0.4.6

What's Changed

• Change default top_k to 50 everywhere for consistency by @rasbt in #1592
• Fix kv-cache clearing in Python API and Serve by @rasbt in #1596
• dynamic KV Cache batching by @aniketmaurya in #1600
• Remove non-used eos_id in Python API by @rasbt in #1594
• Add quantization test and revert lightning version by @rasbt in #1605
• Dynamically set kv-cache size in serve by @rasbt in #1602
• Update LitData version and restore previous LitData assertions in tests by @awaelchli in #1609
• Gemma 2: 9b and 27b versions by @Andrei-Aksionov in #1545
• Update config hub table qlora sections by @rasbt in #1611
• max_returned_tokens -> max_new_tokens by @rasbt in #1612
• Add warning about pretrain preprocessing by @rasbt in #1618
• Print warning about unsupported repo_ids by @rasbt in #1617
• Restore capability to load alternative weights by @rasbt in #1620
• Enable unbalanced number of layers in sequential generation by @awaelchli in #1623
• Llama 3.1 8B and 70B checkpoints by @rasbt in #1619
• Add Llama 3.1 405B config by @awaelchli in #1622
• Bumb version to 0.4.6 for next release (Gemma 2 and Llama 3.1) by @rasbt in #1626

Full Changelog: v0.4.5...v0.4.6