Skip to content

v0.1.5
Compare
Choose a tag to compare
@blefaudeux blefaudeux released this 03 Feb 22:20
v0.1.5
4401ced
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Added

  • Pytorch compatibility for OSS checkpoints (#310)
  • Elastic checkpoints for OSS, world size can vary in between save and loads (#310)
  • Tensor views for OSS bucketing, reduced CPU use (#300)
  • Bucket calls in ShardedDDP, for faster inter node communications (#327)
  • FlattenParamWrapper, which flattens module parameters into a single tensor seamlessly (#317)
  • AMPnet experimental support (#304)

Fixed

  • ShardedDDP properly handles device changes via .to() (#353)
  • Add a new interface for AdaScale, AdaScaleWrapper, which makes it compatible with OSS (#347)
Assets 2
Loading