0.4.3 release

What's Changed

• [docs][fix] Update example to use offload_model by @anj-s in #806
• Switch default branch from master to main by @tmarkstrum in #807
• [FairScale] Remove refs to "cpu_offload" in code comments by @rohan-varma in #814
• [chore] Remove deprecated THCudaCheck by @anj-s in #818
• [feat] layer memory tracking by @QuentinDuval in #808
• [chore] Add log for the new experimental memory tracker feature. by @anj-s in #819
• [chore] Update the PyTorch version that we run CPU tests with by @anj-s in #809
• [chore] Update the PyTorch version that we run benchmarks with. by @anj-s in #823
• Extend auto shard capabilities to work around torch.fx edge cases. by @EugenHotaj in #817
• [fix] Update golden data for account for the speed regression by @anj-s in #825
• [chore] Fix main breakage temporarily by relaxing constraints by @anj-s in #828
• Use correct node names for param counting in auto_shard. by @EugenHotaj in #830
• [chore] Update requirements file to reflect latest config by @anj-s in #832
• [fix]: Fixes an issue with pre_backward hook registering by @min-xu-ai in #833
• [feature] Skip creating the CPU grad tensor when training by @anj-s in #821
• [test] improve a test's coverage by @min-xu-ai in #798
• [fix] Decouple move_params_to_cpu from the mixed_precision. by @anj-s in #822
• [fix] fix test on main by @min-xu-ai in #835
• [feature] Add the low level SSD APIs by @anj-s in #829
• [feat] [FSDP]: add experimental support to shared weights by @min-xu-ai in #836
• update nightly torch and test the flaky test by @min-xu-ai in #837
• [chore] Fix broken main due to updated github URL requirements by @anj-s in #838
• [chore] Update Sphinx version in docs requirements file by @vtantia in #841
• [feat] experimental MEVO layer by @min-xu-ai in #840
• [feat] Gossip/SlowMo by @blefaudeux in #378
• [feature]Add support for SSD offload with FSDP for eval workloads by @anj-s in #839
• [chore] 0.4.2 release by @anupambhatnagar in #846
• CI config changes by @anupambhatnagar in #847
• Setup pre-commit github action and apply pre-commit to all files by @anupambhatnagar in #849
• Allow sharded grad scaler to cpu offload with FSDP by @anupambhatnagar in #831
• Update changelog, removed meta.yml and requirements cleanup by @anupambhatnagar in #853
• [feature] Add a OffloadConfig object to specify offloading params to disk. by @anj-s in #855
• [POC] Testing Manual dispatch by @anupambhatnagar in #859
• [fix] [MEVO]: make mevo work with eval and optim_state checkpointing by @min-xu-ai in #851
• [chore] 0.4.3 release by @min-xu-ai in #860

New Contributors

• @rohan-varma made their first contribution in #814
• @EugenHotaj made their first contribution in #817
• @vtantia made their first contribution in #841

Full Changelog: v0.4.1...v0.4.3

FairScale Release v0.4.2

[chore] 0.4.2 release

v0.4.1: [chore] 0.4.1 release (#803)

Released version 0.4.1 for FairScale.

v0.4.0: [chore] 0.4.0 release (#757)

Co-authored-by: Min Xu <[email protected]>

v0.3.9: [chore] 0.3.9 release (#750)

* [chore] 0.3.9 release

* update changelog

* address comments

Co-authored-by: Min Xu <[email protected]>

v0.3.8

[chore] 0.3.8 release

v0.3.7: [chore] 0.3.7 release (#686)

* [chore] 0.3.7 release

* fixed changelog

Co-authored-by: Min Xu <[email protected]>

v0.3.6: [chore] 0.3.6 release (#631)

* [chore] 0.3.6 release

* try redo the caches

Co-authored-by: Min Xu <[email protected]>

v0.3.5: [chore] 0.3.5 release (#616)

* [chore] 0.3.5 release

* address comment

Co-authored-by: Min Xu <[email protected]>

v0.3.4

[0.3.4] - 2021-04-13

Added

• FSDP: Add no broadcast optim state option (#560)

Fixed

• ShardedDDP: Properly handle .eval() mode (#587)
• ShardedDDP: Handle model being moved back to CPU prior to state consolidation (#573)
• FSDP: much faster state consolidation (#595)
• FSDP: Add gradient pre-divide to prevent overflow with large world sizes (#565)
• Offload: (experimental) Fix activation offloading to CPU (#588