Make load_state_dict use strict=False

[neelnanda-io]

Your library is too strict, which creates errors if loading state dicts that don't eg contain a scaling factor. I added a quick fix that won't raise errors when loading a state dict that only contains some parameters. This runs the risk that someone may accidentally load a state dict with too few parameters, which is bad? But seems pretty unlikely to me

[codecov]

Codecov Report

Attention: Patch coverage is 25.00000% with 3 lines in your changes are missing coverage. Please review.

Project coverage is 57.62%. Comparing base (6a056b7) to head (c22fbbd).

Files	Patch %	Lines
sae_lens/training/sparse_autoencoder.py	25.00%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #95      +/-   ##
==========================================
- Coverage   57.74%   57.62%   -0.13%     
==========================================
  Files          16       16              
  Lines        1394     1397       +3     
  Branches      227      228       +1     
==========================================
  Hits          805      805              
- Misses        543      545       +2     
- Partials       46       47       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[hijohnnylin]

Perhaps checking that the state dict’s keys is a superset of a set of required keys?

I recently made a very similar change to sae_vis here: https://github.com/callummcdougall/sae_vis/pull/39/files

[jbloomAus]

Hmm. So this is resulting from the decoder fine-tuning changes which added a new state_dict parameter to SAEs. This parameter doesn't effect the output if unused.

I think we should:

• explicitly check whether the extra weight is the scale parameter.
• deliberately allow strict = false if that's the only difference.