AssertionError: ('Pointer shape torch.Size([256]) and array shape (64,) mismatched', torch.Size([256]), (64,))

[etetteh]

Getting the following error when converting my ckpt to huggingface's pytorch. I am using the same config file I used for the pretraining.

Traceback (most recent call last):
  File "/home/enoch/dl_repos/transformers/src/transformers/models/electra/convert_electra_original_tf_checkpoint_to_pytorch.py", line 78, in <module>
    args.tf_checkpoint_path, args.config_file, args.pytorch_dump_path, args.discriminator_or_generator
  File "/home/enoch/dl_repos/transformers/src/transformers/models/electra/convert_electra_original_tf_checkpoint_to_pytorch.py", line 43, in convert_tf_checkpoint_to_pytorch
    model, config, tf_checkpoint_path, discriminator_or_generator=discriminator_or_generator
  File "/home/enoch/dl_repos/transformers/src/transformers/models/electra/modeling_electra.py", line 140, in load_tf_weights_in_electra
    ), f"Pointer shape {pointer.shape} and array shape {array.shape} mismatched"
AssertionError: ('Pointer shape torch.Size([256]) and array shape (64,) mismatched', torch.Size([256]), (64,))

Also, coverting other ckpt does start at all except the 1M training step, which is also failing here

[hyunssong]

were you able to solve this issue?

[etetteh]

No, I wasn't able to resolve it

[stefan-it]

Could you please give more details e.g. config.json and the exact command for converting (discriminator or generator).

There's one known problem with small generator models (needs a config change).

[hyunssong]

I think it could be a problem of the config. I was experiencing the same problem, but it was because I was using the small model's config when I was converting the base model. Changed the config and works now.

[zeno17]

@stefan-it could you tell me what the known problem with the small generator entails? Does it have to do with the setting of:
self.generator_hidden_size = 0.25 # frac of discrim hidden size for gen

[stefan-it]

It was related to this configuration change (that is needed to convert the model correctly)

google-research/electra#94 (comment)