How do I output the attention from the model?

[jkb-ag]

Hi,
When running dnabert2, I'm trying to output not only the CLS and hidden states but also the attentions.

I tried:

`db2_model = AutoModel.from_pretrained(dirname, output_attentions=True, trust_remote_code=True)

output = db2_model(input_ids)
`

but the output is only a tuple of length 2 (the hidden states and the CLS tensor).

Any help would be much appreciated!

Thanks!

[ZoeyXiang1202]

Hi,

I also have the same question. Could you please show where the finetuned model weight is saved?

Thanks a lot!