Where is the sequence parameter in SimPo_loss funciton

[yunyiliu]

for the code https://github.com/princeton-nlp/SimPO/blob/main/scripts/simpo_trainer.py#L560
I have a question where is the parameter that represents the sentence sequence length, which is used to limit the text length mentioned in the paper?

[yumeng5]

Hi @yunyiliu

The length normalization is computed here:

SimPO/scripts/simpo_trainer.py

Line 681 in 0aaca09

return (per_token_logps * loss_mask).sum(-1) / loss_mask.sum(-1)

Best,
Yu