Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API improvement of use_softmax and zero_infinity #180

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

NKNaN
Copy link

@NKNaN NKNaN commented Mar 4, 2024

From a suggestion proposed by PaddlePaddle community, it is recommended to add use_softmax and zero_infinity options to warpctc, comparing to the Pytorch API of torch.nn.functional.ctc_loss(log_probs, targets, input_lengths, target_lengths, blank=0, reduction='mean', zero_infinity=False).

I made an attempt to add this two options in this PR, and the logic is shown the following:

  • use_softmax: The attribute of log_probs in Pytorch receives the output of logsoftmax operation. Therefore, it is suggested when the users want to use the output from logsoftmax operation as an input, we could omit the softmax operation in our code. The proposed improvement is when use_softmax=False, perform exponential operation on the input logits.

  • zero_infinity: It is hoped to zero the infinity cost and associated gradient when zero_infinity=True. So first to make infinity value of cost available, I omit the truncation process done to the logits. And then I do zero operation after the calculation of cost and grad for each batch of the input.

I also add the corresponding test for this two cases. (PS. The test of inf_test seems only to pass when the truncation processes are omited.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant