Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/multiclass recall macro avg ignore index #2710

Open
wants to merge 22 commits into
base: master
Choose a base branch
from

Conversation

rittik9
Copy link
Contributor

@rittik9 rittik9 commented Sep 1, 2024

What does this PR do?

Fixes #2441

Details
  • Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

Did you have fun?

Yes

Issue:

  • The root of the problem seems to be that the ignore_index information is not being properly propagated to the final averaging step i.e. the _adjust_weights_safe_divide function doesn't know that which class should be ignored.

Solution:

  • To address this issue, I updated the code to ensure that the ignore_index information is preserved throughout the entire process, making sure it is correctly passed through all intermediate steps up to the final averaging stage i.e. _adjust_weights_safe_divide function .
  • Updated the _adjust_weights_safe_divide function to accept an additional ignore_index parameter, which is passed through the _precision_recall_reduce function, called in the compute method of the MulticlassRecall class. This change adjusts the weights in the _adjust_weights_safe_divide function, setting the weight of the ignored class to 0.

📚 Documentation preview 📚: https://torchmetrics--2710.org.readthedocs.build/en/2710/

Copy link
Member

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, can we add also test for this case...

@rittik9 rittik9 marked this pull request as draft September 2, 2024 19:12
@rittik9 rittik9 marked this pull request as ready for review September 2, 2024 22:15
@rittik9
Copy link
Contributor Author

rittik9 commented Sep 2, 2024

looks good, can we add also test for this case...

Sure

@rittik9 rittik9 requested a review from Borda September 4, 2024 11:26
Borda
Borda previously approved these changes Sep 4, 2024
@Borda Borda self-requested a review September 6, 2024 09:10
@Borda Borda dismissed their stale review September 6, 2024 09:10

pending on adding test

@Borda Borda added the bug / fix Something isn't working label Sep 6, 2024
@rittik9
Copy link
Contributor Author

rittik9 commented Sep 6, 2024

@Borda What do I have to modify?

@Borda
Copy link
Member

Borda commented Sep 11, 2024

@rittik9 mind checking the changed docstest values and whether it is correct?

@mergify mergify bot removed the has conflicts label Sep 16, 2024
@mergify mergify bot added the has conflicts label Oct 8, 2024
@mergify mergify bot removed the has conflicts label Oct 8, 2024
@mergify mergify bot removed the has conflicts label Oct 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug / fix Something isn't working topic: Classif
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incorrect result in computing MulticlassRecall macro average when ignore_index is specified
3 participants