Commits on Oct 16, 2024

1. add GPU sync and numerical value tests (pytorch#2194) …

   Summary:
   
   **TLDR: Significant clean up of recmetrics tests and fixing long standing testing issues with particular metrics. Because RecMetrics is community sourced, the quality of metric and test implementations can have high variance which leads to SEVs later on due to the high surface area RecMetrics operates on.**
   
   Added GPU sync tests to simulate gathering metric states on to rank 0 and computing. Tests don't cover this case before, which has resulted in SEVs in the past as users aren't aware of how RecMetrics collects and computes metrics.
   
   Added numerical value tests, most metrics are do not have this which can result in issues down the line if metrics need to be changed/accommodate future changes. Also we've found inconsistencies sometimes from other methods, so always good to check here. We compare each metric to a reference implementation from literature to ensure the values are as expected.
   
   Fixed and cleaned up some tests, particularly enforcing a standard of quality that can be referenced for future metric implementations and tests.
   
   Reviewed By: henrylhtsang
   
   Differential Revision: D59173140

   

   iamzainhuda authored and facebook-github-bot committed Oct 16, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for 6c353a8
   • Browse repository at this point

   Copy the full SHA

   6c353a8 View commit details

   Browse the repository at this point in the history