Add nightly benchmark and trigger in keyword #740
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
|
@NicolasHug is there a setup script I need to run beforehand? https://github.com/pytorch/data/runs/7868534954?check_suite_focus=true#step:9:23 |
That only works on the internal AI cluster. I will message you more details. |
ejguan
reviewed
Aug 17, 2022
Comment on lines
+6
to
+7
| pull_request: | ||
|
|
There was a problem hiding this comment.
Note: If you want to tigger in keyword, you can rely on issue_comment
Reference: https://github.com/orgs/community/discussions/25389
NivekT
reviewed
Aug 17, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR runs Nicolas' benchmark suite nightly and stores the results in a github artifact so anyone on the team can inspect them. It's also possible to trigger the benchmark suite manually by typing
RUN_BENCHMARKin the PR description which is how this benchmark run was actually triggeredEventually we'd need to add more benchmarks, log them to the same directory, maybe do a dashboard and of course integrate it into an AWS infra but at least this way the team has a way of easily running benchmarks now without needing to manually spin up machine
h/t @xuzhao9