[tuner] Use JSON for benchmark output #256
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mihaescuvlad
commented
Oct 5, 2024
mihaescuvlad
commented
Oct 6, 2024
kuhar
changed the title
kuhar
requested changes
Oct 8, 2024
tuner/tuner/libtuner_test.py
Outdated
Comment on lines
193
to
222
| path_config = libtuner.PathConfig() | ||
| object.__setattr__(path_config, "output_json", "output.json") |
There was a problem hiding this comment.
Tests shouldn't write files in the current working directory. We can create a temporary directory for tests artifacts if needed.
mihaescuvlad
force-pushed
the
users/mihaescuvlad/benchmark-format-json
branch
from
c965b6d
to
a49ef9c
Compare
kuhar
reviewed
Oct 9, 2024
kuhar
reviewed
Oct 9, 2024
mihaescuvlad
force-pushed
the
users/mihaescuvlad/benchmark-format-json
branch
from
a49ef9c
to
761bc2e
Compare
kuhar
reviewed
Oct 9, 2024
tuner/tuner/libtuner_test.py
Outdated
Comment on lines
193
to
222
| path_config = libtuner.PathConfig() | ||
| object.__setattr__(path_config, "output_json", "output.json") |
mihaescuvlad
force-pushed
the
users/mihaescuvlad/benchmark-format-json
branch
3 times, most recently
from
550f721
to
30ee7d7
Compare
kuhar
reviewed
Oct 10, 2024
mihaescuvlad
force-pushed
the
users/mihaescuvlad/benchmark-format-json
branch
from
30ee7d7
to
6c250d9
Compare
kuhar
requested changes
Oct 11, 2024
mihaescuvlad
force-pushed
the
users/mihaescuvlad/benchmark-format-json
branch
from
6c250d9
to
20ae718
Compare
kuhar
force-pushed
the
users/mihaescuvlad/benchmark-format-json
branch
from
20ae718
to
49ec0ad
Compare
kuhar
approved these changes
Oct 11, 2024
6 tasks
monorimet
pushed a commit
that referenced
this pull request
Oct 16, 2024
### Notes - Adds `extract_benchmark_from_run_result` method to help in fetching the "benchmarks" data - Updates `IREEBenchmarkResult` model to reflect that the result is no longer stored as string but as a list of benchmarks - Updates the parsing of dispatches and models to read from Json ### Testing - Updates tests to verify that `get_mean_time` functions as expected - Updates tests to verify that Json data is properly parsed and processed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Notes
extract_benchmark_from_run_resultmethod to help in fetching the "benchmarks" dataIREEBenchmarkResultmodel to reflect that the result is no longer stored as string but as a list of benchmarksTesting
get_mean_timefunctions as expected