Document onnxruntime_perf_test usage, input data, and options#29215
Open
ArsalanShakil wants to merge 1 commit into
Open
Document onnxruntime_perf_test usage, input data, and options#29215ArsalanShakil wants to merge 1 commit into
ArsalanShakil wants to merge 1 commit into
Conversation
The perf test README was out of date: it predated the move to absl flags (so most options were missing or wrong), the sample output no longer matched the tool, and the test-data layout was described incompletely, which left users unsure how to provide inputs (see microsoft#4934). Rewrite the README to cover: - an overview including the warm-up run and what is measured, - how to build and invoke the tool, - the two ways to provide input: auto-generated input via -I, and test-data directories, with a worked onnx_test_data_utils.py example and a note that each subdirectory of the model folder is one input set, - a curated table of common options with a pointer to --help for the complete, always-current list, - runnable examples and updated sample output. Fixes microsoft#4934.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Rewrites
onnxruntime/test/perftest/README.md, which had drifted out of date:input0.pbrather thaninput_0.pb), which left users unsure how to provide inputs.The rewrite covers:
-I(no data files needed), andtools/python/onnx_test_data_utils.pyexample and a note that each subdirectory of the model folder is treated as one input set andinput_<N>.pbfiles are bound to inputs by position.--helpfor the complete, always-current list (so the README does not go stale again as options are added).All option names, defaults, the directory-scanning behavior, the warm-up run, and the output format were verified against the current source (
command_args_parser.cc,performance_runner.cc, andTestCase.cc).Motivation and Context
Fixes #4934. The issue and its comments asked for clearer documentation of
onnxruntime_perf_test, in particular how to construct input data and what the directory structure should be.