Tuner will hit a runtime error with --benchmark-timing-method=rocprof:
File "amd-shark-ai/amdsharktuner/amdsharktuner/libtuner.py", line 1113, in get_valid_time_us
if result.is_valid()
^^^^^^^^^^^^^^^
AttributeError: 'RocProfBenchmarkResult' object has no attribute 'is_valid'
This happens because rocm_common.RocProfBenchmarkResult does not inherit from libtuner.BenchmarkResult, and the tool-agnostic benchmark design fails at:
benchmark_tool_config = rocm_common.RocProfConfig(benchmark_fn=rocm_common.run_rocprof_command)
result = benchmark_tool_config.benchmark_fn(BenchmarkPack(...)) # returned RocProfBenchmarkResult
There are multiple ways to fix this bug:
- Add
is_valid() to RocProfBenchmarkResult
- Move
BenchmarkResult from libtuner to common, replace RocProfBenchmarkResult in rocm_common
- Add
benchmark_tool_config.convert_benchmark_result
This bug bypassed mypy check due to the inconsistent and overly loose typing between tuner and its target backend, some other risky codes I found are:
run_rocprof_command(benchmark_pack: Any)
benchmark_fn: Callable
When needed, we can rehome some class in libtuner all together and leave in the same place, for example:
BenchmarkResult BenchmarkPack, CandidateTracker, CompilePack, not limited to common.py because they are not used by other files
Tuner will hit a runtime error with
--benchmark-timing-method=rocprof:This happens because
rocm_common.RocProfBenchmarkResultdoes not inherit fromlibtuner.BenchmarkResult, and the tool-agnostic benchmark design fails at:There are multiple ways to fix this bug:
is_valid()toRocProfBenchmarkResultBenchmarkResultfrom libtuner to common, replaceRocProfBenchmarkResultin rocm_commonbenchmark_tool_config.convert_benchmark_resultThis bug bypassed mypy check due to the inconsistent and overly loose typing between tuner and its target backend, some other risky codes I found are:
When needed, we can rehome some class in libtuner all together and leave in the same place, for example:
BenchmarkResultBenchmarkPack,CandidateTracker,CompilePack, not limited to common.py because they are not used by other files