Skip to content

Commit 99979db

Browse files
html same to flaggems
1 parent 2ec1dda commit 99979db

4 files changed

Lines changed: 450 additions & 84 deletions

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ python run_flagsparse_pytest.py --phase both --mode quick --gpus 0,1 --benchmark
3939

4040
By default, `run_flagsparse_accuracy.py` and `run_flagsparse_performance.py` read operator ids from `conf/operators.yaml`, filter by `--stages`, and distribute operators across `--gpus`. `run_flagsparse_pytest.py --phase both` remains available when one command should run both phases. `--ops` and `--op-list` override the YAML selection. The default sweep excludes manual-test entries `alpha_spmm_alg1` and `spmv_coo_tocsr`; include them explicitly with `--ops` or `--op-list` when needed. Helper APIs such as `spsv_descriptor_api` and `sparse_format_constructors` are not operator test entries.
4141

42-
The accuracy phase launches `pytest tests/pytest -m <operator marker> --mode quick|normal --record json --output <op>/accuracy_result.json` and uses synthetic CUDA data. The performance phase launches the configured `tests/test_*.py` benchmark command for each operator; MatrixMarket-backed commands receive `--benchmark-input` (default `tests/data`, or pass `matrix` for the local matrix directory), and the CSV output is also normalized into a FlagGems-style `<op>/performance_result.json`. Results are written under `pytest_results_<timestamp>/` unless `--results-dir` is provided. Each operator directory contains `accuracy_stdout.log`, `accuracy_stderr.log`, `accuracy_result.json`, `accuracy_detail.json`, `performance_stdout.log`, `performance_stderr.log`, `performance.csv`, `performance_result.json`, and `performance_detail.json` when those phases run. The root `summary.json` uses the FlagGems `timestamp` / `env` / `result` structure. FlagSparse-only fields such as GPU id, commands, logs, totals, parsed pytest cases, and normalized benchmark records are kept in `summary_flat.json` and the per-operator `*_detail.json` files. `summary.csv` and optional `summary.xlsx` provide table-friendly views, and `result.html` is generated automatically for browser inspection.
42+
The accuracy phase launches `pytest tests/pytest -m <operator marker> --mode quick|normal --record json --output <op>/accuracy_result.json` and uses synthetic CUDA data. The performance phase launches the configured `tests/test_*.py` benchmark command for each operator; MatrixMarket-backed commands receive `--benchmark-input` (default `tests/data`, or pass `matrix` for the local matrix directory), and the CSV output is also normalized into a FlagGems-style `<op>/performance_result.json`. Results are written under `pytest_results_<timestamp>/` unless `--results-dir` is provided. Each operator directory contains `accuracy_stdout.log`, `accuracy_stderr.log`, `accuracy_result.json`, `accuracy_detail.json`, `performance_stdout.log`, `performance_stderr.log`, `performance.csv`, `performance_result.json`, and `performance_detail.json` when those phases run. The root `summary.json` uses the FlagGems `timestamp` / `env` / `result` structure. FlagSparse-only fields such as GPU id, commands, logs, totals, parsed pytest cases, and normalized benchmark records are kept in `summary_flat.json` and the per-operator `*_detail.json` files. `summary.csv` and optional `summary.xlsx` provide table-friendly views, and `result.html` is generated automatically for browser inspection. The generated `result.html` is rendered from `summary_flat.json`; `summary.json` remains the compact FlagGems-compatible summary for external tools.
4343

4444
**Direct pytest accuracy suite** - development-oriented accuracy checks, selectable by marker:
4545

README_cn.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ python run_flagsparse_pytest.py --phase both --mode quick --gpus 0,1 --benchmark
3737

3838
默认情况下,`run_flagsparse_accuracy.py``run_flagsparse_performance.py``conf/operators.yaml` 读取算子 id,可用 `--stages` 过滤,并按 `--gpus` 把算子分配到不同 GPU。需要一个命令同时跑两个阶段时,仍可使用 `run_flagsparse_pytest.py --phase both``--ops``--op-list` 会覆盖 YAML 选择。默认全量测试会排除手工测试项 `alpha_spmm_alg1``spmv_coo_tocsr`;需要运行时用 `--ops``--op-list` 显式指定。`spsv_descriptor_api``sparse_format_constructors` 这类辅助接口不是算子测试项。
3939

40-
精度阶段会启动 `pytest tests/pytest -m <operator marker> --mode quick|normal --record json --output <op>/accuracy_result.json`,使用合成 CUDA 数据。性能阶段会按算子启动对应的 `tests/test_*.py` benchmark 命令;依赖 MatrixMarket 矩阵的命令接收 `--benchmark-input`(默认 `tests/data`,本地矩阵目录可传 `matrix`),CSV 输出也会规范化成 FlagGems 风格的 `<op>/performance_result.json`。结果默认写入 `pytest_results_<timestamp>/`,也可通过 `--results-dir` 指定。每个算子目录在对应阶段运行后包含 `accuracy_stdout.log`、`accuracy_stderr.log`、`accuracy_result.json`、`accuracy_detail.json`、`performance_stdout.log`、`performance_stderr.log`、`performance.csv`、`performance_result.json` 和 `performance_detail.json`。根目录 `summary.json` 使用 FlagGems 的 `timestamp` / `env` / `result` 结构。GPU id、命令、日志、totals、pytest case 明细和规范化 benchmark 记录等 FlagSparse 扩展字段保存在 `summary_flat.json` 和各算子的 `*_detail.json` 中。`summary.csv` 和可选 `summary.xlsx` 用于表格查看,并会自动生成可在浏览器查看的 `result.html`。
40+
精度阶段会启动 `pytest tests/pytest -m <operator marker> --mode quick|normal --record json --output <op>/accuracy_result.json`,使用合成 CUDA 数据。性能阶段会按算子启动对应的 `tests/test_*.py` benchmark 命令;依赖 MatrixMarket 矩阵的命令接收 `--benchmark-input`(默认 `tests/data`,本地矩阵目录可传 `matrix`),CSV 输出也会规范化成 FlagGems 风格的 `<op>/performance_result.json`。结果默认写入 `pytest_results_<timestamp>/`,也可通过 `--results-dir` 指定。每个算子目录在对应阶段运行后包含 `accuracy_stdout.log`、`accuracy_stderr.log`、`accuracy_result.json`、`accuracy_detail.json`、`performance_stdout.log`、`performance_stderr.log`、`performance.csv`、`performance_result.json` 和 `performance_detail.json`。根目录 `summary.json` 使用 FlagGems 的 `timestamp` / `env` / `result` 结构。GPU id、命令、日志、totals、pytest case 明细和规范化 benchmark 记录等 FlagSparse 扩展字段保存在 `summary_flat.json` 和各算子的 `*_detail.json` 中。`summary.csv` 和可选 `summary.xlsx` 用于表格查看,并会自动生成可在浏览器查看的 `result.html`。自动生成的 `result.html` 使用 `summary_flat.json` 渲染;`summary.json` 保持为面向外部工具的精简 FlagGems 兼容汇总。
4141

4242
**直接运行 pytest 精度测试** - 面向开发调试的小规模正确性用例,可按 marker 选择:
4343

0 commit comments

Comments
 (0)