Please download summary.xlsx to view the most recent results.
[2025-01-30 08:45:19,394 submission_checker.py:3263 INFO] Results=7, NoResults=0, Power Results=0
[2025-01-30 08:45:19,394 submission_checker.py:3270 INFO] ---
[2025-01-30 08:45:19,394 submission_checker.py:3271 INFO] Closed Results=1, Closed Power Results=0
[2025-01-30 08:45:19,394 submission_checker.py:3276 INFO] Open Results=6, Open Power Results=0
[2025-01-30 08:45:19,394 submission_checker.py:3281 INFO] Network Results=0, Network Power Results=0
[2025-01-30 08:45:19,394 submission_checker.py:3286 INFO] ---
[2025-01-30 08:45:19,394 submission_checker.py:3288 INFO] Systems=4, Power Systems=0
[2025-01-30 08:45:19,394 submission_checker.py:3292 INFO] Closed Systems=1, Closed Power Systems=0
[2025-01-30 08:45:19,395 submission_checker.py:3297 INFO] Open Systems=3, Open Power Systems=0
[2025-01-30 08:45:19,395 submission_checker.py:3302 INFO] Network Systems=0, Network Power Systems=0
[2025-01-30 08:45:19,395 submission_checker.py:3307 INFO] ---
[2025-01-30 08:45:19,395 submission_checker.py:3312 INFO] SUMMARY: submission looks OK
INFO:root: ! call "postprocess" from /home/runner/CM/repos/mlcommons@mlperf-automations/script/run-mlperf-inference-submission-checker/customize.py
| Organization | Availability | Division | SystemType | SystemName | Platform | Model | MlperfModel | Scenario | Result | Accuracy | number_of_nodes | host_processor_model_name | host_processors_per_node | host_processor_core_count | accelerator_model_name | accelerators_per_node | Location | framework | operating_system | notes | compliance | errors | version | inferred | has_power | Units | weight_data_types | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | MLCommons | available | open | edge | 34127abc9663 | mlc-server-reference-gpu-pytorch_v2.2.2-cu124 | pointpainting | pointpainting | SingleStream | 1568.9 | mAP: 54.247453827643064 | 1 | Intel(R) Xeon(R) Platinum 8480+ | 2 | 56 | NVIDIA H100 80GB HBM3 | 8 | open/MLCommons/results/mlc-server-reference-gpu-pytorch_v2.2.2-cu124/pointpainting/singlestream | pytorch v2.2.2 | Ubuntu 22.04 (linux-5.15.0-113-generic-glibc2.35) | nan | 1 | 0 | v5.0 | 0 | False | Latency (ms) | fp32 |
| 1 | MLCommons | available | open | datacenter,edge | RTX4090x2 | RTX4090x2-nvidia-gpu-TensorRT-default_config | retinanet | retinanet | Server | 1414.96 | nan | 1 | Intel(R) Xeon(R) w7-2495X | 1 | 24 | NVIDIA GeForce RTX 4090 | 2 | open/MLCommons/results/RTX4090x2-nvidia-gpu-TensorRT-default_config/retinanet/server | TensorRT | Ubuntu 20.04 (linux-6.8.0-51-generic-glibc2.31) | nan | 1 | 0 | v5.0 | 0 | False | Queries/s | int8 |
| 2 | MLCommons | available | open | datacenter,edge | RTX4090x2 | RTX4090x2-nvidia-gpu-TensorRT-default_config | retinanet | retinanet | SingleStream | 1.72409 | nan | 1 | Intel(R) Xeon(R) w7-2495X | 1 | 24 | NVIDIA GeForce RTX 4090 | 2 | open/MLCommons/results/RTX4090x2-nvidia-gpu-TensorRT-default_config/retinanet/singlestream | TensorRT | Ubuntu 20.04 (linux-6.8.0-51-generic-glibc2.31) | nan | 1 | 0 | v5.0 | 0 | False | Latency (ms) | int8 |
| 3 | MLCommons | available | open | datacenter,edge | RTX4090x2 | RTX4090x2-nvidia-gpu-TensorRT-default_config | retinanet | retinanet | MultiStream | 5.65672 | nan | 1 | Intel(R) Xeon(R) w7-2495X | 1 | 24 | NVIDIA GeForce RTX 4090 | 2 | open/MLCommons/results/RTX4090x2-nvidia-gpu-TensorRT-default_config/retinanet/multistream | TensorRT | Ubuntu 20.04 (linux-6.8.0-51-generic-glibc2.31) | nan | 1 | 0 | v5.0 | 0 | False | Latency (ms) | int8 |
| 4 | MLCommons | available | open | datacenter,edge | RTX4090x2 | RTX4090x2-nvidia-gpu-TensorRT-default_config | retinanet | retinanet | Offline | 1735.3 | nan | 1 | Intel(R) Xeon(R) w7-2495X | 1 | 24 | NVIDIA GeForce RTX 4090 | 2 | open/MLCommons/results/RTX4090x2-nvidia-gpu-TensorRT-default_config/retinanet/offline | TensorRT | Ubuntu 20.04 (linux-6.8.0-51-generic-glibc2.31) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | int8 |
| 5 | MLCommons | available | open | edge | gh_action | gh_action-reference-gpu-pytorch_v2.5.1-cu124 | stable-diffusion-xl | stable-diffusion-xl | Offline | 0.821368 | nan | 1 | Intel(R) Xeon(R) Platinum 8480+ | 2 | 56 | NVIDIA H100 80GB HBM3 | 8 | open/MLCommons/results/gh_action-reference-gpu-pytorch_v2.5.1-cu124/stable-diffusion-xl/offline | pytorch v2.5.1 | Ubuntu 22.04 (linux-5.15.0-113-generic-glibc2.35) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 6 | MLCommons | available | closed | datacenter | 5234c0b61ae3 | mlc-server-reference-gpu-pytorch_v2.4.0-cu124 | rgat | rgat | Offline | 282.683 | acc: 72.838 | 1 | Intel(R) Xeon(R) Platinum 8480+ | 2 | 56 | NVIDIA H100 80GB HBM3 | 8 | closed/MLCommons/results/mlc-server-reference-gpu-pytorch_v2.4.0-cu124/rgat/offline | pytorch v2.4.0 | Ubuntu 22.04 (linux-5.15.0-113-generic-glibc2.35) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |