Skip to content

Commit 7b9fc03

Browse files
amd-arsureshamd-arsuresh
authored
Merge branch 'mlcommons:dev' into dev
2 parents d342132 + 639f5d5 commit 7b9fc03

4 files changed

Lines changed: 86 additions & 80 deletions

File tree

‎.github/workflows/build_wheel_off.yml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,6 @@ on:
77
paths:
88
- VERSION
99

10-
1110
jobs:
1211
build_wheels:
1312
if: github.repository_owner == 'mlcommons'

‎.github/workflows/test-mlc-script-features.yml‎

Lines changed: 1 addition & 79 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
name: MLC script automation features test
22

33
on:
4-
pull_request_target:
4+
pull_request:
55
branches: [ "main", "dev" ]
66
paths:
77
- '.github/workflows/test-mlc-script-features.yml'
@@ -130,81 +130,3 @@ jobs:
130130
run: |
131131
mlc experiment script --tags=detect,os --quiet --exp.repeat,=1,2,3
132132
mlc experiment script --tags=detect,cpu --quiet --exp.explore=2:10:2
133-
134-
test_mlperf_retinanet_cpp_venv:
135-
runs-on: ubuntu-latest
136-
strategy:
137-
fail-fast: false
138-
matrix:
139-
python-version: ["3.12", "3.8"]
140-
141-
steps:
142-
- uses: actions/checkout@v4
143-
- name: Set up Python ${{ matrix.python-version }}
144-
uses: actions/setup-python@v3
145-
with:
146-
python-version: ${{ matrix.python-version }}
147-
- name: Pull MLOps repository
148-
shell: bash
149-
env:
150-
REPO: ${{ github.event.pull_request.head.repo.html_url }}
151-
BRANCH: ${{ github.event.pull_request.head.ref }}
152-
run: |
153-
pip install mlcflow
154-
mlc pull repo "$REPO" --branch="$BRANCH"
155-
156-
- name: Run MLPerf Inference Retinanet with native and virtual Python
157-
run: |
158-
mlcr app,mlperf,inference,generic,_cpp,_retinanet,_onnxruntime,_cpu --adr.python.version_min=3.8 --adr.compiler.tags=gcc --adr.openimages-preprocessed.tags=_50 --scenario=Offline --mode=accuracy --test_query_count=10 --rerun --quiet
159-
160-
mlcr app,mlperf,inference,generic,_cpp,_retinanet,_onnxruntime,_cpu --adr.python.version_min=3.8 --adr.compiler.tags=gcc --adr.openimages-preprocessed.tags=_50 --scenario=Offline --mode=performance --test_query_count=10 --rerun --quiet
161-
162-
mlcr install,python-venv --version=3.10.8 --name=mlperf --quiet
163-
164-
export MLC_SCRIPT_EXTRA_CMD="--adr.python.name=mlperf"
165-
166-
mlcr run,mlperf,inference --adr.python.version_min=3.8 --adr.compiler.tags=gcc --adr.openimages-preprocessed.tags=_50 --submitter=MLCommons --implementation=cpp --hw_name=default --model=retinanet --backend=onnxruntime --device=cpu --scenario=Offline --quiet
167-
168-
# Step for Linux/MacOS
169-
- name: Randomly Execute Step (Linux/MacOS)
170-
if: runner.os != 'Windows'
171-
run: |
172-
RANDOM_NUMBER=$((RANDOM % 10))
173-
echo "Random number is $RANDOM_NUMBER"
174-
if [ "$RANDOM_NUMBER" -eq 0 ]; then
175-
echo "run_step=true" >> $GITHUB_ENV
176-
else
177-
echo "run_step=false" >> $GITHUB_ENV
178-
fi
179-
180-
# Step for Windows
181-
- name: Randomly Execute Step (Windows)
182-
if: runner.os == 'Windows'
183-
run: |
184-
$RANDOM_NUMBER = Get-Random -Maximum 10
185-
Write-Host "Random number is $RANDOM_NUMBER"
186-
if ($RANDOM_NUMBER -eq 0) {
187-
Write-Host "run_step=true" | Out-File -FilePath $Env:GITHUB_ENV -Append
188-
} else {
189-
Write-Host "run_step=false" | Out-File -FilePath $Env:GITHUB_ENV -Append
190-
}
191-
192-
- name: Load secrets
193-
if: github.repository_owner == 'mlcommons' && env.run_step == 'true'
194-
id: op-load-secrets
195-
uses: 1password/load-secrets-action@v3
196-
env:
197-
OP_SERVICE_ACCOUNT_TOKEN: ${{ secrets.OP_SERVICE_ACCOUNT_TOKEN }}
198-
PAT: op://7basd2jirojjckncf6qnq3azai/bzbaco3uxoqs2rcyu42rvuccga/credential
199-
- name: Push Results
200-
env:
201-
GITHUB_TOKEN: ${{ steps.op-load-secrets.outputs.PAT }}
202-
if: github.repository_owner == 'mlcommons' && env.run_step == 'true'
203-
run: |
204-
git config --global user.name "mlcommons-bot"
205-
git config --global user.email "mlcommons-bot@users.noreply.github.com"
206-
git config --global credential.https://github.com.helper ""
207-
git config --global credential.https://github.com.helper "!gh auth git-credential"
208-
git config --global credential.https://gist.github.com.helper ""
209-
git config --global credential.https://gist.github.com.helper "!gh auth git-credential"
210-
mlcr push,github,mlperf,inference,submission --repo_url=https://github.com/mlcommons/mlperf_inference_test_submissions_v5.0 --repo_branch=auto-update --commit_message="Results from R50 GH action on ${{ matrix.os }}" --quiet

‎script/benchmark-program-geekbench/README.md‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# README for benchmark-program-geekbench
2+
This README is automatically generated. Create and add custom content in info.md. Please follow the [script execution document](https://docs.mlcommons.org/mlcflow/targets/script/execution-flow/) to understand more about the MLC script execution.
3+
4+
`mlcflow` stores all local data under `$HOME/MLC` by default. So, if there is space constraint on the home directory and you have more space on say `/mnt/$USER`, you can do
5+
```
6+
mkdir /mnt/$USER/MLC
7+
ln -s /mnt/$USER/MLC $HOME/MLC
8+
```
9+
You can also use the `ENV` variable `MLC_REPOS` to control this location but this will need a set after every system reboot.
10+
11+
## Setup
12+
13+
If you are not on a Python development environment please refer to the [official docs](https://docs.mlcommons.org/mlcflow/install/) for the installation.
14+
15+
```bash
16+
python3 -m venv mlcflow
17+
. mlcflow/bin/activate
18+
pip install mlcflow
19+
```
20+
21+
- Using a virtual environment is recommended (per `pip` best practices), but you may skip it or use `--break-system-packages` if needed.
22+
23+
### Pull mlperf-automations
24+
25+
Once `mlcflow` is installed:
26+
27+
```bash
28+
mlc pull repo mlcommons@mlperf-automations --pat=<Your Private Access Token>
29+
```
30+
- `--pat` or `--ssh` is only needed if the repo is PRIVATE
31+
- If `--pat` is avoided, you'll be asked to enter the password where you can enter your Private Access Token
32+
- `--ssh` option can be used instead of `--pat=<>` option if you prefer to use SSH for accessing the github repository.
33+
## Run Commands
34+
35+
```bash
36+
mlcr benchmark,geekbench,benchmark-geekbench,benchmark-program-geekbench
37+
```
38+
39+
### Script Inputs
40+
41+
| Name | Description | Choices | Default |
42+
|------|-------------|---------|------|
43+
| `--license_email` | | | `` |
44+
| `--license_key` | | | `` |
45+
| `--workload` | | | `` |
46+
| `--local_file` | | | `` |
47+
### Generic Script Inputs
48+
49+
| Name | Description | Choices | Default |
50+
|------|-------------|---------|------|
51+
| `--input` | Input to the script passed using the env key `MLC_INPUT` | | `` |
52+
| `--output` | Output from the script passed using the env key `MLC_OUTPUT` | | `` |
53+
| `--outdirname` | The directory to store the script output | | `cache directory ($HOME/MLC/repos/local/cache/<>) if the script is cacheable or else the current directory` |
54+
| `--outbasename` | The output file/folder name | | `` |
55+
| `--search_folder_path` | The folder path where executables of a given script need to be searched. Search is done recursively upto 4 levels. | | `` |
56+
| `--name` | | | `` |
57+
| `--extra_cache_tags` | Extra cache tags to be added to the cached entry when the script results are saved | | `` |
58+
| `--skip_compile` | Skip compilation | | `False` |
59+
| `--skip_run` | Skip run | | `False` |
60+
| `--skip_sudo` | Skip SUDO detection | | `False` |
61+
| `--accept_license` | Accept the required license requirement to run the script | | `False` |
62+
| `--skip_system_deps` | Skip installing any system dependencies | | `False` |
63+
| `--git_ssh` | Use SSH for git repos | | `False` |
64+
| `--gh_token` | Github Token | | `` |
65+
| `--hf_token` | Huggingface Token | | `` |
66+
| `--verify_ssl` | Verify SSL | | `False` |
67+
## Variations
68+
69+
### Upload-mode
70+
71+
- `no-upload` (default)
72+
- `upload`
73+
74+
### Workload
75+
76+
- `compute`
77+
- `cpu` (default)
78+
- `gpu-cuda`
79+
- `gpu-metal`
80+
- `gpu-opencl`
81+
- `gpu-vulkan`

‎script/benchmark-program-geekbench/meta.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,10 @@ tags:
2424
- benchmark-geekbench
2525
- benchmark-program-geekbench
2626
uid: eec4b864cb654974
27+
tests:
28+
run_inputs:
29+
- variations_list:
30+
- upload
2731
variations:
2832
cpu:
2933
default: true

0 commit comments

Comments
 (0)