Skip to content

Add support for generating scripts to run benchmarks locally. #43

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

mattmccutchen-cci
Copy link
Collaborator

@mattmccutchen-cci mattmccutchen-cci commented Jan 7, 2022

Work in progress, to be used as part of the oopsla2022-artifact branch (which may have additional, unrelated changes for the artifact) and later polished up and merged to main. In the meantime, anyone who wants can still manually check out the run-locally branch to run benchmarks locally if they are OK with the current limitations.

Example usage:

./generate-workflow.py local --output local.sh \
  --workflow-config main --benchmark zlib --subvariant no_expand_macros_no_alltypes \
  --3c-source-dir ~/3c --3c-build-dir ~/3c/build \
  --checkedc-benchmarks-dir ~/checkedc-benchmarks \
  --work-dir ~/benchmarks/gwl-work
./local.sh

TODO:

  • Stats generation: zips are saved to a stats directory in the same format as GitHub artifacts
  • Error handling: --skip-post-conversion-build is available as a stopgap

A good starting point for further development.
No support for statistics generation yet.
We'll want this capability for local debugging even if we don't need it
for the artifact.
I already made the mistake of specifying a nonexistent directory to
`generate-workflow.py local` once.
component directory that happens to be named *.json.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant