[WIP] introduce run_gha#749
Open
fmessmer wants to merge 4 commits into
Open
Conversation
mathias-luedtke
left a comment
mathias-luedtke
left a comment
Member
There was a problem hiding this comment.
Ah, great! This can even exist next to #743 (which lacks matrix selection)
|
|
||
| args, extra_env = parse_extra_args(argv[1:]) | ||
|
|
||
| path, args = find_config_file(args, ['.github/workflows/main.yml','.github/workflows/main.yaml']) |
Member
There was a problem hiding this comment.
What about passing the yaml file as an argument?
| m = [] | ||
| g = '' | ||
| if isinstance(env, dict): | ||
| for l in env['strategy']['matrix']['include']: |
Member
There was a problem hiding this comment.
This could even compose the actual matrix
| path, args = find_config_file(args, ['.github/workflows/main.yml','.github/workflows/main.yaml']) | ||
| config = read_yaml(path) | ||
| global_env, job_envs = read_env(config['jobs']['industrial_ci']) | ||
| allow_failures = read_allow_failures(config) |
Member
There was a problem hiding this comment.
This is not supported by GHA (?)
Comment on lines
+85
to
+96
| def read_allow_failures(config): | ||
| try: | ||
| af = config['matrix']['allow_failures'] | ||
| except: | ||
| return list() | ||
| return list(x['env'] for x in af) | ||
|
|
||
| def read_num_include(config): | ||
| try: | ||
| return len(config['matrix']['include']) | ||
| except: | ||
| return 0 |
Member
There was a problem hiding this comment.
These are Travis-specific
Comment on lines
+123
to
+148
| def print_help(cmd): | ||
| print(""" | ||
| Usage: %s [PATH] [RANGE*] [-- [ENV[=VALUE]*]] | ||
|
|
||
| Parses the travis config in the given path or in the current working directory and runs all specified tests sequentially | ||
| If no range is given, the list of jobs will get printed. | ||
|
|
||
| The number of tests can be reduced by specifying one or more ranges: | ||
| * single job: 1 (only first) | ||
| * range: 2-3 (only second and third) | ||
| * open start, only as first range: -4 (jobs 1 to 4) | ||
| * open end, only as last range: 7- (job 7 and all following jobs) | ||
| * open range: - (all jobs) | ||
|
|
||
| Complex examples for a matrix wih 12 jobs: | ||
|
|
||
| * -4 7 8: runs jobs 1 2 3 4 7 8 | ||
| * 1 7-9: runs jobs 1 7 8 9 | ||
| * 1 7-9 11-: runs jobs 1 7 8 9 11 12 | ||
| * -: runs all jobs | ||
|
|
||
| The jobs will be run in clean environments. | ||
| Only DOCKER_PORT, SSH_AUTH_SOCK, and TERM will be kept. | ||
| Additional variable names can be passed at the end. | ||
|
|
||
| """ % cmd) |
Member
There was a problem hiding this comment.
This is run_travis-specific
Comment on lines
+62
to
+97
| if len(args) == 0: | ||
| if(len(global_env) > 0): | ||
| print('Globals: %s' % str(highlight_diff(global_env))) | ||
| jobs = len(job_envs) | ||
| digits = len(str(jobs)) | ||
| for i in range(jobs): | ||
| print('Job %s%s: %s' % ( str(i+1).rjust(digits), | ||
| ' (allow_failures)' if job_envs[i] in allow_failures else '', | ||
| highlight_diff(job_envs[i]) if job_envs[i] is not None else "<unsupported job from 'include' section>")) | ||
| print("run all with %s -" % sys.argv[0]) | ||
| sys.exit(0) | ||
|
|
||
| ranges = parse_ranges(args, -1) | ||
|
|
||
| run_ci = [os.path.join(scripts_dir, "run_ci"), os.path.dirname(path), filter_env(global_env)] | ||
| run_ci_diff = [os.path.join(scripts_dir, "run_ci"), os.path.dirname(path), highlight_diff(global_env, 44)] | ||
|
|
||
| bash = ['env', '-i'] +list(map(gen_env, ['DOCKER_PORT', 'HOME', 'PATH', 'SSH_AUTH_SOCK', 'TERM'])) + ['bash','-e'] | ||
|
|
||
| selection = set(apply_ranges(ranges, len(job_envs))) | ||
|
|
||
| for i in selection: | ||
| if job_envs[i] is None: | ||
| print("\033[1;43mSkipped job %d, because jobs from 'include' section are not supported\033[1;m" %(i+1,)) | ||
| continue | ||
| cmd = ' '.join(run_ci + [filter_env(job_envs[i])] + list(map(gen_env, extra_env))) | ||
| cmd_diff = ' '.join(run_ci_diff + [highlight_diff(job_envs[i], 44)] + list(map(gen_env, extra_env))) | ||
| print('\033[1;44mRunning job %d%s: %s\033[1;m' %(i+1, ' (allow_failures)' if job_envs[i] in allow_failures else '', cmd_diff)) | ||
|
|
||
| proc = subprocess.Popen(bash, stdin=subprocess.PIPE) | ||
| proc.communicate(cmd.encode()) | ||
| if proc.returncode: | ||
| print('\033[1;41mFailed job %d: %s\033[1;m' %(i+1, cmd)) | ||
| if job_envs[i] not in allow_failures: | ||
| sys.exit(proc.returncode) | ||
|
|
Member
There was a problem hiding this comment.
If I am not mistaken this is common for both.
Or at least it could be broken into smaller chunks to be reused.
Member
|
In some place "True"/"False" need to be rewritten to true/false, but I am not sure where. Probably in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
this picks up PR #582 and makes use of #717
can probably still be harmonized even further...feedback welcome