Skip to content

Refactoring interface common routines#1079

Draft
matburt wants to merge 1 commit intoansible:develfrom
matburt:fix_up_entrypoints
Draft

Refactoring interface common routines#1079
matburt wants to merge 1 commit intoansible:develfrom
matburt:fix_up_entrypoints

Conversation

@matburt
Copy link
Member

@matburt matburt commented May 19, 2022

This should also fix up direct role execution when run from the command line using an execution environment.

Still need to sort out some tests and make sure the cli entrypoint is doing the right thing in all circumstances.

@matburt
Refactoring interface common routines
4fb5cfe 
This should also fix up direct role execution when run from the command line using an execution environment
@github-actions github-actions bot added the needs_triage New item that needs to be triaged label May 20, 2022
matburt
matburt commented May 20, 2022
View reviewed changes
ansible_runner/interface.py
# If running via the transmit-worker-process method, we must only extract things as read-only
# inside of one of these commands. That could be either transmit or worker.
if kwargs.get('streamer') not in ('worker', 'process'):
if kwargs.get('process_isolation') or kwargs.get('streamer') not in ('worker', 'process'):
Copy link
Member Author

@matburt matburt May 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These lines are what I was really looking for in here. Trying to use direct role execution from the CLI using an execution environment didn't work because of the path assignment behavior.

Copy link
Contributor

@jbradberry jbradberry Jun 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoa, wait. Just on the surface of it, it seems like making this change would break the transmit phase of streaming runner.

Copy link
Contributor

@jbradberry jbradberry Jun 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... or I misread the logic.

Copy link
Contributor

@jbradberry jbradberry Jun 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After reconsidering, though, I'm not sure how it helps to do the dump_artifacts writing to disk in the worker and processor cases when we have process isolation? The worker should already have everything by unzipping everything that came over the wire from the transmit phase, and I don't think that the processor phase needs any of it.

Copy link
Member

@AlanCoding AlanCoding Jun 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In practice, artifacts shouldn't be getting dumped because of suppress_env_files

@Akasurde Akasurde removed the needs_triage New item that needs to be triaged label May 31, 2022
@matburt matburt mentioned this pull request Jun 1, 2022
Merged
AlanCoding
AlanCoding reviewed Jun 1, 2022
View reviewed changes
ansible_runner/__main__.py
if vargs.get('command') in ('transmit', 'worker', 'process'):
streamer = vargs.get('command')

# TODO: Remove / Refactor unused role context manager
Copy link
Member

@AlanCoding AlanCoding Jun 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like it was removed?

@AlanCoding
Copy link
Member

AlanCoding commented Jun 1, 2022

I tested in conjunction with AWX / AAP controller, and normal jobs error with this from the worker:

{'private_data_dir': '/tmp/awx_91_2oshmw_9', 'ident': UUID('91c14a59-dd60-4e57-b589-cc430806c44b'), 'binary': None, 'playbook': None, 'module': None, 'module_args': None, 'host_pattern': None, 'verbosity': None, 'quiet': False, 'rotate_artifacts': 0, 'ignore_logging': False, 'json_mode': False, 'omit_event_data': False, 'only_failed_event_data': False, 'inventory': None, 'forks': None, 'project_dir': None, 'artifact_dir': None, 'role': None, 'roles_path': None, 'process_isolation': None, 'process_isolation_executable': None, 'process_isolation_path': None, 'process_isolation_hide_paths': None, 'process_isolation_show_paths': None, 'process_isolation_ro_paths': None, 'container_image': None, 'container_volume_mounts': None, 'container_options': None, 'directory_isolation_base_path': None, 'cmdline': None, 'limit': None, 'streamer': 'worker', 'suppress_env_files': False}
{"status": "starting", "runner_ident": "91", "command": ["podman", "run", "--rm", "--tty", "--interactive", "--workdir", "/runner/project", "-v", "/tmp/awx_91_2oshmw_9/:/runner/:Z", "-v", "/tmp/ansible_runner_plugins_ztvw3gwc/callback/:/home/runner/.ansible/plugins/callback/:Z", "--authfile=/tmp/ansible_runner_registry_91_sk4ys67n/auth.json", "-v", "/etc/pki/ca-trust/:/etc/pki/ca-trust/:O", "-v", "/opt/tmpawx/:/opt/tmpawx/:z", "-v", "/usr/share/pki/:/usr/share/pki/:O", "-v", "/var/lib/awx/projects/_80__project_texthat/:/var/lib/awx/projects/_80__project_texthat/:z", "-v", "/var/lib/awx/projects/.__awx_cache/_80__project_texthat/:/var/lib/awx/projects/.__awx_cache/_80__project_texthat/:z", "--env-file", "/tmp/awx_91_2oshmw_9/artifacts/91/env.list", "--quiet", "--name", "ansible_runner_91", "--user=root", "--network", "slirp4netns:enable_ipv6=true", "brew.registry.redhat.io/rh-osbs/ansible-automation-platform-22-ee-supported-rhel8:latest", "ansible-playbook", "-t", "update_git,install_roles,install_collections", "-i", "/runner/inventory/hosts", "-e", "@/runner/env/extravars", "main.json"], "env": {"ANSIBLE_UNSAFE_WRITES": "1", "AWX_ISOLATED_DATA_DIR": "/runner/artifacts/91", "ANSIBLE_FORCE_COLOR": "True", "ANSIBLE_HOST_KEY_CHECKING": "False", "ANSIBLE_INVENTORY_UNPARSED_FAILED": "True", "ANSIBLE_PARAMIKO_RECORD_HOST_KEYS": "False", "AWX_PRIVATE_DATA_DIR": "/tmp/awx_91_2oshmw_9", "ANSIBLE_RETRY_FILES_ENABLED": "False", "ANSIBLE_ASK_PASS": "False", "ANSIBLE_BECOME_ASK_PASS": "False", "DISPLAY": "", "TMP": "/tmp", "PROJECT_UPDATE_ID": "91", "ANSIBLE_ROLES_PATH": "roles", "ANSIBLE_STDOUT_CALLBACK": "awx_display", "RUNNER_OMIT_EVENTS": "False", "RUNNER_ONLY_FAILED_EVENTS": "False"}, "cwd": "/runner/project"}
{"status": "running", "runner_ident": "91"}
{"event": "verbose", "uuid": "1a72a67a-a609-4f4e-8178-ee6b772b7489", "counter": 1, "stdout": "\u001b[0;31mERROR! role definitions must contain a role name\u001b[0m", "start_line": 0, "end_line": 1, "runner_ident": "91", "created": "2022-06-01T20:38:57.916933"}
{"status": "failed", "runner_ident": "91"}
{"zipfile": 2110}
UEsDBBQAAAAIANqkwVQwb5Ix7AIAAJYGAAAHAAAAY29tbWFuZJVU70/bMBD9V1A/k2YwxKAS0tIkHVnTpEpSWDdNlpuaxjSJM9tpKYj/fef8QCkUbXyqz/feu+e7S596McsynC97g6NfvYIt4dw7PurxsvrRNJ7Vv1Lu6gPNJeE4lnRD6ost4+sl5SrQgZYTrhec3ZNYVvlNlZBZoePtA7o8QadMJNkWXeqDFj74+RqZC7pICarzqEjLFc0FepSb7efVNtZjnKYLHK9BImEZaXX6DU9vCB1cXUHDpUzuaEquDpXhZEWF5DtlUqzPduL8S64rRv9esLxjkchYL9YU5DXJSyHBxtsrv0NghVQFoQEA7QaPHVApuC4SzEklNHgdd/U2mOspXaiOtr0WOrr4hFATIUkeZIKVsw9gH/9Voo+QGmKM44R8qN5/EOviGsk3mprQu1uDuaR3WIlenuiA7qcwtJr7p6SkOeY4qyRejfjypE6XgvArzliLJlJtsQpESnlxBhe5GJAcKy4tNudXMNVKcMHJtt9uChyWYL5Pmc4TjYmFaHdKbRrLsKQs14oUyzvGM+30VCNEE2VRMC7JUuMJSS8GkCX1C1ouEHYLxio/WpUpiyWg0IrKY9hrCWuNOEuJeIlilqbQTignKhbtfo4035BcMr6Dr0XIGlA95muLgD7qMAiOYXpVPsM0r9f+N0TxVv0/HPq8gQiZp57hhc7QtdHMC42RjW4DJ7JDxakabtz+QE7ou0ZkW8gyIgNZTtBV7A61IjRyIz8wbWT6rl/Bo2YIbfraDyM0tufIvLbNseN9U6ARTsUeyvFubC/ygzm4mxpBCB5GhuPa1iFJABgTZ+yjwDb9wHopEe5Lw4OmgXMDD9p/z4GN7aoHdgQ2RlA8RLZnDBsTbywb4RichOHB5BCcTd7BWE44dY25ulNhNJm2tlQ4Dfzvthmh2dRSzp2q+H7HA195mxrRtcpVS9ZNh5HlzyJkGq47NMyxwqjnLqlQS6uQwczz7AD5EydCtmr8vsE27bnzZgpvUM/PfwFQSwMEFAAAAAgA2qTBVCLrAkISAQAA6AEAAAgAAABlbnYubGlzdGWQzU7DMBAG73mZCG49+LCxN9TEsa2105LTKoIiKvVPSavC2+OgNrRw9VjfjBZs0IVBbmyAEnlJOmIQDxksX1gHZyCiYgURWGkSeX/a7VZ93vXH9Xv3ehzyWfp6mSgdSWTpjCMR+9NqAnMXIlfYspyjrLR9EmW3GX65tgu00VGbKjxQSMYStEF1P5MQ1LpyTCgdqWk2XOdSsie9SMk3xcftIe/Onzx74Mf98LE982xaJIxJWiZVYLRQjMr7NAhV8obw57lIBfU/qnTwBlqRxdr/iDNP7hll5MarsUorcXMvcqPXQ5yLfr9ZDRMIUbkmsgRjCpCVGPPf1sNh031l1FiLxK7WkXE821V+Bda0l+vd8W9QSwMEFAAAAAgA26TBVLWxEZlpAQAAswQAABAAAABjb2xsZWN0aW9ucy5qc29uhZTdcoIwEIXv+xQO151FQNtpX8aJYZWtIaG7AbSO714UdShBewHk58tJds+GYxTXwrEUijFWVmhtMNbOGNSenJXb2GowFn3OjpEq1Y+zoNq+2yBLN9e1owxSmEen11l0XQvaWc/n9TxmFwFr0WtXlpfpkew8RFvHuzGYBJqVE9qHWDbCBHXN5A8hOd659mSCuFN4g+QP1pLNXZigBD5Gegdlt092ZRKvAF2gtLwzmkQ7UKIep61HKFSZQPaBU2PI7p8KGVfnMG1kCsk1Txtm6J4QuMnQuoRvVrl6wnzVlirkazEIdP1nJ9vV6wuM0h2PMdRNIbuArkLbSEteFzBoT0XTKzPmhfJAnafbwk+4Pof3IXgWlYI2/l9JLppQ7Q2WQ0aUR2PIBxFl9zoX5IY0WtcCeSmnLkQfulSmtjvAiRK/pdEz2rwkzQ5yxKq7Oo/Zpmy7vwv0nxWjTEa86NmDEzi/Hjh4Or38AlBLAwQUAAAACADcpMFUFAebPxgAAAAWAAAAEwAAAGFuc2libGVfdmVyc2lvbi50eHRLzCvOTMpJVYhOzi9KVTDSMzTWM4jlAgBQSwMEFAAAAAgA3aTBVHeji54IAAAABgAAAAYAAABzdGF0dXNLS8zMSU0BAFBLAwQUAAAACADdpMFUt+/cgwMAAAABAAAAAgAAAHJjMwQAUEsDBBQAAAAAANqkwVQAAAAAAAAAAAAAAAALAAAAam9iX2V2ZW50cy9QSwECFAMUAAAACADapMFUMG+SMewCAACWBgAABwAAAAAAAAAAAAAAgIEAAAAAY29tbWFuZFBLAQIUAxQAAAAIANqkwVQi6wJCEgEAAOgBAAAIAAAAAAAAAAAAAACkgREDAABlbnYubGlzdFBLAQIUAxQAAAAIANukwVS1sRGZaQEAALMEAAAQAAAAAAAAAAAAAACkgUkEAABjb2xsZWN0aW9ucy5qc29uUEsBAhQDFAAAAAgA3KTBVBQHmz8YAAAAFgAAABMAAAAAAAAAAAAAAKSB4AUAAGFuc2libGVfdmVyc2lvbi50eHRQSwECFAMUAAAACADdpMFUd6OLnggAAAAGAAAABgAAAAAAAAAAAAAAgIEpBgAAc3RhdHVzUEsBAhQDFAAAAAgA3aTBVLfv3IMDAAAAAQAAAAIAAAAAAAAAAAAAAICBVQYAAHJjUEsBAhQDFAAAAAAA2qTBVAAAAAAAAAAAAAAAAAsAAAAAAAAAAAAQAMBBeAYAAGpvYl9ldmVudHMvUEsFBgAAAAAHAAcAhwEAAKEGAAAAAA=={"eof": true}

In this case, it should be running a normal playbook, but it seems to somehow interpret that as direct role execution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers

@jbradberry jbradberry jbradberry left review comments

@AlanCoding AlanCoding AlanCoding left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@matburt @AlanCoding @jbradberry @Akasurde

Comments