fix: add inventory_hostname to host list

[josephwhite13]

It seems that the even though a default value is defined, it is now expected to be explicitly listed in the options.

Fixes the following warning message: [WARNING]: The "ansible_collections.juniper.device.plugins.connection.pyez" connection plugin has an improperly configured remote target value, forcing "inventory_hostname" templated value instead of the string

[ed-ud]

This is still a problem and the above fix does work to correct the error.

[chidanandpujar]

Hi @josephwhite13 , @ed-ud ,
Could you please share the steps to replicate this warning message, so that I can validate the fix.

Thanks
Chidanand

[ed-ud]

Hi @josephwhite13 , @ed-ud , Could you please share the steps to replicate this warning message, so that I can validate the fix.

Thanks Chidanand

Actually today we found that when you create a new python3.12 venv, and install Ansible and the juniper.device collection, the warning is still present but it also breaks juniper.device.pyez althogether unless you add the inventory_hostname to the list of vars as noted.

Steps:

$ python3.12 -m venv venv
$ . venv/bin/activate
$ pip install -U pip
$ pip install ansible
$ ansible-galaxy collection install juniper.device
$ pip install -r ~/.ansible/collections/ansible_collections/juniper/device/requirements.txt

Run a playbook that uses juniper.device.facts:

---
- name: Get Device Facts
  hosts: "{{ hostlist | split(',') }}"
  connection: juniper.device.pyez
  gather_facts: no

  tasks:
  - name: Retrieve facts
    juniper.device.facts:
    register: out1

  - debug: var=out1

PLAY [Get Device Facts] *****************************************************************************************

TASK [Retrieve facts] *******************************************************************************************
[WARNING]: The "juniper.device.pyez" connection plugin has an improperly configured remote target value, forcing
"inventory_hostname" templated value instead of the string
<unknown>:1812: SyntaxWarning: invalid escape sequence '\*'

fatal: [sw-nc38-326-20]: FAILED! => {
  "changed": false,
  "module_stderr": "/tmp/ansible_juniper.device.facts_payload_wlyanm_2/ansible_juniper.device.facts_payload.zip/ansible_collections/juniper/device/plugins/module_utils/juniper_junos_common.py:1812: SyntaxWarning: invalid escape sequence '\\*'
/tmp/ansible_juniper.device.facts_payload_wlyanm_2/ansible_juniper.device.facts_payload.zip/ansible_collections/juniper/device/plugins/module_utils/juniper_junos_common.py:1812: SyntaxWarning: invalid escape sequence '\\*'
Traceback (most recent call last):
  File \"/home/netadmin/.ansible/tmp/ansible-local-306021lvzkqw7h/ansible-tmp-1724434681.6407797-306024-64901492023059/AnsiballZ_facts.py\", line 107, in <module>
    _ansiballz_main()
  File \"/home/netadmin/.ansible/tmp/ansible-local-306021lvzkqw7h/ansible-tmp-1724434681.6407797-306024-64901492023059/AnsiballZ_facts.py\", line 99, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File \"/home/netadmin/.ansible/tmp/ansible-local-306021lvzkqw7h/ansible-tmp-1724434681.6407797-306024-64901492023059/AnsiballZ_facts.py\", line 47, in invoke_module
    runpy.run_module(mod_name='ansible_collections.juniper.device.plugins.modules.facts', init_globals=dict(_module_fqn='ansible_collections.juniper.device.plugins.modules.facts', _modlib_path=modlib_path),
  File \"<frozen runpy>\", line 226, in run_module
  File \"<frozen runpy>\", line 98, in _run_module_code
  File \"<frozen runpy>\", line 88, in _run_code
  File \"/tmp/ansible_juniper.device.facts_payload_wlyanm_2/ansible_juniper.device.facts_payload.zip/ansible_collections/juniper/device/plugins/modules/facts.py\", line 367, in <module>
  File \"/tmp/ansible_juniper.device.facts_payload_wlyanm_2/ansible_juniper.device.facts_payload.zip/ansible_collections/juniper/device/plugins/modules/facts.py\", line 309, in main
  File \"/tmp/ansible_juniper.device.facts_payload_wlyanm_2/ansible_juniper.device.facts_payload.zip/ansible_collections/juniper/device/plugins/module_utils/juniper_junos_common.py\", line 631, in __init__
  File \"/tmp/ansible_juniper.device.facts_payload_wlyanm_2/ansible_juniper.device.facts_payload.zip/ansible_collections/juniper/device/plugins/module_utils/juniper_junos_common.py\", line 692, in get_connection
  File \"/tmp/ansible_juniper.device.facts_payload_wlyanm_2/ansible_juniper.device.facts_payload.zip/ansible_collections/juniper/device/plugins/module_utils/juniper_junos_common.py\", line 708, in get_capabilities
  File \"/tmp/ansible_juniper.device.facts_payload_wlyanm_2/ansible_juniper.device.facts_payload.zip/ansible/module_utils/connection.py\", line 199, in __rpc__
ansible.module_utils.connection.ConnectionError: Unable to make a PyEZ connection: ConnectUnknownHostError(inventory_hostname)
",
  "module_stdout": "",
  "msg": "MODULE FAILURE
See stdout/stderr for the exact error",
  "rc": 1
}

PLAY RECAP ******************************************************************************************************
sw-nc38-326-20             : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

Note that without the fix, it's trying to connect to the host "inventory_hostname" instead of the value of that variable.
Add inventory_hostname to the vars list:

$ diff -c ~/.ansible/collections/ansible_collections/juniper/device/plugins/connection/pyez.py.broken ~/.ansible/collections/ansible_collections/juniper/device/plugins/connection/pyez.py
*** /home/netadmin/.ansible/collections/ansible_collections/juniper/device/plugins/connection/pyez.py.broken    2024-08-23 13:34:57.744778129 -0400
--- /home/netadmin/.ansible/collections/ansible_collections/juniper/device/plugins/connection/pyez.py   2024-08-23 13:35:08.100793642 -0400
***************
*** 48,53 ****
--- 48,54 ----
        to.
      default: inventory_hostname
      vars:
+     - name: inventory_hostname
      - name: ansible_host
      - name: host
      - name: hostname

... and the warning is fixed and the connection succeeds.

Regarding the other warning - the syntax error causing "invalid escape *" is in "juniper/device/plugins/module_utils/juniper_junos_common.py" in this section:

    def local_md5(self, package, action):
        """
        Computes the MD5 checksum value on the local package file.
        :param str package:
            File-path to the package (\*.tgz) file on the local server
        :returns: MD5 checksum (str)
        :raises IOError: when **package** file does not exist
        """

There is no need to try to escape the asterisk inside the comment.

The other disturbing thing that I've come across while testing this over and over in a pristine environment is that the connections never ask for verification of any host's SSH public key. I've tried a dozen new hosts and it just blindly connects without any host key verification, despite explicity setting "host_key_auto_add=false" in ansible.cfg (which is the default).

[jeffsw]

I encountered this problem today and was surprised to find this PR unmerged. This patch does solve the problem.

[josephwhite13]

@chidanandpujar - were you able to validate the fix as outlined by @ed-ud? Thanks!

[chidanandpujar]

@chidanandpujar - were you able to validate the fix as outlined by @ed-ud? Thanks!

Hi @josephwhite13 @ed-ud
Thanks for sharing the details.
I will try to replicate this issue with the steps provided.

Thanks
Chidanand

[chidanandpujar]

Hi @ed-ud,
I tried the steps provided; I am getting following error.
Could you please share the inventory and ansible.cfg details.

ansible-playbook issue_593.yml 
ERROR! The field 'hosts' has an invalid value, which includes an undefined variable.. Unable to look up a name or access an attribute in template string ({{ hostlist | split(',') }}).
Make sure your variable name does not contain invalid characters like '-': descriptor 'split' for 'str' objects doesn't apply to a 'AnsibleUndefined' object. descriptor 'split' for 'str' objects doesn't apply to a 'AnsibleUndefined' object

The error appears to be in '/root/ansible_issue_593/ansible-junos-stdlib/tests/issue_593.yml': line 2, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

---
- name: Get Device Facts
  ^ here

Thanks
Chidanand

[ed-ud]

Hi @ed-ud, I tried the steps provided; I am getting following error. Could you please share the inventory and ansible.cfg details.

ansible-playbook issue_593.yml 
ERROR! The field 'hosts' has an invalid value, which includes an undefined variable.. Unable to look up a name or access an attribute in template string ({{ hostlist | split(',') }}).
Make sure your variable name does not contain invalid characters like '-': descriptor 'split' for 'str' objects doesn't apply to a 'AnsibleUndefined' object. descriptor 'split' for 'str' objects doesn't apply to a 'AnsibleUndefined' object

The error appears to be in '/root/ansible_issue_593/ansible-junos-stdlib/tests/issue_593.yml': line 2, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

---
- name: Get Device Facts
  ^ here

Thanks Chidanand

Pass the hostlist variable on the command-line to choose which host in YOUR inventory will be tried.

[chidanandpujar]

Hi @ed-ud, I tried the steps provided; I am getting following error. Could you please share the inventory and ansible.cfg details.

ansible-playbook issue_593.yml 
ERROR! The field 'hosts' has an invalid value, which includes an undefined variable.. Unable to look up a name or access an attribute in template string ({{ hostlist | split(',') }}).
Make sure your variable name does not contain invalid characters like '-': descriptor 'split' for 'str' objects doesn't apply to a 'AnsibleUndefined' object. descriptor 'split' for 'str' objects doesn't apply to a 'AnsibleUndefined' object

The error appears to be in '/root/ansible_issue_593/ansible-junos-stdlib/tests/issue_593.yml': line 2, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

---
- name: Get Device Facts
  ^ here

Thanks Chidanand

Pass the hostlist variable on the command-line to choose which host in YOUR inventory will be tried.

Hi @ed-ud ,
Thanks very much for sharing the required details.
I am able to replicate this issue on my test setup.

ansible-playbook -i inventory issue_593.yml -e "hostlist=x.x.x.x" --become-user root -k
  

SSH password: 

PLAY [Get Device Facts] ***************************************************************************************************************************************************

TASK [Retrieve facts] *****************************************************************************************************************************************************
[WARNING]: The "ansible_collections.juniper.device.plugins.connection.pyez" connection plugin has an improperly configured remote target value, forcing
"inventory_hostname" templated value instead of the string
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: AttributeError: 'JuniperJunosModule' object has no attribute '_pyez_conn'
fatal: [10.220.2.221]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n  File \"/root/.ansible/tmp/ansible-local-1812s9zab1jz/ansible-tmp-1731074869.9963074-1816-207211164039892/AnsiballZ_facts.py\", line 107, in <module>\n    _ansiballz_main()\n  File \"/root/.ansible/tmp/ansible-local-1812s9zab1jz/ansible-tmp-1731074869.9963074-1816-207211164039892/AnsiballZ_facts.py\", line 99, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/root/.ansible/tmp/ansible-local-1812s9zab1jz/ansible-tmp-1731074869.9963074-1816-207211164039892/AnsiballZ_facts.py\", line 47, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.juniper.device.plugins.modules.facts', init_globals=dict(_module_fqn='ansible_collections.juniper.device.plugins.modules.facts', _modlib_path=modlib_path),\n  File \"/usr/local/lib/python3.10/runpy.py\", line 224, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/local/lib/python3.10/runpy.py\", line 96, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/usr/local/lib/python3.10/runpy.py\", line 86, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_juniper.device.facts_payload_gubcdjbu/ansible_juniper.device.facts_payload.zip/ansible_collections/juniper/device/plugins/modules/facts.py\", line 367, in <module>\n  File \"/tmp/ansible_juniper.device.facts_payload_gubcdjbu/ansible_juniper.device.facts_payload.zip/ansible_collections/juniper/device/plugins/modules/facts.py\", line 309, in main\n  File \"/tmp/ansible_juniper.device.facts_payload_gubcdjbu/ansible_juniper.device.facts_payload.zip/ansible_collections/juniper/device/plugins/module_utils/juniper_junos_common.py\", line 610, in __init__\n  File \"/tmp/ansible_juniper.device.facts_payload_gubcdjbu/ansible_juniper.device.facts_payload.zip/ansible_collections/juniper/device/plugins/module_utils/juniper_junos_common.py\", line 742, in fail_json\n  File \"/tmp/ansible_juniper.device.facts_payload_gubcdjbu/ansible_juniper.device.facts_payload.zip/ansible_collections/juniper/device/plugins/module_utils/juniper_junos_common.py\", line 1213, in close_configuration\nAttributeError: 'JuniperJunosModule' object has no attribute '_pyez_conn'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

PLAY RECAP ****************************************************************************************************************************************************************
x.x.x.x               : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   

[gneville-ot]

Can confirm this is still an issue today with Ansible 11.5.0 / Core 2.18.5 / juniper.device 1.0.8.
Applying the fix in this PR works

[ed-ud]

I installed juniper.device 1.0.8... it doesn't appear to have the fix in ansible_collections/juniper/device/plugins/connection/pyez.py ...

# /opt/prod/netadmin/collections/ansible_collections
Collection                               Version
---------------------------------------- -------
juniper.device                           1.0.8

        "Package               Version",
        "--------------------- ---------",
        "ansible               11.5.0",
        "ansible-core          2.18.5",

If you look at the relevant section in the file - it's missing the "inventory_hostname" in the vars list for the "host" option...

DOCUMENTATION = """author: Juniper Automation Team
connection: pyez
short_description: Use pyez to run command on JUNOS appliances
description:
- This connection plugin provides a connection to remote devices over the junos-pyez library.
options:
  host:
    description:
    - Specifies the remote device FQDN or IP address to establish the SSH connection
      to.
    default: inventory_hostname
    vars:
    - name: ansible_host
    - name: host
    - name: hostname
    - name: ip
  port:
    description:

... so we get the exact same error as was originally reported stating that it couldn't connect to host called "inventory_hostname".

[WARNING]: The "juniper.device.pyez" connection plugin has an improperly configured remote target value, forcing
"inventory_hostname" templated value instead of the string
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ansible.module_utils.connection.ConnectionError: Unable to make a PyEZ connection: ConnectUnknownHostError(inventory_hostname)

[slefol]

Can confirm this is still an issue today with ansible-core 2.18.8 / juniper.device 1.0.8 or 1.0.9.
Applying the fix in this PR works

[TafkaMax]

Any updates on this issue?!

[TafkaMax]

@chidanandpujar please merge this, i am having the same issue with 3.12.12 ansible core 2.17 and juniper.device 1.0.9

I have to manually specify ansible_host=X.X.X.X to each host.