Open
Description
SUMMARY
This issue is similar to ansible/ansible#69512, but the issue is not in junos_rpc or junos_command, and as such relevant to this repository since it relies on a change to the netconf connection.
The ncclient netconf connection needs to be opened with huge_tree=True, which was added to the library in 2019 (ncclient/ncclient#319) and requires ncclient>=0.6.5.
This is required to support fetching large xml trees from network devices. This issue happens e.g. when fetching support information from a JunOS router.
ISSUE TYPE
- Bug Report
COMPONENT NAME
netconf
ANSIBLE VERSION
ansible 2.9.19
config file = /home/hexa/git/man-da/ansible-network/ansible.cfg
configured module search path = ['/home/hexa/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible
executable location = /home/hexa/git/man-da/ansible-network/.env/bin/ansible
python version = 3.8.8 (default, Feb 19 2021, 11:04:50) [GCC 10.2.0]
CONFIGURATION
DEFAULT_HOST_LIST(/home/hexa/git/man-da/ansible-network/ansible.cfg) = ['/home/hexa/git/man-da/ansible-network/inventory']
DEFAULT_STDOUT_CALLBACK(/home/hexa/git/man-da/ansible-network/ansible.cfg) = yaml
PERSISTENT_COMMAND_TIMEOUT(/home/hexa/git/man-da/ansible-network/ansible.cfg) = 600
OS / ENVIRONMENT
NixOS Unstable
Tested against Junos MX240
# pip freeze
ansible==2.9.19
appdirs @ file:///build/appdirs-1.4.4/dist/appdirs-1.4.4-py2.py3-none-any.whl
bcrypt==3.2.0
cffi==1.14.5
cryptography==3.4.7
distlib @ file:///build/distlib-0.3.1/dist/distlib-0.3.1-py3-none-any.whl
filelock @ file:///build/filelock-3.0.12/dist/filelock-3.0.12-py3-none-any.whl
Jinja2==2.11.3
lxml==4.6.3
MarkupSafe==1.1.1
ncclient==0.6.10
paramiko==2.7.2
pycparser==2.20
PyNaCl==1.4.0
PyYAML==5.4.1
six @ file:///build/six-1.15.0/dist/six-1.15.0-py2.py3-none-any.whl
virtualenv @ file:///build/virtualenv-20.2.2/dist/virtualenv-20.2.2-py2.py3-none-any.whl
STEPS TO REPRODUCE
- Run the rpc call
get-support-informationagainst a JunOS router (Mx240 in our case)
- name: "Get support information via RPC call"
junos_rpc:
rpc: get-support-information
output: text
register: rsiEXPECTED RESULTS
To get the support information in the rsi variable.
ACTUAL RESULTS
task path: /home/hexa/git/man-da/ansible-network/get_support.yml:30
<core1.fra1.ix.f.man-da.net> attempting to start connection
<core1.fra1.ix.f.man-da.net> using connection plugin netconf
<core1.fra1.ix.f.man-da.net> found existing local domain socket, using it!
<core1.fra1.ix.f.man-da.net>
<core1.fra1.ix.f.man-da.net> local domain socket path is /home/hexa/.ansible/pc/a6233c2946
<core1.fra1.ix.f.man-da.net> Using network group action junos for junos_rpc
<core1.fra1.ix.f.man-da.net> ESTABLISH LOCAL CONNECTION FOR USER: hexa
<core1.fra1.ix.f.man-da.net> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi `"&& mkdir "` echo /home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225 `" && echo ansible-tmp-1617890385.6309385-12393-213464275606225="` echo /home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225 `" ) && sleep 0'
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/network/junos/__init__.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/network/common/netconf.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/network/common/__init__.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/network/__init__.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/basic.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/six/__init__.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/network/junos/junos.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/connection.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/_text.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/json.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/__init__.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/_collections_compat.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/collections.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/text/formatters.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/parsing/__init__.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/parsing/convert_bool.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/compat/selectors.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/_json_compat.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/pycompat24.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/validation.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/text/converters.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/process.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/text/__init__.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/compat/__init__.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/sys_info.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/parameters.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/_utils.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/file.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/compat/_selectors2.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/distro/__init__.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/distro/_distro.py
Using module file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/modules/network/junos/junos_rpc.py
<core1.fra1.ix.f.man-da.net> PUT /home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/tmpzos1lj_7 TO /home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/AnsiballZ_junos_rpc.py
<core1.fra1.ix.f.man-da.net> EXEC /bin/sh -c 'chmod u+x /home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/ /home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/AnsiballZ_junos_rpc.py && sleep 0'
<core1.fra1.ix.f.man-da.net> EXEC /bin/sh -c '/usr/bin/env python3 /home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/AnsiballZ_junos_rpc.py && sleep 0'
<core1.fra1.ix.f.man-da.net> EXEC /bin/sh -c 'rm -f -r /home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/module_utils/network/common/netconf.py", line 83, in parse_rpc_error
File "src/lxml/etree.pyx", line 3237, in lxml.etree.fromstring
File "src/lxml/parser.pxi", line 1896, in lxml.etree._parseMemoryDocument
File "src/lxml/parser.pxi", line 1784, in lxml.etree._parseDoc
File "src/lxml/parser.pxi", line 1141, in lxml.etree._BaseParser._parseDoc
File "src/lxml/parser.pxi", line 615, in lxml.etree._ParserContext._handleParseResultDoc
File "src/lxml/parser.pxi", line 725, in lxml.etree._handleParseResult
File "src/lxml/parser.pxi", line 654, in lxml.etree._raiseParseError
File "<string>", line 1
lxml.etree.XMLSyntaxError: Start tag expected, '<' not found, line 1, column 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/AnsiballZ_junos_rpc.py", line 102, in <module>
_ansiballz_main()
File "/home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/AnsiballZ_junos_rpc.py", line 94, in _ansiballz_main
invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
File "/home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/AnsiballZ_junos_rpc.py", line 40, in invoke_module
runpy.run_module(mod_name='ansible.modules.network.junos.junos_rpc', init_globals=None, run_name='__main__', alter_sys=True)
File "/nix/store/5kznrncnp9v828a7i723j72bh9wkbkg7-python3-3.8.8/lib/python3.8/runpy.py", line 207, in run_module
return _run_module_code(code, init_globals, run_name, mod_spec)
File "/nix/store/5kznrncnp9v828a7i723j72bh9wkbkg7-python3-3.8.8/lib/python3.8/runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/nix/store/5kznrncnp9v828a7i723j72bh9wkbkg7-python3-3.8.8/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/modules/network/junos/junos_rpc.py", line 173, in <module>
File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/modules/network/junos/junos_rpc.py", line 154, in main
File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/module_utils/network/common/netconf.py", line 53, in exec_rpc
File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/module_utils/network/common/netconf.py", line 76, in __rpc__
File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/module_utils/network/common/netconf.py", line 108, in parse_rpc_error
ansible.module_utils.connection.ConnectionError: b'xmlSAX2Characters: huge text node, line 127359, column 84 (<string>, line 127359)'
fatal: [core1.fra1.ix.f.man-da.net]: FAILED! => changed=false
module_stderr: |-
Traceback (most recent call last):
File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/module_utils/network/common/netconf.py", line 83, in parse_rpc_error
File "src/lxml/etree.pyx", line 3237, in lxml.etree.fromstring
File "src/lxml/parser.pxi", line 1896, in lxml.etree._parseMemoryDocument
File "src/lxml/parser.pxi", line 1784, in lxml.etree._parseDoc
File "src/lxml/parser.pxi", line 1141, in lxml.etree._BaseParser._parseDoc
File "src/lxml/parser.pxi", line 615, in lxml.etree._ParserContext._handleParseResultDoc
File "src/lxml/parser.pxi", line 725, in lxml.etree._handleParseResult
File "src/lxml/parser.pxi", line 654, in lxml.etree._raiseParseError
File "<string>", line 1
lxml.etree.XMLSyntaxError: Start tag expected, '<' not found, line 1, column 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/AnsiballZ_junos_rpc.py", line 102, in <module>
_ansiballz_main()
File "/home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/AnsiballZ_junos_rpc.py", line 94, in _ansiballz_main
invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
File "/home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/AnsiballZ_junos_rpc.py", line 40, in invoke_module
runpy.run_module(mod_name='ansible.modules.network.junos.junos_rpc', init_globals=None, run_name='__main__', alter_sys=True)
File "/nix/store/5kznrncnp9v828a7i723j72bh9wkbkg7-python3-3.8.8/lib/python3.8/runpy.py", line 207, in run_module
return _run_module_code(code, init_globals, run_name, mod_spec)
File "/nix/store/5kznrncnp9v828a7i723j72bh9wkbkg7-python3-3.8.8/lib/python3.8/runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/nix/store/5kznrncnp9v828a7i723j72bh9wkbkg7-python3-3.8.8/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/modules/network/junos/junos_rpc.py", line 173, in <module>
File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/modules/network/junos/junos_rpc.py", line 154, in main
File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/module_utils/network/common/netconf.py", line 53, in exec_rpc
File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/module_utils/network/common/netconf.py", line 76, in __rpc__
File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/module_utils/network/common/netconf.py", line 108, in parse_rpc_error
ansible.module_utils.connection.ConnectionError: b'xmlSAX2Characters: huge text node, line 127359, column 84 (<string>, line 127359)'
module_stdout: ''
msg: |-
MODULE FAILURE
See stdout/stderr for the exact error
rc: 1
...ignoring
Metadata
Metadata
Assignees
Labels
No labels