Skip to content

Can't configure Junos 18.x using napalm #255

@mkhachfeh

Description

@mkhachfeh

Describe the bug
Hi All,

I was using salt-sproxy for a while with our Juniper-based network, and all was working fine.

I started to find some problems with the new Junos code we use (18.x)

I can communicate with the device (net.connected, net.cli, grains....etc), but I can't configure anything on it via neither net.cli nor net.load_config/template, the device isn't reporting back any diff, and sees that the new configs are already configured, even if the new set file contains new configs not configured on the device for sure.

And I can see nothing coming back from the device via rpc-reply while debugging the request.

I tested that for multiple Juniper devices with the same code, and all are the same wrong behaviour.

For example:
- The working operational scenario:

salt-sproxy '10-220-1-9' net.cli 'show system license'
10-220-1-9:
    ----------
    comment:
    out:
        ----------
        show system license:

            License usage: none

            Licenses installed: none
    result:
        True

- The non-working configurational scenario 1, again: vlan888 isn't configured on the device 100%:

salt-sproxy '10-220-1-9' net.load_config text='set vlans vlan888 vlan-id 888'
10-220-1-9:
    ----------
    already_configured:
        True
    comment:
        Already configured.
    diff:
    loaded_config:
    result:
        True


mkh@sw220-1-9> show configuration vlans vlan888

{master:0}
mkh@sw220-1-9>

- The non-working configurational scenario 2:

root@/etc/pillar# cat /etc/salt/ext/REQ.set
set vlans vlan888 vlan-id 888


root@/etc/pillar# salt-sproxy '10-220-1-9' net.load_config filename='/etc/salt/ext/REQ.set'
10-220-1-9:
    ----------
    already_configured:
        True
    comment:
        Already configured.
    diff:
    loaded_config:
    result:
        True


mkh@sw220-1-9> show configuration vlans vlan888

{master:0}
mkh@sw220-1-9>

Within debug what I see during that is the following:

[INFO    ] [host 10.220.1.9 session-id 82730] Requesting 'ExecuteRpc'
[DEBUG   ] [host 10.220.1.9 session-id 82730] queueing <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:ff66d123-c081-487d-971f-5d7e2932524a"><lock-configuration/></nc:rpc>
[DEBUG   ] [host 10.220.1.9 session-id 82730] Sync request, will wait for timeout=60
[DEBUG   ] [host 10.220.1.9 session-id 82730] Sending message
[INFO    ] [host 10.220.1.9 session-id 82730] Sending:
<?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:ff66d123-c081-487d-971f-5d7e2932524a"><lock-configuration/></nc:rpc>]]>]]>
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[INFO    ] [host 10.220.1.9 session-id 82730] Received message from host
[DEBUG   ] [host 10.220.1.9 session-id 82730] Received:
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:ff66d123-c081-487d-971f-5d7e2932524a">
</rpc-reply>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to different listeners: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:ff66d123-c081-487d-971f-5d7e2932524a">
</rpc-reply>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.operations.rpc.RPCReplyListener object at 0x7fa31e732670>
[DEBUG   ] [host 10.220.1.9 session-id 82730] Delivering to <ncclient.operations.third_party.juniper.rpc.ExecuteRpc object at 0x7fa31efeac10>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.transport.session.NotificationHandler object at 0x7fa31e644130>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <jnpr.junos.device.DeviceSessionListener object at 0x7fa31e628370>
[INFO    ] [host 10.220.1.9 session-id 82730] Requesting 'ExecuteRpc'
[DEBUG   ] [host 10.220.1.9 session-id 82730] queueing <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:5f53421e-6d44-4c58-91e1-c19ccea38b1e"><load-configuration format="text" action="set"><configuration-set>set vlans vlan888 vlan-id 888
</configuration-set></load-configuration></nc:rpc>
[DEBUG   ] [host 10.220.1.9 session-id 82730] Sync request, will wait for timeout=60
[DEBUG   ] [host 10.220.1.9 session-id 82730] Sending message
[INFO    ] [host 10.220.1.9 session-id 82730] Sending:
<?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:5f53421e-6d44-4c58-91e1-c19ccea38b1e"><load-configuration format="text" action="set"><configuration-set>set vlans vlan888 vlan-id 888
</configuration-set></load-configuration></nc:rpc>]]>]]>
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[INFO    ] [host 10.220.1.9 session-id 82730] Received message from host
[DEBUG   ] [host 10.220.1.9 session-id 82730] Received:
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:5f53421e-6d44-4c58-91e1-c19ccea38b1e">
<load-configuration-results>
<ok/>
</load-configuration-results>
</rpc-reply>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to different listeners: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:5f53421e-6d44-4c58-91e1-c19ccea38b1e">
<load-configuration-results>
<ok/>
</load-configuration-results>
</rpc-reply>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.operations.rpc.RPCReplyListener object at 0x7fa31e732670>
[DEBUG   ] [host 10.220.1.9 session-id 82730] Delivering to <ncclient.operations.third_party.juniper.rpc.ExecuteRpc object at 0x7fa31f160fd0>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.transport.session.NotificationHandler object at 0x7fa31e644130>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <jnpr.junos.device.DeviceSessionListener object at 0x7fa31e628370>
[INFO    ] [host 10.220.1.9 session-id 82730] Requesting 'ExecuteRpc'
[DEBUG   ] [host 10.220.1.9 session-id 82730] queueing <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:aa19bafc-a8b9-42ee-a9ae-f18de0448176"><get-configuration compare="rollback" rollback="0" format="text"/></nc:rpc>
[DEBUG   ] [host 10.220.1.9 session-id 82730] Sync request, will wait for timeout=60
[DEBUG   ] [host 10.220.1.9 session-id 82730] Sending message
[INFO    ] [host 10.220.1.9 session-id 82730] Sending:
<?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:aa19bafc-a8b9-42ee-a9ae-f18de0448176"><get-configuration compare="rollback" rollback="0" format="text"/></nc:rpc>]]>]]>
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[INFO    ] [host 10.220.1.9 session-id 82730] Received message from host
[DEBUG   ] [host 10.220.1.9 session-id 82730] Received:
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:aa19bafc-a8b9-42ee-a9ae-f18de0448176">
<configuration-information>
<configuration-output>
</configuration-output>
</configuration-information>
</rpc-reply>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to different listeners: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:aa19bafc-a8b9-42ee-a9ae-f18de0448176">
<configuration-information>
<configuration-output>
</configuration-output>
</configuration-information>
</rpc-reply>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.operations.rpc.RPCReplyListener object at 0x7fa31e732670>
[DEBUG   ] [host 10.220.1.9 session-id 82730] Delivering to <ncclient.operations.third_party.juniper.rpc.ExecuteRpc object at 0x7fa31f0edb80>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.transport.session.NotificationHandler object at 0x7fa31e644130>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <jnpr.junos.device.DeviceSessionListener object at 0x7fa31e628370>
[DEBUG   ] Discarding the config
[DEBUG   ] {'result': True, 'comment': '', 'already_configured': False, 'loaded_config': '', 'diff': ''}
[INFO    ] [host 10.220.1.9 session-id 82730] Requesting 'ExecuteRpc'
[DEBUG   ] [host 10.220.1.9 session-id 82730] queueing <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:4ef1fa52-c32d-49c4-aab2-e2db867a3761"><load-configuration compare="rollback" rollback="0"/></nc:rpc>
[DEBUG   ] [host 10.220.1.9 session-id 82730] Sync request, will wait for timeout=60
[DEBUG   ] [host 10.220.1.9 session-id 82730] Sending message
[INFO    ] [host 10.220.1.9 session-id 82730] Sending:
<?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:4ef1fa52-c32d-49c4-aab2-e2db867a3761"><load-configuration compare="rollback" rollback="0"/></nc:rpc>]]>]]>
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[INFO    ] [host 10.220.1.9 session-id 82730] Received message from host
[DEBUG   ] [host 10.220.1.9 session-id 82730] Received:
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:4ef1fa52-c32d-49c4-aab2-e2db867a3761">
<load-configuration-results>
<ok/>
</load-configuration-results>
</rpc-reply>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to different listeners: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:4ef1fa52-c32d-49c4-aab2-e2db867a3761">
<load-configuration-results>
<ok/>
</load-configuration-results>
</rpc-reply>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.operations.rpc.RPCReplyListener object at 0x7fa31e732670>
[DEBUG   ] [host 10.220.1.9 session-id 82730] Delivering to <ncclient.operations.third_party.juniper.rpc.ExecuteRpc object at 0x7fa31f160f70>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.transport.session.NotificationHandler object at 0x7fa31e644130>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <jnpr.junos.device.DeviceSessionListener object at 0x7fa31e628370>
[INFO    ] [host 10.220.1.9 session-id 82730] Requesting 'ExecuteRpc'
[DEBUG   ] [host 10.220.1.9 session-id 82730] queueing <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:ddb0f24c-387a-4366-9703-9a618e7ad6f4"><unlock-configuration/></nc:rpc>
[DEBUG   ] [host 10.220.1.9 session-id 82730] Sync request, will wait for timeout=60
[DEBUG   ] [host 10.220.1.9 session-id 82730] Sending message
[INFO    ] [host 10.220.1.9 session-id 82730] Sending:
<?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:ddb0f24c-387a-4366-9703-9a618e7ad6f4"><unlock-configuration/></nc:rpc>]]>]]>
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[INFO    ] [host 10.220.1.9 session-id 82730] Received message from host
[DEBUG   ] [host 10.220.1.9 session-id 82730] Received:
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:ddb0f24c-387a-4366-9703-9a618e7ad6f4">
</rpc-reply>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to different listeners: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:ddb0f24c-387a-4366-9703-9a618e7ad6f4">
</rpc-reply>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.operations.rpc.RPCReplyListener object at 0x7fa31e732670>
[DEBUG   ] [host 10.220.1.9 session-id 82730] Delivering to <ncclient.operations.third_party.juniper.rpc.ExecuteRpc object at 0x7fa31f0ed430>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <ncclient.transport.session.NotificationHandler object at 0x7fa31e644130>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to listener: <jnpr.junos.device.DeviceSessionListener object at 0x7fa31e628370>
[INFO    ] [host 10.220.1.9 session-id 82730] Requesting 'CloseSession'
[DEBUG   ] [host 10.220.1.9 session-id 82730] queueing <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:f8b65a65-5563-46b3-a7a1-36f2da78487e"><nc:close-session/></nc:rpc>
[DEBUG   ] [host 10.220.1.9 session-id 82730] Sync request, will wait for timeout=60
[DEBUG   ] [host 10.220.1.9 session-id 82730] Sending message
[INFO    ] [host 10.220.1.9 session-id 82730] Sending:
<?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:f8b65a65-5563-46b3-a7a1-36f2da78487e"><nc:close-session/></nc:rpc>]]>]]>
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[DEBUG   ] [host 10.220.1.9 session-id 82730] parsing netconf v1.0
[INFO    ] [host 10.220.1.9 session-id 82730] Received message from host
[DEBUG   ] [host 10.220.1.9 session-id 82730] Received:
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:f8b65a65-5563-46b3-a7a1-36f2da78487e">
<ok/>
</rpc-reply>
[DEBUG   ] [host 10.220.1.9 session-id 82730] dispatching message to different listeners: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/18.2R3/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:f8b65a65-5563-46b3-a7a1-36f2da78487e">
<ok/>
</rpc-reply>

Versions Report

Salt Version:
           Salt: 3003.3
    Salt SProxy: 2021.6.1

Dependency Versions:
        Ansible: Not Installed
           cffi: 1.14.6
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 3.0.2
     junos-eznc: 2.6.3
       jxmlease: 1.0.3
        libgit2: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 1.0.2
         NAPALM: 3.3.1
       ncclient: 0.6.9
        Netmiko: 3.4.0
       paramiko: 2.8.0
      pycparser: 2.20
       pycrypto: Not Installed
   pycryptodome: 3.11.0
         pyeapi: 0.8.4
         pygit2: Not Installed
       PyNetBox: Not Installed
          PyNSO: Not Installed
         Python: 3.8.10 (default, Sep 28 2021, 16:10:42)
   python-gnupg: Not Installed
         PyYAML: 5.4.1
          PyZMQ: 22.3.0
            scp: 0.14.1
          smmap: Not Installed
        textfsm: 1.1.2
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.3.4`

Just last note that the same set file+ net commands are working for another Junos code different than 18, tested on 20.4 and 15.1 with no similar problem

Wish it is clear, and thanks in advance.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions