Release 2.2.0 (#560)

* Canonical interfaces fix dictionary update behavior.

* Fixing argument reference.

* Fixing NXOS file write issue Python3

* Added mocking method for nxos

* add tests for and fix canonical name

* add optional arg, and re-order optional args

* create extend driver docs

* clean up docs, add to index

* update doc strings

* fix pylama

* fix rev update dict

* add section headers

* spelling

* more consitent test, fix st

* Fix nxos_ssh mac table processing bug

* adding optional arguments for load_replace_candidate  (#508)

* adding optional arguments for load_replace_candidate and load_merge_candidate

* adding optional argument 'autoComplete'

* adding optional argument 'autoComplete' changing failed tests

* making required changes as discussed

* changing the if condition

* correcting the typo

* Handle multiple spaces in device_id from show lldp neighbors

* Add test to get_lldp_neighbors with hostname with spaces

* Loop through loaded config to convert HEREDOC commands.  Fixes #519 (#520)

(eos) loop through loaded config to convert HEREDOC commands.  Fixes #519

* Add code to build documentation for napalm-ansible (#535)

* Add code to build documentation for napalm-ansible

* pep8

* Change placement in documentation tree

* Change formatting

* Fix formatting for return values

* Sort parameters alphabetically

* Text changes

* Add script to clone napalm-ansible repo

* remove partial install (#554)

* Update documentation to corrrect driver ref (#559)

* [docs] Add "-c /dev/null" arguments to py.test command (#561) (#562)

* Add "-c /dev/null" arguments to py.test command (#561)

Before the NAPALM tests were picked up when running from RTD.

* fix requirements filepaths

* LLDP code with space in the device id (#549)

* LLDP code with space in the device id

* LLDP fixed width for device_id field

Author: dbarrosop

Diff for: .gitignore

@@ -63,3 +63,7 @@ test/unit/test_devices.py
 
 report.json
 tags
+
+docs/integrations/ansible/modules/napalm_*/
+docs/integrations/ansible/modules/source/*.json
+docs/napalm_ansible_repo/

Diff for: MANIFEST.in

@@ -1,4 +1,4 @@
-include requirements/*
+include requirements*
 include napalm/*/templates/*.j2
 include napalm/*/utils/textfsm_templates/*.tpl
 include napalm/junos/utils/*.yml

Diff for: README.md

@@ -26,39 +26,10 @@ You can also watch a [live demo](https://youtu.be/93q-dHC0u0I) of NAPALM to see
 Install
 =======
 
-Full installation
------------------
-
-If you want to fully install NAPALM you can do it by executing:
-
 ```
 pip install napalm
 ```
 
-That will install all the drivers currently available.
-
-
-Partial Installation
---------------------
-
-If you want to install just a subset of the available modules you can just pick them as follows:
-
-```
-pip install --install-option="ios" napalm
-```
-
-To install multiple drivers:
-```
-pip install --install-option="eos" --install-option="junos" napalm
-```
-
-To add drivers to an existing instalation:
-```
-pip install --install-option="eos" --install-option="junos" --force-reinstall -U napalm
-```
-
-Check the ['Supported Network Operating Systems'](#supported-network-operating-systems) section for more information about supported modules.
-
 
 Upgrading
 =========
@@ -69,13 +40,6 @@ We plan to upgrade napalm as fast as possible. Adding new methods and bugfixes.
 pip install napalm -U
 ```
 
-or:
-
-```
-pip install --install-option="eos" --install-option="junos" napalm -U
-```
-
-
 We will be posting news on our slack channel and on Twitter.
 
 

Diff for: docs/ansible-module.j2

@@ -0,0 +1,64 @@
+{{ name }}
+=====================
+
+{{ doc['description'][0] }}
+
+
+Parameters
+----------
+
+.. raw:: html
+
+	<table border="1" class="docutils">
+		<th>Parameter</th>
+		<th>Required</th>
+		<th>Default</th>
+		<th>Choices</th>
+		<th>Description</th>
+	{% for p in doc['options']|sort %}
+			<tr>
+				<td>{{ p }}</td>
+				<td>{{ doc['options'][p]['required'] }}</td>
+				<td>N/A</td>
+				<td>N/A</td>
+				<td>{{ doc['options'][p]['description'][0] }}</td>
+			</tr>
+	{% endfor %}
+
+	</table>
+
+
+Examples
+--------
+
+.. code-block:: yaml
+
+	{% for line in example_lines %}
+	{{ line }}
+	{% endfor %}
+
+{% if return_values %}
+
+Return
+------
+.. raw:: html
+
+	<table border="1" class="docutils">
+		<th>Name</th>
+		<th>Description</th>
+		<th>Returned</th>
+		<th>Type</th>
+		<th>Sample</th>
+	{% for value in return_values %}
+			<tr>
+				<td>{{ value }}</td>
+				<td>{{ return_values[value].description }}</td>
+				<td>{{ return_values[value].returned }}</td>
+				<td>{{ return_values[value].type }}</td>
+				<td>{{ return_values[value].sample|replace('\n', '\\n') }}</td>
+			</tr>
+	{% endfor %}
+	</table>
+
+
+{% endif %}

Diff for: docs/build-ansible-module-docs.sh

@@ -0,0 +1,20 @@
+#!/bin/bash
+
+CWD=`pwd`
+MODULES_OUTPUT="$CWD/integrations/ansible/modules/source"
+
+
+
+git clone https://github.com/napalm-automation/napalm-ansible.git napalm_ansible_repo
+cd napalm_ansible_repo
+
+# Change to master after next napalm-ansible release
+git checkout develop
+
+pip install -r requirements-dev.txt
+pip install .
+py.test -c /dev/null
+cp module_docs/* $MODULES_OUTPUT/
+
+cd $CWD
+

Diff for: docs/conf.py

@@ -20,7 +20,7 @@
 import subprocess
 import sys
 
-
+from glob import glob
 from napalm.base import NetworkDriver
 from jinja2 import Environment, FileSystemLoader
 
@@ -81,7 +81,7 @@
 
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
-exclude_patterns = ['_build']
+exclude_patterns = ['_build', 'napalm_ansible_repo']
 
 # The reST default role (used for this markup: `text`) to use for all
 # documents.
@@ -306,6 +306,40 @@ def _merge_results(last, intermediate):
         return last
 
 
+def build_napalm_ansible_module_docs(app):
+    """Create documentation for Ansible modules."""
+
+    # Add script to clone napalm-ansible repo
+    status = subprocess.call("./build-ansible-module-docs.sh", stdout=sys.stdout, stderr=sys.stderr)
+
+    if status != 0:
+        print("Something bad happened when processing the Ansible modules.")
+        sys.exit(-1)
+
+    env = Environment(loader=FileSystemLoader("."))
+
+    modules_dir = './integrations/ansible/modules/source'
+    module_files = glob('{0}/*.json'.format(modules_dir))
+    for module_file in module_files:
+        with open(module_file, 'r') as f:
+            module = module_file.split('/')[-1].split('.')[0]
+            data = json.loads(f.read())
+            data['name'] = module
+
+        module_dir = './integrations/ansible/modules/{0}'.format(module)
+
+        try:
+            os.stat(module_dir)
+        except Exception:
+            os.mkdir(module_dir)
+
+        template_file = env.get_template("ansible-module.j2")
+        rendered_template = template_file.render(**data)
+
+        with open('{0}/index.rst'.format(module_dir), 'w') as f:
+            f.write(rendered_template)
+
+
 def build_getters_support_matrix(app):
     """Build the getters support matrix."""
     status = subprocess.call("./test.sh", stdout=sys.stdout, stderr=sys.stderr)
@@ -356,6 +390,8 @@ def build_getters_support_matrix(app):
 def setup(app):
     """Map methods to states of the documentation build."""
     app.connect('builder-inited', build_getters_support_matrix)
+    app.connect('builder-inited', build_napalm_ansible_module_docs)
 
 
 build_getters_support_matrix(None)
+build_napalm_ansible_module_docs(None)

Diff for: docs/index.rst

@@ -39,7 +39,7 @@ You can select the driver you need by doing the following:
    >>> get_network_driver('iosxr')
    <class napalm.iosxr.iosxr.IOSXRDriver at 0x10ec90050>
    >>> get_network_driver('junos')
-   <class napalm.ibm.ibm.IBMDriver at 0x10f8f61f0>
+   <class napalm.junos.junos.JunOSDriver at 0x10f8f61f0>
    >>> get_network_driver('nxos')
    <class napalm.nxos.nxos.NXOSDriver at 0x10f9304c8>
    >>> get_network_driver('ios')
@@ -60,6 +60,7 @@ Documentation
    base
    yang
    logs
+   integrations/index
    contributing/index
    development/index
    hackathons/index

Diff for: docs/installation/index.rst

@@ -5,7 +5,7 @@ Installation
 Full installation
 -----------------
 
-If you want to fully install NAPALM you can do it by executing:
+You can install napalm with pip:
 
 .. code-block:: bash
     
@@ -14,32 +14,6 @@ If you want to fully install NAPALM you can do it by executing:
 That will install all the drivers currently available.
 
 
-Partial Installation
---------------------
-
-If you want to install just a subset of the available modules you can just pick them as follows:
-
-.. code-block:: bash
-
-    pip install --install-option="eos" --install-option="junos" napalm
-
-That will install only the ``eos`` and the ``junos`` drivers. If you want to add an extra driver later you can use ``pip --force-reinstall -U`` to do it:
-
-.. code-block:: bash
-
-    pip install --install-option="ios" --force-reinstall -U napalm
-
-
-Note you can pass those options to a requirements file as well:
-
-.. code-block:: bash
-
-    # requrements.txt
-    napalm --install-option="ios" --install-option="eos"
-
-
-Check the `supported devices`_ section for more information on supported drivers.
-
 Dependencies
 ------------
 

Diff for: docs/integrations/ansible/index.rst

@@ -0,0 +1,9 @@
+napalm-ansible
+==============
+
+Below are the ansible modules which are currently available for NAPALM
+
+.. toctree::
+   :maxdepth: 1
+
+   modules/index

Diff for: docs/integrations/ansible/modules/index.rst

@@ -0,0 +1,10 @@
+modules
+=======
+
+Below are the ansible modules which are currently available for NAPALM
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+
+   napalm_*/index

Diff for: docs/integrations/ansible/modules/source/.placeholder

@@ -0,0 +1,9 @@
+Integrations
+============
+
+NAPALM can be integrated with automation frameworks such as Ansible or Salt. In order to use NAPALM with ansible you will need to install napalm-ansible.
+
+.. toctree::
+   :maxdepth: 1
+
+   ansible/index

Diff for: docs/integrations/index.rst

@@ -4,9 +4,14 @@ EOS
 Minimum Version
 ~~~~~~~~~~~~~~~
 
-
 To be able to support the ``compare_config`` method you will require to run at least EOS version `4.15.0F`.
 
+Multi-line/HEREDOC
+~~~~~~~~~~~~~~~~~~
+EOS configuration is loaded via ``pyeapi.eapilib.Node.run_commands()``, which by itself cannot handle multi-line commands
+such as ``banner motd``.  The helper function ``EOSDriver._load_config()`` will attempt to detect HEREDOC commands in the
+input configuration and convert them into a dictionary that eAPI understands
+
 Rollback
 ~~~~~~~~
 

Diff for: docs/support/eos.rst

@@ -108,23 +108,25 @@ NAPALM supports passing certain optional arguments to some drivers. To do that y
 List of supported optional arguments
 ____________________________________
 
-* :code:`port` (eos, iosxr, junos, ios, nxos) - Allows you to specify a port other than the default.
+* :code:`allow_agent` (ios, iosxr) - Paramiko argument, enable connecting to the SSH agent (default: ``False``).
+* :code:`alt_host_keys` (ios, iosxr) - If ``True``, host keys will be loaded from the file specified in ``alt_key_file``.
+* :code:`alt_key_file` (ios, iosxr) - SSH host key file to use (if ``alt_host_keys`` is ``True``).
+* :code:`auto_rollback_on_error` (ios) - Disable automatic rollback (certain versions of IOS support configure replace, but not rollback on error) (default: ``True``).
 * :code:`config_lock` (iosxr, junos) - Lock the config during open() (default: ``False``).
+* :code:`canonical_int` (ios) - Convert operational interface's returned name to canonical name (fully expanded name) (default: ``False``).
 * :code:`dest_file_system` (ios) - Destination file system for SCP transfers (default: ``flash:``).
-* :code:`auto_rollback_on_error` (ios) - Disable automatic rollback (certain versions of IOS support configure replace, but not rollback on error) (default: ``True``).
-* :code:`global_delay_factor` (ios) - Allow for additional delay in command execution (default: ``1``).
-* :code:`transport` (eos, ios, nxos) - Protocol to connect with (see `The transport argument`_ for more information).
 * :code:`enable_password` (eos) - Password required to enter privileged exec (enable) (default: ``''``).
+* :code:`global_delay_factor` (ios) - Allow for additional delay in command execution (default: ``1``).
+* :code:`ignore_warning` (junos) - Allows to set `ignore_warning` when loading configuration to avoid exceptions via junos-pyez. (default: ``False``).
+* :code:`keepalive` (junos, iosxr) - SSH keepalive interval, in seconds (default: ``30`` seconds).
+* :code:`key_file` (junos, ios, iosxr) - Path to a private key file. (default: ``False``).
+* :code:`port` (eos, iosxr, junos, ios, nxos) - Allows you to specify a port other than the default.
 * :code:`secret` (ios) - Password required to enter privileged exec (enable) (default: ``''``).
+* :code:`ssh_config_file` (junos, ios, iosxr) - File name of OpenSSH configuration file.
 * :code:`ssh_strict` (iosxr, ios) - Automatically reject unknown SSH host keys (default: ``False``, which means unknown SSH host keys will be accepted).
-* :code:`allow_agent` (ios, iosxr) - Paramiko argument, enable connecting to the SSH agent (default: ``False``).
+* :code:`transport` (eos, ios, nxos) - Protocol to connect with (see `The transport argument`_ for more information).
 * :code:`use_keys` (iosxr, ios, panos) - Paramiko argument, enable searching for discoverable private key files in ``~/.ssh/`` (default: ``False``).
-* :code:`key_file` (junos, ios, iosxr) - Path to a private key file. (default: ``False``).
-* :code:`ssh_config_file` (junos, ios, iosxr) - File name of OpenSSH configuration file.
-* :code:`alt_host_keys` (ios, iosxr) - If ``True``, host keys will be loaded from the file specified in ``alt_key_file``.
-* :code:`alt_key_file` (ios, iosxr) - SSH host key file to use (if ``alt_host_keys`` is ``True``).
-* :code:`keepalive` (junos, iosxr) - SSH keepalive interval, in seconds (default: ``30`` seconds).
-* :code:`ignore_warning` (junos) - Allows to set `ignore_warning` when loading configuration to avoid exceptions via junos-pyez. (default: ``False``).
+* :code:`eos_autoComplete` (eos) - Allows to set `autoComplete` when running commands. (default: ``None`` equivalent to ``False``)
 
 The transport argument
 ______________________

Diff for: docs/support/index.rst

@@ -4,7 +4,7 @@ TEST_RESULTS_PATH="$CWD/support/tests"
 
 if [ ! -f "report.json" ]; then
     set -e
-    pip install -r ../requirements/all -r ../requirements/dev
+    pip install -r ../requirements.txt -r ../requirements-dev.txt
 
     set +e
     py.test -c /dev/null --cov=./ -vs --json=report.json ../test*/*/test_getters.py