Skip to content

Commit 5acda21

Browse files
authored
Merge pull request #797 from nautobot/release-v3.7.0
2 parents 8a19633 + c7feafa commit 5acda21

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

51 files changed

+2696
-1715
lines changed

.github/workflows/ci.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,8 @@ jobs:
103103
uses: "networktocode/gh-action-setup-poetry-environment@v6"
104104
with:
105105
poetry-version: "1.8.5"
106+
- name: "Pip install virtualenv to avoid issue with virtualenv --wheel"
107+
run: "~/.local/share/pypoetry/venv/bin/pip install virtualenv==20.30.0"
106108
- name: "Constrain Nautobot version and regenerate lock file"
107109
env:
108110
INVOKE_NAUTOBOT_SSOT_LOCAL: "true"
@@ -160,6 +162,8 @@ jobs:
160162
uses: "networktocode/gh-action-setup-poetry-environment@v6"
161163
with:
162164
poetry-version: "1.8.5"
165+
- name: "Pip install virtualenv to avoid issue with virtualenv --wheel"
166+
run: "~/.local/share/pypoetry/venv/bin/pip install virtualenv==20.30.0"
163167
- name: "Constrain Nautobot version and regenerate lock file"
164168
env:
165169
INVOKE_NAUTOBOT_SSOT_LOCAL: "true"

development/development.env

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ NAUTOBOT_SSOT_DEVICE42_HOST=""
7878
NAUTOBOT_SSOT_DEVICE42_USERNAME=""
7979
NAUTOBOT_SSOT_DEVICE42_PASSWORD=""
8080

81-
NAUTOBOT_SSOT_ENABLE_DNA_CENTER="False"
81+
NAUTOBOT_SSOT_ENABLE_DNA_CENTER="True"
8282
NAUTOBOT_DNAC_SSOT_DNA_CENTER_IMPORT_GLOBAL="True"
8383
NAUTOBOT_DNAC_SSOT_DNA_CENTER_IMPORT_MERAKIS="False"
8484
NAUTOBOT_DNAC_SSOT_DNA_CENTER_UPDATE_LOCATIONS="True"

docs/admin/integrations/infoblox_setup.md

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -140,11 +140,9 @@ The purpose of this setting is to provide flexibility, enabling you to:
140140
The setting is represented as a dictionary, where the keys are Infoblox network views, and the values are the corresponding Nautobot namespaces.
141141

142142
```json
143-
[
144-
{
145-
"default": "Global"
146-
}
147-
]
143+
{
144+
"default": "Global"
145+
}
148146
```
149147

150148
### Configuring Infoblox DNS View Mapping

docs/admin/release_notes/version_3.6.md

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
21
# v3.6 Release Notes
32

43
This document describes all new features and changes in the release. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
@@ -7,7 +6,7 @@ This document describes all new features and changes in the release. The format
76

87
The major thing to note about this release is that we've removed support for Python 3.8 from the project. There have been some additional features added to the Bootstrap and DNA Center integrations. In addition there have been a multitude of bugfixes and tweaks made to the project.
98

10-
## [v3.6.0 (2025-04-04)](https://github.com/nautobot/nautobot-app-ssot/releases/tag/v3.6.0)
9+
## [v3.6.0 (2025-04-05)](https://github.com/nautobot/nautobot-app-ssot/releases/tag/v3.6.0)
1110

1211
### Added
1312

@@ -63,3 +62,9 @@ The major thing to note about this release is that we've removed support for Pyt
6362
### Housekeeping
6463

6564
- Rebaked from the cookie `nautobot-app-v2.4.2`.
65+
66+
## New Contributors
67+
* @michalbil made their first contribution in https://github.com/nautobot/nautobot-app-ssot/pull/722
68+
* @justinbrink made their first contribution in https://github.com/nautobot/nautobot-app-ssot/pull/751
69+
70+
**Full Changelog**: https://github.com/nautobot/nautobot-app-ssot/compare/v3.5.0...v3.6.0
Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# v3.7 Release Notes
2+
3+
This document describes all new features and changes in the release. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
4+
5+
## Release Overview
6+
7+
This release focuses on bugfixes for the DNA Center, Citrix ADM, Bootstrap and Slurpit integrations along with some dependency updates.
8+
9+
## [v3.7.0 (2025-05-08)](https://github.com/nautobot/nautobot-app-ssot/releases/tag/v3.7.0)
10+
11+
### Added
12+
13+
- [#457](https://github.com/nautobot/nautobot-app-ssot/issues/457) - Added `sort_relationships()` helper function
14+
- [#457](https://github.com/nautobot/nautobot-app-ssot/issues/457) - Added tests for `sort_relationships()` helper function
15+
- [#457](https://github.com/nautobot/nautobot-app-ssot/issues/457) - Added call to `sort_relationships()` function in contrib `NautobotAdapter`
16+
17+
### Fixed
18+
19+
- [#708](https://github.com/nautobot/nautobot-app-ssot/issues/708) - Fixes Device Building, parent Area if the location_map feature is used.
20+
- [#708](https://github.com/nautobot/nautobot-app-ssot/issues/708) - Also reverted 724 as there should only be one host address ever found as I originally thought.
21+
- [#760](https://github.com/nautobot/nautobot-app-ssot/issues/760) - Fixed issue causing bootstrap scheduled_job to fail when updating the User field.
22+
- [#767](https://github.com/nautobot/nautobot-app-ssot/issues/767) - Fixed IPAddressToInterface model in DNA Center by adding mask_length as identifier. This should allow multiple IPAddresses with same host address.
23+
- [#772](https://github.com/nautobot/nautobot-app-ssot/issues/772) - The default value for the Network Views to Nautobot Namespace setting in the Infoblox integration should be a dictionary, not a list.
24+
- [#778](https://github.com/nautobot/nautobot-app-ssot/issues/778) - Fixed description and tags not updating on Prefix objects after creation.
25+
- [#780](https://github.com/nautobot/nautobot-app-ssot/issues/780) - Fixes syncing devices without a Lat/Long defined in Slurpit.
26+
- [#781](https://github.com/nautobot/nautobot-app-ssot/issues/781) - Fixes syncing devices if they do not have a site defined in Slurpit by adding a default location.
27+
- [#787](https://github.com/nautobot/nautobot-app-ssot/issues/787) - Add `SKIP_UNMATCHED_DST` to Slurpit sync.
28+
- [#793](https://github.com/nautobot/nautobot-app-ssot/issues/793) - Fixed search for parent Prefix of IPAddress to include Namespace to avoid getting multiple results in Citrix ADM integration.
29+
- [#793](https://github.com/nautobot/nautobot-app-ssot/issues/793) - Fixed DNA Center loading of Controller locations with missing parent.
30+
31+
### Dependencies
32+
33+
- [#709](https://github.com/nautobot/nautobot-app-ssot/issues/709) - Update packaging and ipfabric dependencies to allow newer versions to be used.
34+
- [#764](https://github.com/nautobot/nautobot-app-ssot/issues/764) - Update dependency for Device Lifecycle Management App to allow use of 3.x with the various integrations.

docs/dev/jobs.md

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -168,7 +168,7 @@ The methods [`calculate_diff`][nautobot_ssot.jobs.base.DataSyncBaseJob.calculate
168168
Optionally, on your Job class, also implement the [`lookup_object`][nautobot_ssot.jobs.base.DataSyncBaseJob.lookup_object], [`data_mapping`][nautobot_ssot.jobs.base.DataSyncBaseJob.data_mappings], and/or [`config_information`][nautobot_ssot.jobs.base.DataSyncBaseJob.config_information] APIs (to provide more information to the end user about the details of this Job), as well as the various metadata properties on your Job's Meta inner class. Refer to the example Jobs provided in this Nautobot app for examples and further details.
169169
Install your Job via any of the supported Nautobot methods (installation into the `JOBS_ROOT` directory, inclusion in a Git repository, or packaging as part of an app) and it should automatically become available!
170170

171-
### Extra Step: Implementing `create`, `update` and `delete`
171+
### Extra Step 1: Implementing `create`, `update` and `delete`
172172

173173
If you are synchronizing data _to_ Nautobot and not _from_ Nautobot, you can entirely skip this step. The `nautobot_ssot.contrib.NautobotModel` class provides this functionality automatically.
174174

@@ -179,3 +179,27 @@ If you need to perform the `create`, `update` and `delete` operations on the rem
179179

180180
!!! warning
181181
Special care should be taken when synchronizing new Devices with children Interfaces into a Nautobot instance that also defines Device Types with Interface components of the same name. When the new Device is created in Nautobot, its Interfaces will also be created as defined in the respective Device Type. As a result, when SSoT will attempt to create the children Interfaces loaded by the remote adapter, these will already exist in the target Nautobot system. In this scenario, if not properly handled, the sync will fail! Possible remediation steps may vary depending on the specific use-case, therefore this is left as an exercise to the reader/developer to solve for their specific context.
182+
183+
### Extra Step 2: Sorting Many-to-Many and Many-to-One Relationships
184+
185+
Many-to-many (M2M) and many-to-one (N:1) relationships from the one side are stored in the diffsync models as a list of dictionaries. One common issue is when the order of the source list doesn't match the order of the destination list leading to a false update during synchronization. Since lists of dictionaries are often sorted by one of the available keys, we need a way to identify which key to sort by.
186+
187+
The `contrib` module offers a built solution to this issue and automatically sorts them before calculating the diff. Identifying the key to sort lists of dictionaries by is done by adding Python annotations in the `TypedDict` definitions used in relationships and adding some metadata. Example:
188+
189+
```python
190+
from nautobot_ssot.contrib import NautobotModel
191+
from nautobot_ssot.contrib.typeddicts import SortKey
192+
from typing_extensions import Annotated, TypedDict
193+
194+
195+
class LocationModel(NautobotModel):
196+
197+
198+
class TagDict(TypedDict):
199+
"""Many-to-many relationship typed dict explaining which fields are interesting."""
200+
201+
name: Annotated[str, SortKey]
202+
description: Optional[str] = ""
203+
```
204+
205+
The SSoT process will look through the `TypedDict`, find the metadata containing the `SortKey` class, and then sort by that key on both the source and target adapters. It currently only supports

docs/user/modeling.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -167,10 +167,8 @@ Through us defining the model, Nautobot will now be able to dynamically load IP
167167
!!! note
168168
Although `Interface.ip_addresses` is a generic relation, there is only one content type (i.e. `ipam.ipaddress`) that may be related through this relation, therefore we don't have to specific this in any way.
169169

170-
171170
## Filtering Objects Loaded From Nautobot
172171

173-
174172
If you'd like to filter the objects loaded from the Nautobot, you can do so creating a `get_queryset` function in your model class and return your own queryset. Here is an example where the adapter would only load Tenant objects whose name starts with an "s".
175173

176174
```python

mkdocs.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,7 @@ nav:
149149
- Compatibility Matrix: "admin/compatibility_matrix.md"
150150
- Release Notes:
151151
- "admin/release_notes/index.md"
152+
- v3.7: "admin/release_notes/version_3.7.md"
152153
- v3.6: "admin/release_notes/version_3.6.md"
153154
- v3.5: "admin/release_notes/version_3.5.md"
154155
- v3.4: "admin/release_notes/version_3.4.md"

nautobot_ssot/contrib/adapter.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
from typing import DefaultDict, Dict, FrozenSet, Hashable, Tuple, Type, get_args
88

99
import pydantic
10-
from diffsync import Adapter
10+
from diffsync import DiffSync
1111
from diffsync.exceptions import ObjectCrudException
1212
from django.contrib.contenttypes.models import ContentType
1313
from django.db.models import Model
@@ -34,7 +34,7 @@
3434
ParameterSet = FrozenSet[Tuple[str, Hashable]]
3535

3636

37-
class NautobotAdapter(Adapter):
37+
class NautobotAdapter(DiffSync):
3838
"""
3939
Adapter for loading data from Nautobot through the ORM.
4040

nautobot_ssot/contrib/sorting.py

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
"""Functions for sorting DiffSync model lists ensuring they are sorted to prevent false actions."""
2+
3+
import sys
4+
5+
from diffsync import Adapter, DiffSyncModel
6+
from typing_extensions import TypedDict, get_type_hints
7+
8+
from nautobot_ssot.contrib.typeddicts import SortKey
9+
from nautobot_ssot.contrib.types import SortType
10+
11+
12+
def _is_sortable_field(attribute_type_hints) -> bool:
13+
"""Check if a DiffSync attribute is a sortable field."""
14+
minor_ver = sys.version_info[1]
15+
try:
16+
# For Python 3.9 and older
17+
if minor_ver <= 9:
18+
attr_name = attribute_type_hints._name # pylint: disable=protected-access
19+
else:
20+
attr_name = attribute_type_hints.__name__
21+
except AttributeError:
22+
return False
23+
24+
return str(attr_name) in [
25+
"list",
26+
"List",
27+
]
28+
29+
30+
def _get_sort_key_from_typed_dict(sortable_content_type) -> str:
31+
"""Get the dictionary key from a TypedDict if found."""
32+
for key, value in sortable_content_type.__annotations__.items():
33+
try:
34+
metadata = value.__metadata__
35+
except AttributeError:
36+
continue
37+
for entry in metadata:
38+
if entry == SortKey:
39+
return key
40+
return None
41+
42+
43+
def get_sortable_fields_from_model(model: DiffSyncModel) -> dict:
44+
"""Get a list of sortable fields and their sort key from a DiffSync model."""
45+
sortable_fields = {}
46+
model_type_hints = get_type_hints(model, include_extras=True)
47+
48+
for model_attribute_name in model._attributes: # pylint: disable=protected-access
49+
attribute_type_hints = model_type_hints.get(model_attribute_name)
50+
51+
if not _is_sortable_field(attribute_type_hints):
52+
continue
53+
54+
sortable_content_type = attribute_type_hints.__args__[0]
55+
56+
if issubclass(sortable_content_type, dict) or issubclass(sortable_content_type, TypedDict):
57+
sort_key = _get_sort_key_from_typed_dict(sortable_content_type)
58+
if not sort_key:
59+
continue
60+
sortable_fields[model_attribute_name] = {
61+
"sort_type": SortType.DICT,
62+
"sort_key": sort_key,
63+
}
64+
# Add additional items here
65+
66+
return sortable_fields
67+
68+
69+
def _sort_dict_attr(obj, attribute, key):
70+
"""Update the sortable attribute in a DiffSync object."""
71+
sorted_data = None
72+
if key:
73+
sorted_data = sorted(
74+
getattr(obj, attribute),
75+
key=lambda x: x[key],
76+
)
77+
else:
78+
sorted_data = sorted(getattr(obj, attribute))
79+
80+
if sorted_data:
81+
setattr(obj, attribute, sorted_data)
82+
return obj
83+
84+
85+
def sort_relationships(source: Adapter, target: Adapter):
86+
"""Sort relationships based on the metadata defined in the DiffSync model."""
87+
if not source or not target:
88+
return
89+
90+
models_to_sort = {}
91+
# Loop through target's top_level attribute to determine models with sortable attributes
92+
for model_name in getattr(target, "top_level", []):
93+
# Get the DiffSync Model
94+
diffsync_model = getattr(target, model_name)
95+
if not diffsync_model:
96+
continue
97+
98+
# Get sortable fields current model
99+
model_sortable_fields = get_sortable_fields_from_model(diffsync_model)
100+
if not model_sortable_fields:
101+
continue
102+
models_to_sort[model_name] = model_sortable_fields
103+
104+
# Loop through adapaters to sort models
105+
for adapter in (source, target):
106+
for model_name, attrs_to_sort in models_to_sort.items():
107+
for diffsync_obj in adapter.get_all(model_name):
108+
for attr_name, sort_data in attrs_to_sort.items():
109+
sort_type = sort_data["sort_type"]
110+
# Sort the data based on its sort type
111+
if sort_type == SortType.DICT:
112+
diffsync_obj = _sort_dict_attr(diffsync_obj, attr_name, sort_data["sort_key"])
113+
adapter.update(diffsync_obj)

0 commit comments

Comments
 (0)