Release v0.59.2 (#4362)

* Added `force_refresh` parameter for Assessment
([#4183](#4183)). The
assessment workflow has been enhanced with the introduction of a
`force-refresh` parameter, allowing users to forcefully rerun the
assessment and obtain new results, even if it was previously run. This
parameter, which defaults to `false`, provides users with more control
over the assessment workflow, enabling them to manually trigger a
refresh of the assessment data when needed. The `force-refresh`
parameter can be set to `true` to overwrite existing results, and its
usage is optional, allowing users to choose when to refresh the
assessment data. The addition of this parameter modifies the behavior of
the assessment workflow, which previously would not update the output of
previously run assessments. With this update, users can now force a
complete refresh of the assessment data, which can be useful in
scenarios where the assessment results need to be recalculated. However,
users are advised to use this option with caution, as the process can be
time-consuming and resource-intensive for large workspaces.
* Added SECURITY.md for vulnerability reporting policy
([#4357](#4357)). A security
policy has been introduced to govern the security of UCX, outlining
procedures for reporting and addressing vulnerabilities. According to
this policy, security updates will only be applied to the latest version
of UCX, with notable updates highlighted in release notes, and will not
be backported to earlier versions. To report a vulnerability, users
should email the details to a designated address or contact their
Databricks representative, and can expect an acknowledgement of receipt
within 48 hours, after which the reported vulnerabilities will be
reviewed and addressed promptly. Users are also encouraged to follow
security best practices, including using the latest released version of
UCX and reviewing recommended configurations and operational security
considerations in the UCX documentation, to ensure the secure use of
UCX.
* Added handling for PermisssionDenied exception so that a new dashboard
is created ([#4209](#4209)).
The dashboard management functionality has been enhanced to improve its
resilience and robustness, particularly in handling exceptions and
invalid dashboard states. When a `PermissionDenied` exception occurs
while accessing a dashboard, the system now creates a new dashboard
instead of attempting to access the existing one. Additionally, the code
can recreate a dashboard if it is trashed or its reference is corrupted.
To support these enhancements, new methods have been introduced to check
if a dashboard is a Redash dashboard, upgrade a Redash dashboard to
Lakeview, check if a dashboard is trashed, and recover an invalid
dashboard by deleting any dangling files. These changes have also
improved the readability and maintainability of the existing
functionality, making the dashboard management more reliable and robust.
* Convert WASBS to ABFSS experimental workflow
([#4031](#4031)). An
experimental workflow has been introduced to convert Azure Blob Storage
(WASBS) URLs to Azure Data Lake Storage Gen2 (ABFSS) URLs, enabling
users to modernize their storage paths while preserving table metadata.
This feature transforms URLs by changing the protocol and hostname
suffix, identifying tables that use WASBS, and updating them to the more
performant ABFSS format. The implementation includes new methods, such
as `wasbs_to_abfss` and `convert_wasbs_to_adls_gen2`, which utilize the
`urlparse` function and Spark session to crawl tables, filter those that
use WASBS, and update their locations to ABFSS. Additionally, existing
methods have been updated to include return types and docstrings for
better clarity and maintainability, and new test configurations have
been added to validate the conversion process, ensuring a seamless
migration path for users wanting to take advantage of Azure Data Lake
Storage Gen2's improved features.
* Fixed service principal not re-created while creating account groups
([#4360](#4360)). The
account group creation functionality has been enhanced to support
service principals as members, allowing for more comprehensive and
flexible group management. The code now checks for member references
starting with `Users` or `ServicePrincipals` and includes them in the
list of members to add, effectively enabling service principals to be
part of account groups. Additionally, the warning message for skipped
members has been updated to accurately reflect the expected types of
members, which can be users, service principals, or groups. This
modification resolves the issue of service principals not being created
during account group creation and ensures that account groups can
include service principals as members. Furthermore, a new reusable
function has been introduced to retrieve a group by its display name,
retrying for up to two minutes if the group is not found, and the test
suite has been updated to verify the correct creation of groups with
both users and service principals as members.

Author: gueniai

CHANGELOG.md

@@ -1,5 +1,14 @@
 # Version changelog
 
+## 0.59.2
+
+* Added `force_refresh` parameter for Assessment ([#4183](https://github.com/databrickslabs/ucx/issues/4183)). The assessment workflow has been enhanced with the introduction of a `force-refresh` parameter, allowing users to forcefully rerun the assessment and obtain new results, even if it was previously run. This parameter, which defaults to `false`, provides users with more control over the assessment workflow, enabling them to manually trigger a refresh of the assessment data when needed. The `force-refresh` parameter can be set to `true` to overwrite existing results, and its usage is optional, allowing users to choose when to refresh the assessment data. The addition of this parameter modifies the behavior of the assessment workflow, which previously would not update the output of previously run assessments. With this update, users can now force a complete refresh of the assessment data, which can be useful in scenarios where the assessment results need to be recalculated. However, users are advised to use this option with caution, as the process can be time-consuming and resource-intensive for large workspaces.
+* Added SECURITY.md for vulnerability reporting policy ([#4357](https://github.com/databrickslabs/ucx/issues/4357)). A security policy has been introduced to govern the security of UCX, outlining procedures for reporting and addressing vulnerabilities. According to this policy, security updates will only be applied to the latest version of UCX, with notable updates highlighted in release notes, and will not be backported to earlier versions. To report a vulnerability, users should email the details to a designated address or contact their Databricks representative, and can expect an acknowledgement of receipt within 48 hours, after which the reported vulnerabilities will be reviewed and addressed promptly. Users are also encouraged to follow security best practices, including using the latest released version of UCX and reviewing recommended configurations and operational security considerations in the UCX documentation, to ensure the secure use of UCX.
+* Added handling for PermisssionDenied exception so that a new dashboard is created ([#4209](https://github.com/databrickslabs/ucx/issues/4209)). The dashboard management functionality has been enhanced to improve its resilience and robustness, particularly in handling exceptions and invalid dashboard states. When a `PermissionDenied` exception occurs while accessing a dashboard, the system now creates a new dashboard instead of attempting to access the existing one. Additionally, the code can recreate a dashboard if it is trashed or its reference is corrupted. To support these enhancements, new methods have been introduced to check if a dashboard is a Redash dashboard, upgrade a Redash dashboard to Lakeview, check if a dashboard is trashed, and recover an invalid dashboard by deleting any dangling files. These changes have also improved the readability and maintainability of the existing functionality, making the dashboard management more reliable and robust.
+* Convert WASBS to ABFSS experimental workflow ([#4031](https://github.com/databrickslabs/ucx/issues/4031)). An experimental workflow has been introduced to convert Azure Blob Storage (WASBS) URLs to Azure Data Lake Storage Gen2 (ABFSS) URLs, enabling users to modernize their storage paths while preserving table metadata. This feature transforms URLs by changing the protocol and hostname suffix, identifying tables that use WASBS, and updating them to the more performant ABFSS format. The implementation includes new methods, such as `wasbs_to_abfss` and `convert_wasbs_to_adls_gen2`, which utilize the `urlparse` function and Spark session to crawl tables, filter those that use WASBS, and update their locations to ABFSS. Additionally, existing methods have been updated to include return types and docstrings for better clarity and maintainability, and new test configurations have been added to validate the conversion process, ensuring a seamless migration path for users wanting to take advantage of Azure Data Lake Storage Gen2's improved features.
+* Fixed service principal not re-created while creating account groups ([#4360](https://github.com/databrickslabs/ucx/issues/4360)). The account group creation functionality has been enhanced to support service principals as members, allowing for more comprehensive and flexible group management. The code now checks for member references starting with `Users` or `ServicePrincipals` and includes them in the list of members to add, effectively enabling service principals to be part of account groups. Additionally, the warning message for skipped members has been updated to accurately reflect the expected types of members, which can be users, service principals, or groups. This modification resolves the issue of service principals not being created during account group creation and ensures that account groups can include service principals as members. Furthermore, a new reusable function has been introduced to retrieve a group by its display name, retrying for up to two minutes if the group is not found, and the test suite has been updated to verify the correct creation of groups with both users and service principals as members.
+
+
 ## 0.59.1
 
 * Added documentation for [#3963](https://github.com/databrickslabs/ucx/issues/3963) ([#4020](https://github.com/databrickslabs/ucx/issues/4020)). The workflow assessment functionality has been enhanced with an experimental task that analyzes recently executed workflows for migration problems, providing links to relevant documentation and recommendations for addressing identified issues. This task, labeled as experimental, now only runs for workflows executed within the last 30 days, but users can opt to analyze all workflows by running a specific workflow. The assessment findings, including any migration problems detected, can be viewed in the assessment dashboard, offering a centralized location for monitoring and addressing potential issues, and helping users to ensure a smoother migration process.

src/databricks/labs/ucx/__about__.py

@@ -1,2 +1,2 @@
 # DO NOT MODIFY THIS FILE
-__version__ = "0.59.1"
+__version__ = "0.59.2"