Release notes, deprecations, breaking for 9.0.0 (#17507)

(cherry picked from commit d66a2cf)

Author: karenzone

Diff for: docs/release-notes/breaking-changes.md

@@ -17,7 +17,10 @@ Before you upgrade, carefully review the Logstash breaking changes and take the
 
 ## 9.0.0 [logstash-900-breaking-changes]
 
-### Changes to SSL settings in {{ls}} plugins [ssl-settings-9.0]
+
+:::::{dropdown} Changes to SSL settings in {{ls}} plugins
+:name: ssl-deprecations-9.0.0
+
 We’ve removed deprecated SSL settings in some {{ls}} plugins, and have replaced them with updated settings. If your plugin configuration contains any of these obsolete options, the plugin may fail to start.
 
 ::::{dropdown} `logstash-input-beats`
@@ -34,7 +37,6 @@ We’ve removed deprecated SSL settings in some {{ls}} plugins, and have replace
 
 ::::
 
-
 ::::{dropdown} `logstash-input-elastic_agent`
 :name: input-elastic_agent-ssl-9.0
 
@@ -193,72 +195,84 @@ We’ve removed deprecated SSL settings in some {{ls}} plugins, and have replace
 | ssl_cert | [`ssl_certificate`](logstash-docs-md://lsr/plugins-outputs-tcp.md#plugins-outputs-tcp-ssl_certificate) |
 | ssl_enable | [`ssl_enabled`](logstash-docs-md://lsr/plugins-outputs-tcp.md#plugins-outputs-tcp-ssl_enabled) |
 | ssl_verify | [`ssl_client_authentication`](logstash-docs-md://lsr/plugins-outputs-tcp.md#plugins-outputs-tcp-ssl_client_authentication) in `server` mode and [`ssl_verification_mode`](logstash-docs-md://lsr/plugins-outputs-tcp.md#plugins-outputs-tcp-ssl_verification_mode) in `client` mode |
-
 ::::
+:::::
 
-
-### Pipeline buffer type defaults to `heap`[pipeline-buffer-type]
+::::{dropdown} Pipeline buffer type defaults to `heap`
+:name: pipeline-buffer-type]
 
 We've improved memory configuration for certain {{ls}} plugins.
 Input plugins such as `elastic_agent`, `beats`, `tcp`, and `http` allocate buffers in Java memory to read events from the network.
 The default allocation method is `direct` memory rather than `heap` memory to simplify configuration, and to help facilitate debugging memory usage problems through the analysis of heap dumps.
 If you need to re-enable the previous behavior, change the `pipeline.buffer.type` setting in [logstash.yml](/reference/logstash-settings-file.md).
 Check out [off-heap-buffers-allocation](/reference/jvm-settings.md#off-heap-buffers-allocation) for details. [#16500](https://github.com/elastic/logstash/pull/16500)
+::::
 
-### {{ls}} modules removed [removed-modules]
+::::{dropdown} {{ls}} modules removed 
+:name: removed-modules
 
 We have removed the {{ls}} modules framework, and encourage users to try Elastic Integrations
 This includes the netflow, azure and arcsight modules, and the modules framework as a whole. [#16794](https://github.com/elastic/logstash/pull/16794)
+::::
 
 
-### Deprecated configuration settings removed [removed-params]
+::::{dropdown} Deprecated configuration settings removed 
+:name:removed-params
 
 We have removed support for previously deprecated configuration settings:
 
-#### `http.*` prefixed settings for the {{ls}} API
-When configuring the {{ls}} API, settings prefixed by `http.*` have been replaced by the equivalent settings prefixed with `api.*`. [#16552](https://github.com/elastic/logstash/pull/16552)
-
-#### `event_api.tags.illegal`
+- **`http.*` prefixed settings for the {{ls}} API.** Settings prefixed by `http.*` have been replaced by the equivalent settings prefixed with `api.*`. [#16552](https://github.com/elastic/logstash/pull/16552)
 
+- **`event_api.tags.illegal`**
 Any events that include field named tags automatically rename the field _tags to avoid any clash
 with the reserved {{ls}} tags field. 
 Instead, {{ls}} generates `_tagsparsefailure` in the event `tags` and the illegal value is written to the `_tags` field. [#16461](https://github.com/elastic/logstash/pull/16461)
+::::
 
-
-### Ingest converter removed [removed-ingest-converter]
+::::{dropdown} Ingest converter removed 
+:name: removed-ingest-converter
 
 The ingest converter, which was used to convert ingest pipelines to {{ls}} pipelines, has been removed. [#16453](https://github.com/elastic/logstash/pull/16453)
+
 The `logstash-filter-elastic_integration` plugin offers similar functionality, and can help you use [Logstash to extend Elastic integrations](/reference/using-logstash-with-elastic-integrations.md).
+::::
 
-### JDK11 not supported [jdk-11-support-drop]
+::::{dropdown} JDK11 not supported
+:name: jdk-11-support-drop
 
 JDK17 is the minimum version of the JDK required to run Logstash.
 For the best experience, we still recommend running {{ls}} using the bundled-jdk. 
 See [Logstash JVM requirements](/reference/getting-started-with-logstash.md#ls-jvm)
 for details. [#16443](https://github.com/elastic/logstash/pull/16443)
+::::
 
-### Docker base image now UBI9 based [docker-base-image-change]
+::::{dropdown} Docker base image now UBI9 based
+:name: docker-base-image-change
 
 The base image for {{ls}} on Docker has been changed from Ubuntu to UBI9.
 If you create a Docker image based on the {{ls}} image and rely on it being Ubuntu based, you need to change your derived image to take account of this change. 
 For example, if your derived docker image installs additional packages using a package manager, UBI9 uses `microdnf`, rather than `apt`.
 [#16599](https://github.com/elastic/logstash/pull/16599)
+::::
 
-
-### Cannot run {{ls}} as `superuser` by default [disallow-superuser]
+::::{dropdown} Cannot run {{ls}} as `superuser` by default
+:name: [disallow-superuser
 
 We've changed the default behavior to prevent users from accidentally running {{ls}} as a superuser.
 If you try to run {{ls}} as a superuser, it logs an error and fails to start, ensuring that users cannot run Logstash with elevated privileges by accident.
 
-You can change the value of the `allow_superuser` setting to `true` in [logstash.yml](/reference/logstash-settings-file.md) if you want to restore the previous behavior and allow {{ls}} to run with superuser privileges. [#16558(https://github.com/elastic/logstash/pull/16558)
+You can change the value of the `allow_superuser` setting to `true` in [logstash.yml](/reference/logstash-settings-file.md) if you want to restore the previous behavior and allow {{ls}} to run with superuser privileges. [#16558](https://github.com/elastic/logstash/pull/16558)
+::::
 
-### New setting required to continue using legacy internal monitoring [allow-legacy-monitoring]
+::::{dropdown} New setting required to continue using legacy internal monitoring
+:name: allow-legacy-monitoring
 
 To continue using deprecated internal collection to monitor {{ls}}, set `xpack.monitoring.allow_legacy_collection` to `true` in [logstash.yml](/reference/logstash-settings-file.md).
 We encourage you to move to [agent-driven monitoring](/reference/monitoring-logstash-with-elastic-agent.md), the latest, supported way to monitor Logstash [#16586](https://github.com/elastic/logstash/pull/16586)
+::::
 
-
-### Avoiding JSON log lines collision [avoid-collision-on-json-fields]
+::::{dropdown} Avoiding JSON log lines collision
+:name: avoid-collision-on-json-fields
 
 We've improved the way we deal with duplicate `message` fields in `json` documents.
 Some code paths that log in `json` produce log events that include multiple instances of the  `message` field. (The JSON codec plugin is one example.)
@@ -268,9 +282,14 @@ We recently introduced the option to fix duplicates, and made it the default beh
 To re-enable the previous behavior, change the `log.format.json.fix_duplicate_message_fields` setting in [logstash.yml](/reference/logstash-settings-file.md) to `false`.
 
 Check out [Logging in json format can write duplicate message fields](docs-content://troubleshoot/ingest/logstash.md) for more details about the issue. [#16578](https://github.com/elastic/logstash/pull/16578)
+::::
 
-### Enterprise_search integration plugin is deprecated [enterprise_search-deprecated-9.0]
+::::{dropdown} Enterprise_search integration plugin is removed from default Logstash install
+:name: enterprise_search-deprecated-9.0
 
-We’ve deprecated the {{ls}} Enterprise_search integration plugin, and its component App Search and Workplace Search plugins. These plugins will receive only security updates and critical fixes moving forward.
+We’ve removed the {{ls}} Enterprise_search integration plugin, and its component App Search and Workplace Search plugins from the default {{ls}} install. 
+These plugins will receive only security updates and critical fixes moving forward.
 
 We recommend using our native {{es}} tools for your Search use cases. For more details, please visit the [Search solution and use case documentation](docs-content://solutions/search.md).
+::::
+

Diff for: docs/release-notes/deprecations.md

@@ -5,7 +5,8 @@ navigation_title: "Deprecations"
 # Logstash deprecations [logstash-deprecations]
 Over time, certain Elastic functionality becomes outdated and is replaced or removed. To help with the transition, Elastic deprecates functionality for a period before removal, giving you time to update your applications. 
 
-Review the deprecated functionality for Logstash. While deprecations have no immediate impact, we strongly encourage you update your implementation after you upgrade. To learn how to upgrade, check out [Upgrade](docs-content://deploy-manage/upgrade.md).
+Review the deprecated functionality for Logstash. 
+While deprecations have no immediate impact, we strongly encourage you update your implementation after you upgrade.
 
 % ## Next version [logstash-versionnext-deprecations]
 
@@ -23,4 +24,6 @@ Review the deprecated functionality for Logstash. While deprecations have no imm
 % For more information, check [PR #](PR link).
 % **Impact**<br> Impact of deprecation. 
 % **Action**<br> Steps for mitigating deprecation impact.
-% ::::
+% ::::
+
+None at this time

Diff for: docs/release-notes/known-issues.md

@@ -2,6 +2,13 @@
 navigation_title: "Known issues"
 ---
 
-# Logstash known issues
+# Logstash known issues [logstash-known-issues]
 
-% What needs to be done: Write from scratch
+Known issues are significant defects or limitations that may impact your implementation. 
+These issues are actively being worked on and will be addressed in a future release. 
+Review known issues to help you make informed decisions, such as upgrading to a new version.
+
+
+## 9.0.0
+
+None at this time