|
| 1 | +## 2024-03-25 Version 2.7.0 |
| 2 | + |
| 3 | +--- |
| 4 | + |
| 5 | +### Features |
| 6 | + |
| 7 | +* Add a GeoIP processor. ([#253](https://github.com/opensearch-project/data-prepper/issues/253), [#3941](https://github.com/opensearch-project/data-prepper/issues/3941), [#3942](https://github.com/opensearch-project/data-prepper/issues/3942)) |
| 8 | +* Flatten json processor ([#4128](https://github.com/opensearch-project/data-prepper/issues/4128)) |
| 9 | +* Add select_entries processor ([#4147](https://github.com/opensearch-project/data-prepper/pull/4147)) |
| 10 | +* Decompress processor ([#4016](https://github.com/opensearch-project/data-prepper/issues/4016)) |
| 11 | +* Support parsing of XML fields in Events ([#4165](https://github.com/opensearch-project/data-prepper/issues/4165), [#4024](https://github.com/opensearch-project/data-prepper/issues/4024)) |
| 12 | +* Processor for parsing Amazon Ion documents ([#3730](https://github.com/opensearch-project/data-prepper/issues/3730)) |
| 13 | +* Append values to lists in an event ([#4129](https://github.com/opensearch-project/data-prepper/issues/4129)) |
| 14 | +* MapToList processor ([#3935](https://github.com/opensearch-project/data-prepper/issues/3935)) |
| 15 | +* Date processor to convert from epoch_second, epoch_milli, or epoch_nano ([#2929](https://github.com/opensearch-project/data-prepper/issues/2929), [#4076](https://github.com/opensearch-project/data-prepper/issues/4076)) |
| 16 | +* Support reading of old image for delete events on DynamoDB source ([#4261](https://github.com/opensearch-project/data-prepper/issues/4261)) |
| 17 | +* Add string truncate processor to the family of mutate string processor ([#3925](https://github.com/opensearch-project/data-prepper/issues/3925)) |
| 18 | +* Add join function ([#4075](https://github.com/opensearch-project/data-prepper/pull/4075)) |
| 19 | + |
| 20 | + |
| 21 | +### Enhancements |
| 22 | + |
| 23 | +* Support format expressions for routing in the opensearch sink ([#3833](https://github.com/opensearch-project/data-prepper/issues/3833)) |
| 24 | +* Allow . and @ characters to be part of json pointer in expressions ([#4130](https://github.com/opensearch-project/data-prepper/pull/4130)) |
| 25 | +* Support maximum request length configurations in the HTTP and OTel sources ([#3931](https://github.com/opensearch-project/data-prepper/issues/3931)) |
| 26 | +* Provide a config option to do node local aggregation ([#4306](https://github.com/opensearch-project/data-prepper/pull/4306)) |
| 27 | +* Allow peer forwarder to skip sending events to remote peer ([#3996](https://github.com/opensearch-project/data-prepper/issues/3996)) |
| 28 | +* Include encrypted data key in Kafka buffer message. ([#3655](https://github.com/opensearch-project/data-prepper/issues/3655)) |
| 29 | +* Support larger message sizes in Kafka Buffer ([#3916](https://github.com/opensearch-project/data-prepper/pull/3916)) |
| 30 | +* Modify S3 Source to allow multiple SQS workers ([#4239](https://github.com/opensearch-project/data-prepper/pull/4239)) |
| 31 | +* Add support for tracking performance of individual Events in the grok processor ([#4196](https://github.com/opensearch-project/data-prepper/issues/4196)) |
| 32 | +* Support codec on the file source to help with testing ([#4018](https://github.com/opensearch-project/data-prepper/issues/4018)) |
| 33 | +* Provide a delay processor to put a delay in the processor for debugging and testing ([#3938](https://github.com/opensearch-project/data-prepper/issues/3938)) |
| 34 | +* Support ByteCount in plugin parser ([#3191](https://github.com/opensearch-project/data-prepper/issues/3191)) |
| 35 | +* Add Buffer Latency Metric ([#4237](https://github.com/opensearch-project/data-prepper/pull/4237)) |
| 36 | +* Adds an append mode to the file sink ([#3687](https://github.com/opensearch-project/data-prepper/issues/3687)) |
| 37 | + |
| 38 | + |
| 39 | +### Bug Fixes |
| 40 | + |
| 41 | +* Attempting to evaluate if a key is null throws an Exception if the value is a List<String> for conditional expressions ([#4109](https://github.com/opensearch-project/data-prepper/issues/4109)) |
| 42 | +* Data Prepper process threads stop when processors throw exceptions ([#4103](https://github.com/opensearch-project/data-prepper/issues/4103)) |
| 43 | +* Upsert action requires existing document in OpenSearch ([#4036](https://github.com/opensearch-project/data-prepper/issues/4036)) |
| 44 | +* Many Grok failures do not tag events ([#4031](https://github.com/opensearch-project/data-prepper/issues/4031)) |
| 45 | +* Using update, upsert, or delete actions without specifying document_id crashes the pipeline with NPE ([#3988](https://github.com/opensearch-project/data-prepper/issues/3988)) |
| 46 | +* OpenSearch Sink upsert action fails to create new document if it doesn't exist already ([#3934](https://github.com/opensearch-project/data-prepper/issues/3934)) |
| 47 | +* DynamoDb source global state not found for export ([#3579](https://github.com/opensearch-project/data-prepper/issues/3579)) |
| 48 | +* Missing Configuration details in Kafka documentation ([#3157](https://github.com/opensearch-project/data-prepper/issues/3157)) |
| 49 | +* File Source fails to process large files. ([#707](https://github.com/opensearch-project/data-prepper/issues/707)) |
| 50 | +* Add key_value_when conditional to key_value processor ([#4246](https://github.com/opensearch-project/data-prepper/pull/4246)) |
| 51 | +* Adds Kafka producer metrics for buffer usage ([#4139](https://github.com/opensearch-project/data-prepper/pull/4139)) |
| 52 | +* Throw a more useful error when the S3 source is unable to determine bucket ownership ([#4021](https://github.com/opensearch-project/data-prepper/pull/4021)) |
| 53 | +* Add sts_header_overrides to s3 dlq configuration ([#3845](https://github.com/opensearch-project/data-prepper/pull/3845)) |
| 54 | +* Delay reading from the Kafka buffer as long as the circuit breaker is open ([#4135](https://github.com/opensearch-project/data-prepper/pull/4135)) |
| 55 | +* Use timer for sink latency metrics ([#4174](https://github.com/opensearch-project/data-prepper/pull/4174)) |
| 56 | +* Fix bug where process worker would shut down if a processor drops all events ([#4262](https://github.com/opensearch-project/data-prepper/pull/4262)) |
| 57 | +* Send acknowledgements to source when events are forwarded to remote peer ([#4305](https://github.com/opensearch-project/data-prepper/pull/4305)) |
| 58 | +* Injecting timestamp in index name that is not a suffix throws IllegalArgumentException ([#3957](https://github.com/opensearch-project/data-prepper/issues/3957)) |
| 59 | + |
| 60 | + |
| 61 | +### Security |
| 62 | + |
| 63 | +* Fixes CVE-2024-29133 ([#4314](https://github.com/opensearch-project/data-prepper/issues/4314)) |
| 64 | +* Fixes CVE-2024-29131 ([#4313](https://github.com/opensearch-project/data-prepper/issues/4313)) |
| 65 | +* Fixes CVE-2023-52428 ([#4296](https://github.com/opensearch-project/data-prepper/issues/4296)) |
| 66 | +* Fixes CVE-2024-23944 ([#4290](https://github.com/opensearch-project/data-prepper/issues/4290)) |
| 67 | +* Fixes CVE-2023-51775 ([#4282](https://github.com/opensearch-project/data-prepper/issues/4282)) |
| 68 | +* Fixes CVE-2024-22201 ([#4186](https://github.com/opensearch-project/data-prepper/issues/4186)) |
| 69 | +* Fixes CVE-2024-25710 ([#4164](https://github.com/opensearch-project/data-prepper/issues/4164)) |
| 70 | +* Fixes CVE-2024-26308 ([#4163](https://github.com/opensearch-project/data-prepper/issues/4163)) |
| 71 | +* Fixes CVE-2024-21634 ([#3926](https://github.com/opensearch-project/data-prepper/issues/3926)) |
| 72 | +* Fixes CVE-2023-50570 ([#3870](https://github.com/opensearch-project/data-prepper/issues/3870)) |
| 73 | +* Fixes CVE-2023-3635 ([#3068](https://github.com/opensearch-project/data-prepper/issues/3068)) |
| 74 | + |
| 75 | + |
| 76 | +### Maintenance |
| 77 | + |
| 78 | +* Create Kafka buffer integration tests for KMS ([#3980](https://github.com/opensearch-project/data-prepper/issues/3980), [#4040](https://github.com/opensearch-project/data-prepper/issues/4040)) |
| 79 | +* Fixes Dependabot updates are not configured for all projects ([#3301](https://github.com/opensearch-project/data-prepper/issues/3301)) |
| 80 | + |
0 commit comments