From c6bbf2e1552a13fa173a2c27a4245f449210b485 Mon Sep 17 00:00:00 2001 From: James Holgate Date: Mon, 2 Feb 2026 18:17:06 +0000 Subject: [PATCH 01/11] Document Compact Serde caveat with pipelines [CTT-676] There are known caveats to Compact Serialization used in Jet Pipelines; this PR updates documentation to address these. Fixes https://hazelcast.atlassian.net/browse/CTT-676 --- .../pipelines/pages/serialization.adoc | 75 +++++++++++++++++++ .../pages/compact-serialization.adoc | 6 ++ 2 files changed, 81 insertions(+) diff --git a/docs/modules/pipelines/pages/serialization.adoc b/docs/modules/pipelines/pages/serialization.adoc index 85a988af9..4b15fd3b4 100644 --- a/docs/modules/pipelines/pages/serialization.adoc +++ b/docs/modules/pipelines/pages/serialization.adoc @@ -134,6 +134,81 @@ src.mapUsingService(serviceFactory, (formatter, tstamp) -> formatter.format(Instant.ofEpochMilli(tstamp))); ``` +== Using Compact Serialization with Pipelines + +xref:serialization:compact-serialization.adoc[Compact serialization] provides an efficient, +schema-based serialization mechanism for your data objects. While Compact serialization +has the highest priority in Hazelcast's serialization service, there are important caveats +to understand when using it with Jet pipelines. + +=== Java Serializable Takes Precedence for Lambdas and Functions + +Pipeline definitions, including lambda expressions and function objects, are serialized +using standard Java serialization (`java.io.Serializable`). This is because the pipeline +definition itself must be sent to cluster members before the Hazelcast serialization +service is involved. + +When an object implements `Serializable`, Java serialization is used directly and does +not delegate to Hazelcast's serialization service. This means: + +* All fields of a `Serializable` object must also be `Serializable` +* Compact serializers registered for field types are not used during Java serialization +* This applies even if the field's class has a registered `CompactSerializer` + +=== Entry Processors and Captured Variables + +This behavior is particularly relevant when using `Sinks.mapWithEntryProcessor()`. +The `EntryProcessor` interface extends `Serializable`, so any custom entry processor +and all its fields must be Java serializable. + +Consider this example where `OrderStatus` has a registered Compact serializer: + +```java +// OrderStatus has a CompactSerializer registered, but this still fails +// because MergeEntryProcessor implements Serializable (via EntryProcessor) +public class MergeEntryProcessor implements EntryProcessor { + // This field must be Serializable, even though OrderStatus has a CompactSerializer + private final OrderStatus newStatus; + + public MergeEntryProcessor(OrderStatus newStatus) { + this.newStatus = newStatus; + } + + @Override + public Order process(Entry entry) { + Order order = entry.getValue(); + order.setStatus(newStatus); + entry.setValue(order); + return order; + } +} +``` + +In this case, `OrderStatus` must implement `Serializable` in addition to having a +Compact serializer, because Java's `ObjectOutputStream` does not know about Hazelcast's +Compact serialization. + +The same applies to lambdas that capture Compact-serializable variables: + +```java +// orderStatus has a CompactSerializer, but capturing it in a lambda +// requires it to also implement Serializable +OrderStatus status = new OrderStatus("SHIPPED"); +pipeline.readFrom(source) + .filter(order -> order.getStatus().equals(status)); +``` + +=== Workarounds + +To use Compact-serializable objects with pipelines: + +1. **Use service factories**: For non-serializable dependencies, use `mapUsingService()` +to create objects on the target member rather than capturing them in lambdas. + +2. **Implement both interfaces**: Have your classes implement `Serializable` in addition + to registering a Compact serializer. The Compact serializer is still used when + Hazelcast serializes objects for storage in maps or transmission between members. + == Serialization of Data Types The objects you store in xref:data-structures:distributed-data-structures.adoc[Hazelcast data structures] must be serializable. diff --git a/docs/modules/serialization/pages/compact-serialization.adoc b/docs/modules/serialization/pages/compact-serialization.adoc index bbe5f3768..e73bfdd63 100644 --- a/docs/modules/serialization/pages/compact-serialization.adoc +++ b/docs/modules/serialization/pages/compact-serialization.adoc @@ -1663,6 +1663,12 @@ public class Employee implements Serializable { } ---- + +NOTE: When using Jet pipelines, lambda expressions and certain pipeline components (such as +`EntryProcessor` implementations) are serialized using Java serialization, which does not +delegate to Hazelcast's serialization service. In these cases, captured objects must +implement `Serializable` even if they have Compact serializers registered. For details, +see xref:pipelines:serialization.adoc#using-compact-serialization-with-pipelines[Using Compact Serialization with Pipelines]. == Compact Serialization Binary Specification The binary specification of compact serialization is publicly available at xref:ROOT:compact-binary-specification.adoc[this page]. From 59487decda380976d027fa2a690ff5200133e2d8 Mon Sep 17 00:00:00 2001 From: Rob Swain Date: Tue, 3 Feb 2026 15:56:13 +0000 Subject: [PATCH 02/11] Apply suggestion from @Rob-Hazelcast --- docs/modules/serialization/pages/compact-serialization.adoc | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/modules/serialization/pages/compact-serialization.adoc b/docs/modules/serialization/pages/compact-serialization.adoc index e73bfdd63..e89946f77 100644 --- a/docs/modules/serialization/pages/compact-serialization.adoc +++ b/docs/modules/serialization/pages/compact-serialization.adoc @@ -1669,6 +1669,7 @@ NOTE: When using Jet pipelines, lambda expressions and certain pipeline componen delegate to Hazelcast's serialization service. In these cases, captured objects must implement `Serializable` even if they have Compact serializers registered. For details, see xref:pipelines:serialization.adoc#using-compact-serialization-with-pipelines[Using Compact Serialization with Pipelines]. + == Compact Serialization Binary Specification The binary specification of compact serialization is publicly available at xref:ROOT:compact-binary-specification.adoc[this page]. From a819d688768d48da57de11b4beae3bf93900f9e1 Mon Sep 17 00:00:00 2001 From: Rob Swain Date: Tue, 3 Feb 2026 16:02:20 +0000 Subject: [PATCH 03/11] copy edit --- docs/modules/pipelines/pages/serialization.adoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/modules/pipelines/pages/serialization.adoc b/docs/modules/pipelines/pages/serialization.adoc index 4b15fd3b4..d838c5fa1 100644 --- a/docs/modules/pipelines/pages/serialization.adoc +++ b/docs/modules/pipelines/pages/serialization.adoc @@ -151,9 +151,9 @@ service is involved. When an object implements `Serializable`, Java serialization is used directly and does not delegate to Hazelcast's serialization service. This means: -* All fields of a `Serializable` object must also be `Serializable` -* Compact serializers registered for field types are not used during Java serialization -* This applies even if the field's class has a registered `CompactSerializer` +* All fields of a `Serializable` object must also be `Serializable`. +* Compact serializers registered for field types are not used during Java serialization. +* This applies even if the field's class has a registered `CompactSerializer`. === Entry Processors and Captured Variables @@ -161,7 +161,7 @@ This behavior is particularly relevant when using `Sinks.mapWithEntryProcessor() The `EntryProcessor` interface extends `Serializable`, so any custom entry processor and all its fields must be Java serializable. -Consider this example where `OrderStatus` has a registered Compact serializer: +Consider this example, where `OrderStatus` has a registered Compact serializer: ```java // OrderStatus has a CompactSerializer registered, but this still fails From 340ff45de8bc81e0670412c159e4a56e5f2eefb8 Mon Sep 17 00:00:00 2001 From: James Holgate <130981049+JamesHazelcast@users.noreply.github.com> Date: Tue, 3 Feb 2026 18:37:42 +0000 Subject: [PATCH 04/11] Update docs/modules/pipelines/pages/serialization.adoc Co-authored-by: Rob Swain --- docs/modules/pipelines/pages/serialization.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/modules/pipelines/pages/serialization.adoc b/docs/modules/pipelines/pages/serialization.adoc index d838c5fa1..c3e6a43d1 100644 --- a/docs/modules/pipelines/pages/serialization.adoc +++ b/docs/modules/pipelines/pages/serialization.adoc @@ -198,7 +198,7 @@ pipeline.readFrom(source) .filter(order -> order.getStatus().equals(status)); ``` -=== Workarounds +=== Using Compact Serialization with Pipelines To use Compact-serializable objects with pipelines: From 514a04c7494684edb38473ca9dbbbeaeedfea8f6 Mon Sep 17 00:00:00 2001 From: James Holgate <130981049+JamesHazelcast@users.noreply.github.com> Date: Tue, 3 Feb 2026 18:37:52 +0000 Subject: [PATCH 05/11] Update docs/modules/pipelines/pages/serialization.adoc Co-authored-by: Rob Swain --- docs/modules/pipelines/pages/serialization.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/modules/pipelines/pages/serialization.adoc b/docs/modules/pipelines/pages/serialization.adoc index c3e6a43d1..a395e2d05 100644 --- a/docs/modules/pipelines/pages/serialization.adoc +++ b/docs/modules/pipelines/pages/serialization.adoc @@ -134,7 +134,7 @@ src.mapUsingService(serviceFactory, (formatter, tstamp) -> formatter.format(Instant.ofEpochMilli(tstamp))); ``` -== Using Compact Serialization with Pipelines +== Compact Serialization xref:serialization:compact-serialization.adoc[Compact serialization] provides an efficient, schema-based serialization mechanism for your data objects. While Compact serialization From 952a954647a6becba0c91e036612a119f8b99b1a Mon Sep 17 00:00:00 2001 From: James Holgate Date: Mon, 9 Feb 2026 15:00:20 +0000 Subject: [PATCH 06/11] Updates & new benchmark data Benchmarks: https://github.com/hazelcast/internal-benchmarks/pull/69 --- .../pipelines/pages/serialization.adoc | 66 ++++++++++--------- 1 file changed, 35 insertions(+), 31 deletions(-) diff --git a/docs/modules/pipelines/pages/serialization.adoc b/docs/modules/pipelines/pages/serialization.adoc index a395e2d05..3cb90ffc6 100644 --- a/docs/modules/pipelines/pages/serialization.adoc +++ b/docs/modules/pipelines/pages/serialization.adoc @@ -152,14 +152,12 @@ When an object implements `Serializable`, Java serialization is used directly an not delegate to Hazelcast's serialization service. This means: * All fields of a `Serializable` object must also be `Serializable`. -* Compact serializers registered for field types are not used during Java serialization. -* This applies even if the field's class has a registered `CompactSerializer`. +* Any Hazelcast serializers registered for field types are not used during Java serialization. +* This applies for Compact serialization even if the field's class has a registered `CompactSerializer`. === Entry Processors and Captured Variables This behavior is particularly relevant when using `Sinks.mapWithEntryProcessor()`. -The `EntryProcessor` interface extends `Serializable`, so any custom entry processor -and all its fields must be Java serializable. Consider this example, where `OrderStatus` has a registered Compact serializer: @@ -200,31 +198,33 @@ pipeline.readFrom(source) === Using Compact Serialization with Pipelines -To use Compact-serializable objects with pipelines: - -1. **Use service factories**: For non-serializable dependencies, use `mapUsingService()` -to create objects on the target member rather than capturing them in lambdas. - -2. **Implement both interfaces**: Have your classes implement `Serializable` in addition - to registering a Compact serializer. The Compact serializer is still used when - Hazelcast serializes objects for storage in maps or transmission between members. +To use Compact-serializable objects with pipelines you should Implement both options: Have your classes implement +`Serializable` in addition to registering a Compact serializer. The Compact serializer is still used when Hazelcast +serializes objects for storage in maps or transmission between members. == Serialization of Data Types The objects you store in xref:data-structures:distributed-data-structures.adoc[Hazelcast data structures] must be serializable. Another case that requires serializable objects is sending computation -results between members, for example when grouping by key. To catch +results between members, for example, when grouping by key. To catch serialization issues early on, we recommend using a 2-member local cluster for development and testing. -Currently, Hazelcast supports 4 interfaces to serialize custom +Currently, Hazelcast supports five interfaces to serialize custom types: - link:https://docs.oracle.com/javase/8/docs/api/java/io/Serializable.html[java.io.Serializable] - link:https://docs.oracle.com/javase/8/docs/api/java/io/Externalizable.html[java.io.Externalizable] - link:https://docs.hazelcast.org/docs/{os-version}/javadoc/com/hazelcast/nio/serialization/Portable.html[com.hazelcast.nio.serialization.Portable] - link:https://docs.hazelcast.org/docs/{os-version}/javadoc/com/hazelcast/nio/serialization/StreamSerializer.html[com.hazelcast.nio.serialization.StreamSerializer] +- link:https://docs.hazelcast.org/docs/{os-version}/javadoc/com/hazelcast/nio/serialization/compact/CompactSerializer.html[com.hazelcast.nio.serialization.compact.CompactSerializer] + +[CAUTION] +.Deprecation Notice for Portable Serialization +==== +Portable Serialization has been deprecated. We recommend you use Compact Serialization as Portable Serialization will be removed as of version 7.0. +==== The following table provides a comparison between them to help you in deciding which interface to use in your applications. @@ -248,11 +248,15 @@ deciding which interface to use in your applications. |StreamSerializer |Fastest and lightest |Requires implementation and registration + +|CompactSerializer +|Faster and more space efficient than Serializable. Supports versioning and partial deserialization. Does not require implementation or registration +|Not as fast or lightweight as StreamSerializer |=== Below you can find rough performance numbers you can expect when employing each of those strategies. A straightforward benchmark that -continuously serializes and then deserializes this simple object: +serializes and then deserializes 1,000 unique versions of this simple object: ```java class Person { @@ -266,24 +270,26 @@ class Person { yields following throughputs: ``` -# Processor: Intel(R) Core(TM) i7-4700HQ CPU @ 2.40GHz -# VM version: JDK 13, OpenJDK 64-Bit Server VM, 13+33 - -Benchmark Mode Cnt Score Error Units -SerializationBenchmark.serializable thrpt 3 0.259 ± 0.087 ops/us -SerializationBenchmark.externalizable thrpt 3 0.846 ± 0.057 ops/us -SerializationBenchmark.portable thrpt 3 1.171 ± 0.539 ops/us -SerializationBenchmark.stream thrpt 3 4.828 ± 1.227 ops/us +# Processor: Intel(R) Core(TM) i9-12900H CPU @ 2.50GHz +# VM version: JDK 17.0.6, OpenJDK 64-Bit Server VM, 17.0.6+10-LTS + +Benchmark Mode Cnt Score Error Units +SerializationBenchmark.serializable thrpt 10 1.807 ± 0.043 ops/us +SerializationBenchmark.externalizable thrpt 10 2.735 ± 0.060 ops/us +SerializationBenchmark.portable thrpt 10 1.851 ± 0.049 ops/us +SerializationBenchmark.stream thrpt 10 2.995 ± 0.099 ops/us +SerializationBenchmark.compact thrpt 10 2.844 ± 0.044 ops/us ``` -Here are the sizes of the serialized form by each serializer: +Here are the sizes of the serialized form for the same data by each serializer: ``` Strategy Number of Bytes Overhead % -java.io.Serializable 162 523 -java.io.Externalizable 87 234 -com.hazelcast.nio.serialization.Portable 104 300 -com.hazelcast.nio.serialization.StreamSerializer 26 0 +java.io.Serializable 154 327.8 +java.io.Externalizable 93 158.3 +com.hazelcast.nio.serialization.Portable 114 216.7 +com.hazelcast.nio.serialization.StreamSerializer 36 0.0 +com.hazelcast.nio.serialization.compact.CompactSerializer 50 38.9 ``` You can see that using plain `Serializable` can easily become a @@ -348,9 +354,7 @@ travel through the pipeline (over distributed DAG edges) and get saved to snapshots. Job-level serializers can also be used with xref:sources-sinks.adoc[sources and sinks] that use in-memory data structures. You can read from/write to a local -`Observable`, `IList`, `IMap` or `ICache`. We are working on adding the -ability to read from an `IMap` using a user-defined predicate and -projections, update an `IMap`, and read from `EventJournal`. +`Observable`, `IList`, `IMap` or `ICache`. == Register a Serializer with the Hazelcast cluster From 58cc2ffed2103c270bc49413732c21d81900251f Mon Sep 17 00:00:00 2001 From: James Holgate Date: Fri, 20 Feb 2026 19:54:08 +0000 Subject: [PATCH 07/11] Feedback changes --- .../pipelines/pages/serialization.adoc | 56 +++++++++++-------- 1 file changed, 32 insertions(+), 24 deletions(-) diff --git a/docs/modules/pipelines/pages/serialization.adoc b/docs/modules/pipelines/pages/serialization.adoc index 3cb90ffc6..aba3c6d48 100644 --- a/docs/modules/pipelines/pages/serialization.adoc +++ b/docs/modules/pipelines/pages/serialization.adoc @@ -158,33 +158,39 @@ not delegate to Hazelcast's serialization service. This means: === Entry Processors and Captured Variables This behavior is particularly relevant when using `Sinks.mapWithEntryProcessor()`. +This sink accepts `toEntryProcessorFn`, a function that creates an `EntryProcessor`. +Because `toEntryProcessorFn` is part of the pipeline definition, it is serialized with Java serialization. +Only variables captured by `toEntryProcessorFn` must be Java-serializable. -Consider this example, where `OrderStatus` has a registered Compact serializer: +The following examples omit unrelated method parameters for brevity: ```java -// OrderStatus has a CompactSerializer registered, but this still fails -// because MergeEntryProcessor implements Serializable (via EntryProcessor) -public class MergeEntryProcessor implements EntryProcessor { - // This field must be Serializable, even though OrderStatus has a CompactSerializer - private final OrderStatus newStatus; - - public MergeEntryProcessor(OrderStatus newStatus) { - this.newStatus = newStatus; - } +// No captured state +Sinks.mapWithEntryProcessor(MergeEntryProcessor::new); - @Override - public Order process(Entry entry) { - Order order = entry.getValue(); - order.setStatus(newStatus); - entry.setValue(order); - return order; - } -} +// No captured state: OrderStatus is created inside the lambda +Sinks.mapWithEntryProcessor(() -> new MergeEntryProcessor(new OrderStatus("SHIPPED"))); + +// Captures mep, so mep must be Serializable +MergeEntryProcessor mep = new MergeEntryProcessor(); +Sinks.mapWithEntryProcessor(() -> mep); + +// Captures status, so status must be Serializable +OrderStatus status = new OrderStatus("SHIPPED"); +Sinks.mapWithEntryProcessor(() -> new MergeEntryProcessor(status)); ``` -In this case, `OrderStatus` must implement `Serializable` in addition to having a -Compact serializer, because Java's `ObjectOutputStream` does not know about Hazelcast's -Compact serialization. +This capture behavior is standard Java serialization behavior and is not Jet-specific. +The same rule applies when you use an `EntryProcessor` directly with the `IMap` API. + +After `toEntryProcessorFn` runs, the created `EntryProcessor` instance is serialized for `IMap` execution. +At this stage, requirements depend on the `EntryProcessor` serialization strategy: + +* If `MergeEntryProcessor` is serialized with Java serialization (for example by relying on `Serializable`), all its fields must be Java-serializable. +* If `MergeEntryProcessor` is serialized with Compact serialization, its fields can use Compact serializers. + +Target members must also be able to deserialize `MergeEntryProcessor`. +Make this class available where an `IMap` deserializes it (for example member classpath or User Code Namespace), not only in Jet job resources. The same applies to lambdas that capture Compact-serializable variables: @@ -198,9 +204,11 @@ pipeline.readFrom(source) === Using Compact Serialization with Pipelines -To use Compact-serializable objects with pipelines you should Implement both options: Have your classes implement -`Serializable` in addition to registering a Compact serializer. The Compact serializer is still used when Hazelcast -serializes objects for storage in maps or transmission between members. +Compact and Java serialization can both be involved in the same pipeline, but at different points. + +* For pipeline definition objects (for example lambdas and captured variables), Java serialization rules apply. +* For runtime data movement and storage, Compact serialization rules apply. +* If a class is used in both contexts, it may need both a Java serialization compatible form and a Compact serializer. == Serialization of Data Types From c8c099717c641d259f4f8e8edcee42cf38c4ec4c Mon Sep 17 00:00:00 2001 From: James Holgate Date: Wed, 25 Feb 2026 21:54:20 +0000 Subject: [PATCH 08/11] Benchmark updates --- .../pipelines/pages/serialization.adoc | 20 +++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/docs/modules/pipelines/pages/serialization.adoc b/docs/modules/pipelines/pages/serialization.adoc index aba3c6d48..d961fb372 100644 --- a/docs/modules/pipelines/pages/serialization.adoc +++ b/docs/modules/pipelines/pages/serialization.adoc @@ -281,14 +281,18 @@ yields following throughputs: # Processor: Intel(R) Core(TM) i9-12900H CPU @ 2.50GHz # VM version: JDK 17.0.6, OpenJDK 64-Bit Server VM, 17.0.6+10-LTS -Benchmark Mode Cnt Score Error Units -SerializationBenchmark.serializable thrpt 10 1.807 ± 0.043 ops/us -SerializationBenchmark.externalizable thrpt 10 2.735 ± 0.060 ops/us -SerializationBenchmark.portable thrpt 10 1.851 ± 0.049 ops/us -SerializationBenchmark.stream thrpt 10 2.995 ± 0.099 ops/us -SerializationBenchmark.compact thrpt 10 2.844 ± 0.044 ops/us +Benchmark Mode Cnt Score Error Units +SerializationBenchmark.serializable thrpt 10 2.080 ± 0.056 ops/us +SerializationBenchmark.externalizable thrpt 10 3.171 ± 0.056 ops/us +SerializationBenchmark.portable thrpt 10 3.330 ± 0.051 ops/us +SerializationBenchmark.compact_zeroConfig thrpt 10 5.337 ± 0.108 ops/us +SerializationBenchmark.compact_registered thrpt 10 8.108 ± 0.160 ops/us +SerializationBenchmark.stream thrpt 10 18.035 ± 0.322 ops/us ``` +NOTE: `compact_zeroConfig` represents serialization using Zero Config Compact Serialization, while `compact_registered` +represents serialization using a serializer defined and registered with Compact. + Here are the sizes of the serialized form for the same data by each serializer: ``` @@ -296,13 +300,13 @@ Strategy Number of Bytes Overhead % java.io.Serializable 154 327.8 java.io.Externalizable 93 158.3 com.hazelcast.nio.serialization.Portable 114 216.7 -com.hazelcast.nio.serialization.StreamSerializer 36 0.0 com.hazelcast.nio.serialization.compact.CompactSerializer 50 38.9 +com.hazelcast.nio.serialization.StreamSerializer 36 0.0 ``` You can see that using plain `Serializable` can easily become a bottleneck in your application, as even with this simple data type it's -more than an order of magnitude slower than other serialization options, +significantly slower than some of the other serialization options, not to mention very wasteful with memory. == Write a Custom Serializer From 6657383543bff2369d8c32c141508420613d3150 Mon Sep 17 00:00:00 2001 From: James Holgate <130981049+JamesHazelcast@users.noreply.github.com> Date: Thu, 26 Feb 2026 12:09:51 +0000 Subject: [PATCH 09/11] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Krzysztof Jamróz <79092062+k-jamroz@users.noreply.github.com> --- docs/modules/pipelines/pages/serialization.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/modules/pipelines/pages/serialization.adoc b/docs/modules/pipelines/pages/serialization.adoc index d961fb372..c983e4ebd 100644 --- a/docs/modules/pipelines/pages/serialization.adoc +++ b/docs/modules/pipelines/pages/serialization.adoc @@ -171,11 +171,11 @@ Sinks.mapWithEntryProcessor(MergeEntryProcessor::new); // No captured state: OrderStatus is created inside the lambda Sinks.mapWithEntryProcessor(() -> new MergeEntryProcessor(new OrderStatus("SHIPPED"))); -// Captures mep, so mep must be Serializable +// Captures mep, so MergeEntryProcessor must be Serializable MergeEntryProcessor mep = new MergeEntryProcessor(); Sinks.mapWithEntryProcessor(() -> mep); -// Captures status, so status must be Serializable +// Captures status, so OrderStatus must be Serializable OrderStatus status = new OrderStatus("SHIPPED"); Sinks.mapWithEntryProcessor(() -> new MergeEntryProcessor(status)); ``` From 46e8b0f737a8cbcdf92d1d45e39df0a3b47741dc Mon Sep 17 00:00:00 2001 From: James Holgate Date: Thu, 26 Feb 2026 12:56:37 +0000 Subject: [PATCH 10/11] Updates from feedback --- docs/modules/pipelines/pages/serialization.adoc | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/docs/modules/pipelines/pages/serialization.adoc b/docs/modules/pipelines/pages/serialization.adoc index c983e4ebd..701f302a4 100644 --- a/docs/modules/pipelines/pages/serialization.adoc +++ b/docs/modules/pipelines/pages/serialization.adoc @@ -162,6 +162,10 @@ This sink accepts `toEntryProcessorFn`, a function that creates an `EntryProcess Because `toEntryProcessorFn` is part of the pipeline definition, it is serialized with Java serialization. Only variables captured by `toEntryProcessorFn` must be Java-serializable. +Other `IMap` sinks that accept lambdas follow the same captured variable rule, +but `Sinks.mapWithEntryProcessor()` adds one extra level of indirection because you pass +a factory for `EntryProcessor` instances. + The following examples omit unrelated method parameters for brevity: ```java @@ -210,6 +214,15 @@ Compact and Java serialization can both be involved in the same pipeline, but at * For runtime data movement and storage, Compact serialization rules apply. * If a class is used in both contexts, it may need both a Java serialization compatible form and a Compact serializer. +=== Compact Serializers and Jet Jobs + +When using Compact serialization with Jet, keep the following limits and recommendations in mind: + +* Unlike `StreamSerializer`, `CompactSerializer` is currently not supported for single-job registration with `JobConfig.registerSerializer()`. +* Explicit `CompactSerializer` implementations must be registered in member configuration. +* Avoid Zero Config Compact Serialization for classes attached to jobs, because repeated job classloading can lead to classloader leaks and `ClassCastException`. +* Prefer keeping Compact-serializable DTOs and related classes on the member classpath. + == Serialization of Data Types The objects you store in xref:data-structures:distributed-data-structures.adoc[Hazelcast data structures] must be serializable. From 1f5c2b948bf99984fa448b794e828adf2fe41041 Mon Sep 17 00:00:00 2001 From: James Holgate <130981049+JamesHazelcast@users.noreply.github.com> Date: Thu, 26 Feb 2026 14:43:26 +0000 Subject: [PATCH 11/11] Update docs/modules/pipelines/pages/serialization.adoc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Krzysztof Jamróz <79092062+k-jamroz@users.noreply.github.com> --- docs/modules/pipelines/pages/serialization.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/modules/pipelines/pages/serialization.adoc b/docs/modules/pipelines/pages/serialization.adoc index 701f302a4..cd982c68c 100644 --- a/docs/modules/pipelines/pages/serialization.adoc +++ b/docs/modules/pipelines/pages/serialization.adoc @@ -277,7 +277,7 @@ deciding which interface to use in your applications. Below you can find rough performance numbers you can expect when employing each of those strategies. A straightforward benchmark that -serializes and then deserializes 1,000 unique versions of this simple object: +serializes and then deserializes unique, pooled instances of this simple object: ```java class Person {