Does NATS have an equivalent to Pulsar's Key_Shared mode? This feature would be really useful!

[pozen]

Proposed change

Feature Request: Native Key-Shared Consumption Pattern

Summary

Add a native Key-Shared subscription mode to NATS, similar to Apache Pulsar's feature, to enable ordered message processing per message key while maintaining horizontal scalability across consumers.

Proposed Change

Add a Native Key-Shared Consumption Pattern

Currently, to guarantee ordered processing of messages for a specific key (e.g., order_id, device_id) while maintaining horizontal scalability, one must implement a workaround using clever subject naming (data.{key}) and queue groups. This approach pushes the complexity of partitioning and routing logic to the client applications.

I propose adding a native Key-Shared subscription mode with the following behavior:

1. Producers assign a key (e.g., a string) to each message, without worrying about specific subject structures for partitioning.
2. The NATS server ensures that all messages with the same key are delivered to the same consumer within a queue group, maintaining strict publishing order.
3. Different keys are automatically load-balanced across all available consumers in the queue group.
4. The system should automatically handle consumer scaling and rebalancing while preserving key affinity.

This would provide an elegant, first-class solution for stateful stream processing, bridging the gap between pub/sub simplicity and partitioned message queue power.

Use case

This feature addresses critical scenarios where processing order per entity is mandatory, but overall system throughput demands horizontal scaling.

1. E-commerce Order Fulfillment

• Scenario: Processing sequential steps for an order (validate → reserve inventory → charge payment → ship).
• Requirement: All events for order_id=123 must be processed sequentially to prevent logical errors. Events for other orders must process in parallel.
• Benefit: Guarantees sequential per-order processing while automatically distributing load across consumers.

2. IoT Device Data Processing

• Scenario: Ingesting telemetry data from millions of devices.
• Requirement: Data points from device_id=abc must be processed in generation order for accurate state calculation.
• Benefit: Automatically partitions stream by device_id, ensuring data integrity without sacrificing throughput.

3. Financial Transactions

• Scenario: Handling debit/credit operations on user accounts.
• Requirement: Strict ordering for account_id=789 is non-negotiable for balance accuracy.
• Benefit: Provides strong ordering guarantees with horizontal scalability for high-volume platforms.

4. Real-time User Session Analytics

• Scenario: Tracking user journey (page_view → add_to_cart → purchase).
• Requirement: Events within user_session_id=xyz must be processed sequentially for accurate funnel analysis.
• Benefit: Enables accurate real-time session analytics without per-session stream management.

Additional Considerations

• This could be implemented as an enhancement to existing JetStream consumers
• Should support both push and pull consumer modes
• Key extraction could be configurable (message header, subject part, etc.)
• Would complement existing queue group functionality rather than replace it

Contribution

No response

[jnmoyne]

Please take a look at https://github.com/synadia-io/orbit.go/tree/main/pcgroups

[lefla]

Any update to bring this functionality to other clients (java, rust, ...) ?