Commit 04baacc
committed
Add support for hashing transformations and caching configuration
Enhancements:
- Introduced HashTransform for hashing field values using algorithms like MD5, SHA256, and Murmur.
- Added commit strategy configuration to control commit semantics (manual vs. auto) and enable dead letter queues.
- Implemented caching backend configuration options, including local, Redis, and Kafka-backed caches.
Documentation:
- Updated README and SECRETS.md to reflect new features and usage examples.
- Enhanced examples to demonstrate multi-cluster secure pipelines and secret management.
Benefits:
✅ Improved data processing capabilities with hashing.
✅ Flexible commit strategies for message delivery.
✅ Enhanced caching options for better performance and scalability.1 parent d0f3a60 commit 04baacc
23 files changed
Lines changed: 5635 additions & 54 deletions
File tree
- docs
- examples
- pipelines
- operator
- src
- src
- kafka
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
45 | 45 | | |
46 | 46 | | |
47 | 47 | | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
48 | 63 | | |
49 | 64 | | |
50 | 65 | | |
| |||
60 | 75 | | |
61 | 76 | | |
62 | 77 | | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
0 commit comments