Skip to content
This repository was archived by the owner on Jun 30, 2023. It is now read-only.

Commit 4c8984d

Browse files
author
Bayard Neville
authored
Match fully-qualified name for legacy name proto option (#40)
* Passing test for unqualified legacy name * Test for match on fully qualified name * Match fully qualified name * Proto converter tests for default vs custom legacy name * Regenerate protos to fix bad legacy name * Update major version for breaking change * Update readme for new legacy name path rules * Document protoc command
1 parent 49fdcf1 commit 4c8984d

15 files changed

+1023
-429
lines changed

README.md

+14-7
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# kafka-connect-protobuf-converter
22
Converter plugin for [Kafka Connect](https://docs.confluent.io/current/connect/). A converter
3-
controls the format of the data that will be written to Kafka for source connectors or
3+
controls the format of the data that will be written to Kafka for source connectors or
44
read from Kafka for sink connectors.
55

66
## Compatibility
@@ -11,15 +11,15 @@ The 2.x release series is compatible with Kafka Connect 5.x. and up (older relea
1111

1212
## Usage
1313

14-
Copy the `kafka-connect-protobuf-converter` jar and the jar containing your compiled protocol buffers to
14+
Copy the `kafka-connect-protobuf-converter` jar and the jar containing your compiled protocol buffers to
1515
`/usr/share/java/kafka-serde-tools` on your Kafka Connect instance and restart Kafka Connect.
1616

1717
Converters can be specified on a per-connector basis.
1818

1919
To use the protobuf converter in Kafka Connect, specify the converter as your key and value converter and specify the
2020
protocol buffer class you want to use to deserialize the message (ex: `com.google.protobuf.Int32Value`).
2121

22-
Note: Nested classes must be specified using the `$` notation, for example
22+
Note: Nested classes must be specified using the `$` notation, for example
2323
`com.blueapron.connect.protobuf.NestedTestProtoOuterClass$NestedTestProto`
2424

2525
Example Kafka Connect JDBC source:
@@ -105,7 +105,7 @@ original name and keep the Kafka Connect schema consistent.
105105
You can specify the name for this field option using the `legacyName` configuration item. By default, the field option
106106
used is `legacy_name`
107107

108-
Example: `value.converter.legacyName=legacy_name`
108+
Example: `value.converter.legacyName=blueapron.connect.protobuf.legacy_name`
109109

110110
This annotation provides a hint that allows renamed fields to be mapped to correctly to their output schema names.
111111

@@ -127,7 +127,7 @@ import "path/to/LegacyName.proto";
127127
128128
message TestMessage {
129129
string yet_another_named_string = 1 [(blueapron.connect.protobuf.legacy_name) = "test_string"];
130-
}
130+
}
131131
132132
```
133133

@@ -166,8 +166,8 @@ mvn test
166166
mvn clean package
167167
```
168168

169-
Copy the JAR with dependencies (`kafka-connect-protobuf-converter-*-jar-with-dependencies.jar`) to
170-
`/usr/share/java/kafka-serde-tools` on your local Kafka Connect instance to make the
169+
Copy the JAR with dependencies (`kafka-connect-protobuf-converter-*-jar-with-dependencies.jar`) to
170+
`/usr/share/java/kafka-serde-tools` on your local Kafka Connect instance to make the
171171
converter available in Kafka Connect.
172172

173173
#### Deploy to Maven Central:
@@ -194,3 +194,10 @@ converter available in Kafka Connect.
194194
```
195195
mvn release:perform
196196
```
197+
198+
#### Generate protos:
199+
200+
Run the below command from the repository root.
201+
```
202+
protoc -I=src/test/java/com/blueapron/connect/protobuf/protos --java_out=src/test/java/ src/test/java/com/blueapron/connect/protobuf/protos/*.proto
203+
```

pom.xml

+1-1
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
<groupId>com.blueapron</groupId>
66
<artifactId>kafka-connect-protobuf-converter</artifactId>
77
<packaging>jar</packaging>
8-
<version>2.3.1-SNAPSHOT</version>
8+
<version>3.0.0-SNAPSHOT</version>
99

1010
<name>com.blueapron:kafka-connect-protobuf-converter</name>
1111
<description>Converter plugin for Kafka Connect. A converter controls the format of the data

src/main/java/com/blueapron/connect/protobuf/ProtobufData.java

+1-1
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,7 @@ private String getProtoMapKey(String descriptorContainingTypeName, String connec
7373
private String getConnectFieldName(Descriptors.FieldDescriptor descriptor) {
7474
String name = descriptor.getName();
7575
for (Map.Entry<Descriptors.FieldDescriptor, Object> option: descriptor.getOptions().getAllFields().entrySet()) {
76-
if (option.getKey().getName().equalsIgnoreCase(this.legacyName)) {
76+
if (option.getKey().getFullName().equalsIgnoreCase(this.legacyName)) {
7777
name = option.getValue().toString();
7878
}
7979
}

src/test/java/com/blueapron/connect/protobuf/BytesValueOuterClass.java

+57-29
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)