Skip to content

kcat 1.7.0 does not produce, while 1.5.0 does and kafka-console-producer also... #460

@sachako

Description

@sachako

I've set up a test Kafka cluster and tried validate setup by kcat produce/consume some data. But nothing worked.

After some investigation I observed that the problem is kcat producer:
I can produce a record with java console producer and consume it with kcat.
I also managed to find a working kafkacat version, this one https://hub.docker.com/layers/confluentinc/cp-kafkacat/7.1.13/images/sha256-1a516059e4de567ee056878039611dc2273939d2dedd1ee2832c162fa34ba2d6

Might be an issue with my setup of course...

Probably it can be a hint: process does not die on Ctrl-C and Ctrl-\, I have to docker rm -f

Here is some debug just in case:

docker run --rm -it --network=host edenhill/kcat:1.7.1 -b 10.135.190.191:9092 -d broker  -P -t test
%7|1741913713.020|BRKMAIN|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Enter main broker thread
%7|1741913713.022|BROKER|rdkafka#producer-1| [thrd:app]: 10.135.190.191:9092/bootstrap: Added new broker with NodeId -1
%7|1741913713.023|CONNECT|rdkafka#producer-1| [thrd:app]: 10.135.190.191:9092/bootstrap: Selected for cluster connection: bootstrap servers added (broker has 0 connection attempt(s))
%7|1741913713.023|INIT|rdkafka#producer-1| [thrd:app]: librdkafka v1.8.2 (0x10802ff) rdkafka#producer-1 initialized (builtin.features gzip,snappy,ssl,sasl,regex,lz4,sasl_plain,sasl_scram,plugins,zstd,sasl_oauthbearer, GCC GXX PKGCONFIG GNULD C11THREADS LIBDL PLUGINS ZLIB SSL ZSTD HDRHISTOGRAM SYSLOG SNAPPY SOCKEM SASL_SCRAM SASL_OAUTHBEARER CRC32C_HW, debug 0x2)
%7|1741913713.024|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: leader query
%7|1741913713.024|BRKMAIN|rdkafka#producer-1| [thrd:10.135.190.191:9092/bootstrap]: 10.135.190.191:9092/bootstrap: Enter main broker thread
%7|1741913713.024|CONNECT|rdkafka#producer-1| [thrd:10.135.190.191:9092/bootstrap]: 10.135.190.191:9092/bootstrap: Received CONNECT op
%7|1741913713.024|STATE|rdkafka#producer-1| [thrd:10.135.190.191:9092/bootstrap]: 10.135.190.191:9092/bootstrap: Broker changed state INIT -> TRY_CONNECT
%7|1741913713.024|CONNECT|rdkafka#producer-1| [thrd:10.135.190.191:9092/bootstrap]: 10.135.190.191:9092/bootstrap: broker in state TRY_CONNECT connecting
%7|1741913713.024|STATE|rdkafka#producer-1| [thrd:10.135.190.191:9092/bootstrap]: 10.135.190.191:9092/bootstrap: Broker changed state TRY_CONNECT -> CONNECT
%7|1741913713.024|CONNECT|rdkafka#producer-1| [thrd:10.135.190.191:9092/bootstrap]: 10.135.190.191:9092/bootstrap: Connecting to ipv4#10.135.190.191:9092 (plaintext) with socket 7
%7|1741913713.025|CONNECT|rdkafka#producer-1| [thrd:10.135.190.191:9092/bootstrap]: 10.135.190.191:9092/bootstrap: Connected to ipv4#10.135.190.191:9092
%7|1741913713.025|CONNECTED|rdkafka#producer-1| [thrd:10.135.190.191:9092/bootstrap]: 10.135.190.191:9092/bootstrap: Connected (#1)
%7|1741913713.025|FEATURE|rdkafka#producer-1| [thrd:10.135.190.191:9092/bootstrap]: 10.135.190.191:9092/bootstrap: Updated enabled protocol features +ApiVersion to ApiVersion
%7|1741913713.025|STATE|rdkafka#producer-1| [thrd:10.135.190.191:9092/bootstrap]: 10.135.190.191:9092/bootstrap: Broker changed state CONNECT -> APIVERSION_QUERY
%7|1741913713.027|FEATURE|rdkafka#producer-1| [thrd:10.135.190.191:9092/bootstrap]: 10.135.190.191:9092/bootstrap: Updated enabled protocol features to MsgVer1,ApiVersion,BrokerBalancedConsumer,ThrottleTime,Sasl,SaslHandshake,BrokerGroupCoordinator,LZ4,OffsetTime,MsgVer2,IdempotentProducer,ZSTD,SaslAuthReq
%7|1741913713.027|STATE|rdkafka#producer-1| [thrd:10.135.190.191:9092/bootstrap]: 10.135.190.191:9092/bootstrap: Broker changed state APIVERSION_QUERY -> UP
%7|1741913713.029|BROKER|rdkafka#producer-1| [thrd:main]: 10.135.190.186:9092/12: Added new broker with NodeId 12
%7|1741913713.029|UPDATE|rdkafka#producer-1| [thrd:10.135.190.191:9092/bootstrap]: 10.135.190.191:9092/bootstrap: NodeId changed from -1 to 13
%7|1741913713.029|UPDATE|rdkafka#producer-1| [thrd:10.135.190.191:9092/bootstrap]: 10.135.190.191:9092/13: Name changed from 10.135.190.191:9092/bootstrap to 10.135.190.191:9092/13
%7|1741913713.029|LEADER|rdkafka#producer-1| [thrd:10.135.190.191:9092/bootstrap]: 10.135.190.191:9092/13: Mapped 0 partition(s) to broker
%7|1741913713.029|STATE|rdkafka#producer-1| [thrd:10.135.190.191:9092/bootstrap]: 10.135.190.191:9092/13: Broker changed state UP -> UPDATE
%7|1741913713.029|BROKER|rdkafka#producer-1| [thrd:main]: 10.135.190.192:9092/11: Added new broker with NodeId 11
%7|1741913713.030|CLUSTERID|rdkafka#producer-1| [thrd:main]: 10.135.190.191:9092/13: ClusterId update "" -> "Crg9aKeQTMeld920AHgJmQ"
%7|1741913713.030|CONTROLLERID|rdkafka#producer-1| [thrd:main]: 10.135.190.191:9092/13: ControllerId update -1 -> 13
%7|1741913713.030|TOPBRK|rdkafka#producer-1| [thrd:10.135.190.191:9092/bootstrap]: 10.135.190.191:9092/13: Topic test [0]: joining broker (rktp 0x558890d7e300, 0 message(s) queued)
%7|1741913713.030|STATE|rdkafka#producer-1| [thrd:10.135.190.191:9092/bootstrap]: 10.135.190.191:9092/13: Broker changed state UPDATE -> UP
%7|1741913713.029|BRKMAIN|rdkafka#producer-1| [thrd:10.135.190.186:9092/12]: 10.135.190.186:9092/12: Enter main broker thread
%7|1741913713.030|BRKMAIN|rdkafka#producer-1| [thrd:10.135.190.192:9092/11]: 10.135.190.192:9092/11: Enter main broker thread
hhhhh
llllll
^\
^C

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions