Use common apache kafka client

[erikvanoosten]

... by excluding Confluent's version of the kafka client and adding an explicit version.
Without this, we'll get Confluent's version which is only available in their maven repository.

[aartigao]

But if I'm not mistaken (I use MVN Repository as a source of truth), the main artifact it's already only available in Confluent repo, so excluding it won't prevent you to have it declared, right?

Also this is only for the Vulcan integration which I'd dare to say is not what most of the users use 🤔

[erikvanoosten]

But if I'm not mistaken (I use MVN Repository as a source of truth), the main artifact it's already only available in Confluent repo, so excluding it won't prevent you to have it declared, right?

Yes indeed. However, I already declare that I want the open source version in my build, but then fs2-kafka-vulcan declares it wants the confluent version. The latter wins unless I explicitly disable that. It is super annoying.

Also this is only for the Vulcan integration which I'd dare to say is not what most of the users use 🤔

Does that matter?

[aartigao]

Yes, it matters IMO. Vulkan depends on it, and this means depends on everything transitive.

I don't think excluding and reincluding stuff is good (if they depend on their own version I assume is for a good reason).

As a user you can force another global version via SBT. I don't think it's the lib responsibility.

[erikvanoosten]

Yes, it matters IMO. Vulkan depends on it, and this means depends on everything transitive.

It seems to work fine with the open source version of kafka-client.

if they depend on their own version I assume is for a good reason

I have been unable to find such a reason (I really looked very hard). The best information I could find is this from https://docs.confluent.io/kafka-client/overview.html:

When you use the official Confluent-supported clients, you get the same enterprise-level support that you get with the rest of Confluent Platform:

• The release cycle for Confluent-provided clients follow the Confluent release cycle, as opposed to the Apache Kafka release cycle.
• Confluent Platform maintenance fixes are provided for the 2-3 years (2 years with the Standard Support and 3 years with the Platinum Support) after the initial release of a minor version.

I suspect the only difference is that the confluent client is behind (since its a copy of the apache client). Furthermore, I wonder if the confluent client calls home when you use it.

We are not using Confluent's Kafka (I mean the brokers), so I'd rather stay on the latest and freshest kafka-clients, and those are from Apache (with the additional benefit that they are really open source).

[aartigao]

This has been around for some time.

@vlovgr @bplommer @LMnet I'd love to hear your thoughts to finish the discussion 🙃