Not possible to set auto_start_producers and allow_topic_auto_creation

[kivra-kenand]

It is not possible for me to set auto_start_producers nor allow_topic_auto_creation via app-envs.

Suggested change:

diff --git a/lib/kaffe/config/consumer.ex b/lib/kaffe/config/consumer.ex
index 6e54e49..70b455c 100644
--- a/lib/kaffe/config/consumer.ex
+++ b/lib/kaffe/config/consumer.ex
@@ -93,8 +93,8 @@ defmodule Kaffe.Config.Consumer do
 
   def default_client_consumer_config(config_key) do
     [
-      auto_start_producers: false,
-      allow_topic_auto_creation: false,
+      auto_start_producers: config_get(config_key, :auto_start_producers, :false),
+      allow_topic_auto_creation: config_get(config_key, :allow_topic_auto_creation, :false),
       begin_offset: begin_offset(config_key)
     ]
   end


diff --git a/lib/kaffe/config/producer.ex b/lib/kaffe/config/producer.ex
index d2e09a1..dfca764 100644
--- a/lib/kaffe/config/producer.ex
+++ b/lib/kaffe/config/producer.ex
@@ -46,8 +46,8 @@ defmodule Kaffe.Config.Producer do
 
   def default_client_producer_config do
     [
-      auto_start_producers: true,
-      allow_topic_auto_creation: false,
+      auto_start_producers: config_get(:auto_start_producers, :true),
+      allow_topic_auto_creation: config_get(:allow_topic_auto_creation, :false),
       default_producer_config: [
         required_acks: config_get(:required_acks, -1),
         ack_timeout: config_get(:ack_timeout, 1000),

[TylerWitt]

To be honest, auto_start_producers and allow_topic_auto_creation don't seem to be valid :brod.consumer_config options anyways, and should likely be removed from default_client_consumer_config/1.

For the producer, allow_topic_auto_creation should definitely be configurable, so feel free to make a PR if you like. auto_start_producers is a bit more complicated. This library is intended to wrap :brod. To allow producers to be dynamically started, we should provide an API to start the producer. It's not hard by any means, but just a consideration beyond allowing extra config options.

[TylerWitt]

Looking in to it more deeply, client config and consumer config are currently combined, so this

To be honest, auto_start_producers and allow_topic_auto_creation don't seem to be valid :brod.consumer_config options anyways, and should likely be removed from default_client_consumer_config/1.

does not apply in the current architecture.