Allow runtime configuration of consumers. about kaffe HOT 7 CLOSED

I can’t believe I typed all that on my phone...

well now I feel like we should do it just based on that effort! 😸

from kaffe.

Another thing that would be really nice is if the consumer and producer were behaviours similar to GenServer.

defmodule MyApp.Consumer do
  use Kaffe.Consumer

  def start_link(config, opts \\ []) do
    Kaffe.Consumer.start_link(__MODULE__, config, opts)
  end

  def init(config) do
    {:ok, config}
  end

  def handle_messages(messages, state) do
    ...
  end
end

I can’t believe I typed all that on my phone... 😀

Then you could start your own module in your supervision tree.

from kaffe.

@davidsantoso No, I just happened to use that one as an example.

from kaffe.

👍 thanks for all the suggestions. Going to try and carve out some time this week and next to groom the project a bit and see what we can refactor.

from kaffe.

@keathley based on your code snippet, curious if y'all use/prefer single message consumers more than the Kaffe.GroupMember setup to do batch message processing?

from kaffe.

Closing due to above PR

from kaffe.

Hi,

It seems that this change did not make it to the current master of Kaffe. Currently there is no way to set configuration at runtime when starting the supervision tree, everything must be known up front.

For example, topics: You must list your topics in your configuration, but also somehow in you application code because you may want to match on the topic for logic. There is no more a single source of truth for the topic list.

You could say that you would pull your configuration in your code, using the config as a single source of truth. But then either you cannot do it at compile time, like @my_topics Application.fetch_env(:my_app, MyConsumer)[:topics] or you cannot use runtime.exs config because those values are not available at compile time.

from kaffe.