Serialisers about kaffe HOT 7 CLOSED

Hi @rawkode, thanks for this suggestion!

I generally try to avoid adding pluggable functionality to do something that's already easily accessible. So in this case the value of a config-specified serializer seems low since it's not hard for implementers to know how or where to implement that functionality themselves (right there in the message handler!). Doing otherwise tends to just increase the framework's surface area for little real value. But that's purely from a feature management perspective. Perhaps there's value in having a few common serializer implementations already available?

Happy to hear others' thoughts!

/cc @sdball @objectuser

from kaffe.

@rwdaigle I'm now running 9 different services, all implementing the same piece of functionality. Perhaps I am in a minority with my usage. Although the setup is trivial, I do prefer not repeating myself 😄

from kaffe.

How about making your serializer logic into its own dependency via mix to be called from each handle_message?

def handle_message(message) do
  MyCustomMessageParser.parse!(message)
  |> other_things
  |> for_this_service
end

You'd still get consolidated parsing logic with minimal supporting code. But instead of one line of configuration you'd have one line of message handling code. That keeps Kaffe's contract simpler and keeps all the message handling logic out in the open.

from kaffe.

OK. I'm sensing you aren't keen on this idea 😂

I'll close it and work around it.

from kaffe.

Haha, sorry @rawkode!

I think this is a great use-case that forces us to be more explicit about the contract for Kaffe. In my mind it's a low-level Kafka library that handles much of the nuance associated with practical usage of Kafka. So one-level above a pure adapter (that's what brod is), but not itself a framework. It's more of a convenience library.

I think a good litmus test for this is that I don't ever see Kaffe concerning itself with actual message bodies (parsing, plucking values, etc...), as that starts to get into domain logic. Anything Kafka metadata related however, I would see as fair game.

Hopefully that sounds logical vs. just an arbitrary line that you happen to be on the other side of for this example?

from kaffe.

Unfortunately, that’s an unexpected approach for the Kafka community. The JVM libs and the librdkafka family all expect producers/consumers to specify the serializer/deserializer as part of the config.

from kaffe.

Kaffe was designed to make its use and setup as trivial as possible. In this it’s more like ruby-kafka’s approach to serialization: https://github.com/zendesk/ruby-kafka/blob/master/README.md#serialization

from kaffe.