Inner join by key where key is GenericRecord about kafka-streams-examples HOT 3 CLOSED

The join happens here: https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoin.java#L88-L100

For a join, there is a windowed store for each input side that buffers all incoming records (this happens in an upstream processor: https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamJoinWindow.java#L64).

Additionally, each input record is used to so a lookup in the other store to find matching keys. The store lookup is basically the evaluation of the == predicate on the key plus the timestamp evaluation (cf. the otherWindow.fetch(key, timeFrom, timeTo) operation).

Before the join is done, Kafka Streams ensures that both input topics have the same number of partitions and that data is partitioned on the key. Thus, topic1-partition0 and topic2-partition-0 will contain the same keys and can be processed within a single thread.

Hope this helps.

from kafka-streams-examples.

The key comparison happens on the raw bytes -- thus, as long as both messages have the exact same serialized format they will be joined. What join are you interested in? KStream-KStream, KTable-KTable, or KStream-KTable?

from kafka-streams-examples.

KStream-KStream

from kafka-streams-examples.