ricardoarchive / kafka-topic-cloner Goto Github PK

What's the use case, or how will this tool be different from mirror-maker?

To prevent mistakes, cloning into the source topic should be impossible.
A parameter could be passed to force same-topic cloning.

The temporary topic should be automatically created at the beginning of the cloning process

An async producer would provude better performances thanks to batch producing.
Make sure that the messages order stays untouched.

The message consumed and message produced are filling stdout very quickly. We could log only one in a hundred or something similar.

a second url could be passed as the destination topic's broker. This would allow cloning from a cluster to another.

Move consuming and producing to the kafka package

A timeout to the consumer, similar to what we do with console consumer (timeout.ms) could let the user know when the cloning is ended by terminating the application

Cloning a topic's content currently overrides the producer timestamp which is contained in the event's metadata with the cloning time.
While this can sometimes be the desired behavior, it is not always the case.
A flag could be added with three different behaviors:

• Set the timestamp to the cloning time
• Do not touch the timestamp
• Set the timestamp to the cloning time only for events that don't have one

Add a --help command that displays the list of commands / arguments

A print that would tell how many messages were cloned would be useful

Unit and integration testing :)