144lucky / sparkstreaming_store_kafkatopicoffset_to_hbase Goto Github PK
View Code? Open in Web Editor NEWThis project forked from wangliangbd/sparkstreaming_store_kafkatopicoffset_to_hbase
Kafka delivery semantics in the case of failure depend on how and when offsets are stored. Spark output operations are at-least-once. So if you want the equivalent of exactly-once semantics, you must either store offsets after an idempotent output, or store offsets in an atomic transaction alongside output.There is Spark Streaming how to store Kafka topic offset with HBase.
License: Apache License 2.0