Data streaming platforms connect heterogeneous services through the publish-subscribe paradigm. Currently available platforms provide protection against crash faults, but are not resistant against Byzantine faults like arbitrary hardware faults and intrusions. State machine replication can provide this protection, but the higher resource requirements and the more elaborate communication primitives usually result in a higher overall complexity and a non-negligible performance degradation. As data streaming operates on highly-partitionable append-only state, some of these performance losses can be counteracted by applying speculative execution and sharding. We show the effectiveness of these concepts in a prototype implementation, which only results in a reasonable drop in system throughput and latency during average system utilization, when compared to state-of-the-art data streaming platforms like Apache Kafka, while providing stronger resilience guarantees.

Johannes Köstler

Hans P. Reiser

SmartStream

ivySCI AI Smartly Parses PDF, Answers Researchers' Questions, and Helps You Understand Papers in Seconds

Journal Info

Category	Quartile
COMPUTER SCIENCE, INFORMATION SYSTEMS	4