Grokking Streaming Systems. Josh Fischer, Ning Wang

A friendly, framework-agnostic tutorial that will help you grok how streaming systems work—and how to build your own!

In Grokking Streaming Systems you will learn how to:

Implement and troubleshoot streaming systems

Design streaming systems for complex functionalities

Assess parallelization requirements

Spot networking bottlenecks and resolve back pressure

Group data for high-performance systems

Handle delayed events in real-time systems

Grokking Streaming Systems is a simple guide to the complex concepts behind streaming systems. This friendly and framework-agnostic tutorial teaches you how to handle real-time events, and even design and build your own streaming job that’s a perfect fit for your needs. Each new idea is carefully explained with diagrams, clear examples, and fun dialogue between perplexed personalities!

About the technology

Streaming systems minimize the time between receiving and processing event data, so they can deliver responses in real time. For applications in finance, security, and IoT where milliseconds matter, streaming systems are a requirement. And streaming is hot! Skills on platforms like Spark, Heron, and Kafka are in high demand.

About the book

Grokking Streaming Systems introduces real-time event streaming applications in clear, reader-friendly language. This engaging book illuminates core concepts like data parallelization, event windows, and backpressure without getting bogged down in framework-specific details. As you go, you’ll build your own simple streaming tool from the ground up to make sure all the ideas and techniques stick. The helpful and entertaining illustrations make streaming systems come alive as you tackle relevant examples like real-time credit card fraud detection and monitoring IoT services.

What's inside

Implement and troubleshoot streaming systems

Design streaming systems for complex functionalities

Spot networking bottlenecks and resolve backpressure

Group data for high-performance systems

About the reader

No prior experience with streaming systems is assumed. Examples in Java.

About the author

Josh Fischer is an Apache Committer, and part of the project management committee for the Apache Heron distributed stream processing engine. Josh is a software engineer at Scotcro and has worked with moving large datasets in real time for organizations such as 1904labs and Bayer.

Ning Wang is an Apache Committer, and part of the project management committee for the Apache Heron distributed stream processing engine. Ning is also a software engineer at Amplitude building real-time data pipelines. He was a key contributor of Apache Heron in Twitter’s Real-time Compute team.

Table of Contents

PART 1 GETTING STARTED WITH STREAMING

1 Welcome to Grokking Streaming Systems

2 Hello, streaming systems!

3 Parallelization and data grouping

4 Stream graph

5 Delivery semantics

6 Streaming systems review and a glimpse ahead

PART 2 STEPPING UP

7 Windowed computations

8 Join operations

9 Backpressure

10 Stateful computation

11 Wrap-up: Advanced concepts in streaming systems