Mastering Kafka Streams and Ksqldb: Building Real-Time Data Systems by Example

Working with unbounded and fast-moving data streams has historically been difficult. But with Kafka Streams and ksqlDB, building stream processing applications is easy and fun. This practical guide shows data engineers how to use these tools to build highly scalable stream processing applications for moving, enriching, and transforming large amounts of data in real time.

Mitch Seymour, data services engineer at Mailchimp, explains important stream processing concepts against a backdrop of several interesting business problems. You’ll learn the strengths of both Kafka Streams and ksqlDB to help you choose the best tool for each unique stream processing project. Non-Java developers will find the ksqlDB path to be an especially gentle introduction to stream processing.

• Learn the basics of Kafka and the pub/sub communication pattern
• Build stateless and stateful stream processing applications using Kafka Streams and ksqlDB
• Perform advanced stateful operations, including windowed joins and aggregations
• Understand how stateful processing works under the hood
• Learn about ksqlDB’s data integration features, powered by Kafka Connect
• Work with different types of collections in ksqlDB and perform push and pull queries
• Deploy your Kafka Streams and ksqlDB applications to production

Mitch Seymour is a Senior Data Systems Engineer at Mailchimp. Using Kafka Streams and KSQL, he has built several stream processing applications that process billions of events per day with sub-second latency. He is active in the open source community, has presented about stream processing technologies at international conferences (Kafka Summit London, 2019), speaks about Kafka Streams and KSQL at local meetups, and is a contributor to the Confluent blog.