Kafka streams testing: A deep dive

Day 4 /  / Track 2  /  EN

Even though Kafka streams "come bundled" with TopologyTestDriver, which makes streaming applications testing quite enjoyable, it's not so simple: there are limitations and certain classes of defects that this technology doesn't match. For integration tests, you can check the behavior of a real cluster but it's not simple: we're entering a slippery road of asynchronous tests with timeouts, flakiness, and unsolved questions. Conclusion: you need to use both approaches + head.

Download presentation


Ivan Ponomarev

Tech lead in KURS company, tutor at MIPT. Has more than 15 years of experience in IT. Writes code and manages projects ranging from ERP systems customization to building a real-time webscraping system. Sometimes shares his experience as a speaker at conferences and meetups.

John Roesler
John Roesler
Confluent, Inc.

John Roesler is an Apache Kafka committer and PMC member and a software engineer at Confluent. He focuses on Kafka Streams. Before that, he spent eight years at Bazaarvoice, building their massive-scale, high-throughput stream processing platform.

Invited Experts

Victor Gamov

Viktor Gamov is a Developer Advocate at Confluent, the company that makes a streaming platform based on Apache Kafka.

Working in the field, Viktor Gamov developed comprehensive expertise in building enterprise application architectures using open source technologies. He enjoys helping different organizations design and develop low latency, scalable and highly available distributed systems.

Back in his consultancy days, he co-authored O'Reilly's "Enterprise Web Development".

He is a professional conference speaker on distributed systems, Java, and JavaScript topics, and is regular on events including JavaOne, Devoxx, OSCON, QCon, and others (http://lanyrd.com/gamussa). He blogs at http://gamov.io and produces the podcasts Razbor Poletov (in Russian) and co-hosts DevRelRad.io.

Follow Viktor on Twitter @gamussa, where he posts about gym life, food, open source, and, of course, Kafka and Confluent!