Storm & Kafka : serializing complex data types using Kryo

I recently had to work with Storm and Kafka, which are respectively : a framework for near real-time data processing ; a real-time pub/sub system. Both can be heavily distributed across multiple machines on a cluster.

Even if the promise of Storm combined with Kafka sounds great, things began to get more complex when I wanted to transfer non-primitive data inside and between the two systems. Continue reading