SHK-7401 Netflix Keystone - Real-time Stream Processing Cloud Service | Devoxx US

Netflix Keystone - Real-time Stream Processing Cloud Service


bigdata Big Data

Grand Ballroom 220 C

Thursday from 10:00 AM til 10:50 AM

The need for gleaning answers from data in real-time from unbounded data streams is moving from nicety to a necessity. Netflix is a data driven company and Netflix Keystone Stream Processing Service processes over a Trillion messages a day with at-least once processing semantics in the cloud to derive insights.

To ease extracting intelligence from unbounded streams easily we are building a multi-tenant, self-serve, real-time event processing service infrastructure so that the user can focus on extracting value and not have to worry about building infrastructure. We will share our experience

  • Building the service in AWS by leveraging open source software like Kafka, Samza, Flink, Mesos, and Docker
  • How to evolve the service inflight without downtime
  • Lessons learnt along the way
Monal Daxini Monal Daxini

Monal Daxini leads the Stream Processing team at Netflix building a scalable and multi-tenant real-time event processing pipeline. He has worked on Netflix's Cassandra & Dynamite infrastructure, and was instrumental in developing the encoding compute infrastructure for Netflix content. He has 15 years of experience building distributed systems at organizations like Netflix,, and Cisco.