Back To Schedule
Monday, September 28 • 16:00 - 16:50
Testing Big Data Pipelines Made Super Easy - Pallavi Rao, Inmobi & Pavan Kumar Kolamuri, Inmobi

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Your company has developed a system that crunches humongous data from multiple data sources. It involves multiple and varied processing modules. Each of the individual modules has been well tested. But, when you try to deploy these modules and connect them either in an integration or a staging environment, you face issues. To debug these errors and re-test them requires you to set up a mirror environment and is time consuming. So, you write complex integration tests and equally complex setup scripts. Apache Falcon and Falcon Unit to your rescue! While Apache Falcon alleviates some of the problems of pipeline orchestration, Falcon Unit, a feature of Falcon, helps users test their entire pipeline and data lifecycle without even setting up a test environment. This talk will outline the capabilities of Falcon Unit and how it helps users test data pipelines early on in the development phase.


Pallavi Rao

Pallavi is an Architect at InMobi. She has been working on big data technologies for nearly 4 years now. She has deep knowledge of the Hadoop ecosystem, especially, YARN, PIG, Oozie, HBase, Hive and Storm. Since the past 6 months she has been actively contributing to Apache Falcon... Read More →

Monday September 28, 2015 16:00 - 16:50 CEST

Attendees (0)