Apache: Big Data 2015

These days we have the tools and resources to collect and wrangle data at unprecedented scale, yet we remain plagued by compatibility gaps and semantic nuances with every new source we invite into our domain. Despite the best efforts of well meaning folks for decades, data integration remains a many-to-many problem.

Apache Streams (incubating) is an open-source real-time reference implementation for the Activity Streams specification. Streams contains libraries and patterns for specifying, publishing, and inter-linking schemas, and assists with conversion of activities and objects between the representation, format, and encoding preferred by supported data providers, processors, and indexes.

In this talk I will explain what Streams does, how it works (more or less), and how it can be used to compile a real-time, multi-network, polyglot content repository of profiles, posts, etc.