Apache: Big Data 2015

Identifying a KV store to host terabytes of data, from wide range of choices, for a set of given use cases, is a daunting proposition. Even after crossing the fearsome first step and narrowing down the candidates, it is non-trivial to explain and reason the actual results of benchmarking experiments with the expected results. There are multiple variables at play and it is not often clear on how these interplay at run time.
In this talk, we present our experiences and methodology of how to analyze and effectively benchmark a distributed KV store. This involves monitoring and characterizing key server parameters like RAM, CPU, network, storage, page cache, IO scheduler, JVM size, GC tuning and logically reason out their effects on the overall performance and capabilities of the underlying KV store. These parameters were monitored by utilities like iostat, dstat, iftop, jstat and cachestat