BigFoot · Big Data Analytics of Digital Footprints

The amount of digital information in our world has been exploding and new technologies and services will continue to fuel exponential growth of large pools of data that can be captured, stored, and analyzed. Nowadays, however, tools and services to store, process and interact with data are still in their infancy, represented by scattered solutions that fall short in having a unified vision, that lack common interfaces, and that only offer best-effort services.<br/>The aim of BigFoot is to overcome current drawbacks by designing, implementing and evaluating a Platform-as-a-Service solution for processing and interacting with large volumes of data. The BigFoot stack -- which builds upon and contributes to the Apache Hadoop ecosystem and the OpenStack project, in addition to creating new open source components -- features automatic and self-tuned deployments of storage and processing services for private clouds, going beyond best-effort services currently available in the state-of-the-art. BigFoot takes a novel, cross-layer approach to system optimization, which is evaluated with a thorough experimental methodology using realistic workloads and datasets from two representative application, namely ICT Security and Smart Grid data analytics. In addition, BigFoot aims at making data interaction easy by supporting high-level languages (for batch oriented analytic tasks) and by taking a service-oriented approach to support and optimize latency sensitive queries.