eXtream: a System for Real-time Monitoring of Dynamic Web Sources

In this work, we introduce eXtream, a Big Data platform whose main goal is to deploy modular and customisable processing topologies for massive analysis of web data in real time. The system offers a reduced group of pre-installed modules that can be easily combined in a visual way. Additionally, an advanced user can upload new modules and extend an existing topology. This tool facilitates the development of many Information Retrieval and Big Data applications, such as query-based real-time filtering or topic analysis services on Social Media data. To demonstrate it, we have also developed an initial web-based demonstrator.

keywords: Big Data, Real Time, Web Streams, Datasets