BigOPERA: An OPportunistic and Elastic Resource Allocation for Big Data Frameworks
The increasing demand for data-intensive applications calls for more scalable, flexible,
and sustainable resource management in Big Data frameworks. This thesis introduces
BigOPERA, a hybrid resource allocation system integrating opportunistic computing
into Apache Spark. By combining dedicated and non-dedicated resources, BigOPERA
enhances elasticity and fault tolerance without requiring changes to Spark core or user
applications.
The architecture leverages containerized execution and a two-tier scheduler to allocate
resources dynamically based on availability. Experimental results confirm
improvements in performance, efficiency, and environmental impact. BigOPERA offers
a practical and sustainable extension to existing Spark deployments, paving the way for
more adaptive and energy-aware cluster computing.
keywords:
Publication: Thesis
1760527633995
October 15, 2025
/research/publications/bigopera-an-opportunistic-and-elastic-resource-allocation-for-big-data-frameworks2
The increasing demand for data-intensive applications calls for more scalable, flexible,
and sustainable resource management in Big Data frameworks. This thesis introduces
BigOPERA, a hybrid resource allocation system integrating opportunistic computing
into Apache Spark. By combining dedicated and non-dedicated resources, BigOPERA
enhances elasticity and fault tolerance without requiring changes to Spark core or user
applications.
The architecture leverages containerized execution and a two-tier scheduler to allocate
resources dynamically based on availability. Experimental results confirm
improvements in performance, efficiency, and environmental impact. BigOPERA offers
a practical and sustainable extension to existing Spark deployments, paving the way for
more adaptive and energy-aware cluster computing. - Pablo Vázquez Caderno
publications_en