PhD Defense: 'BigOPERA: An OPportunistic and Elastic Resource Allocation for Big Data Frameworks'

The increasing demand for data-intensive applications calls for more scalable, flexible, and sustainable resource management in Big Data frameworks. This thesis introduces BigOPERA, a hybrid resource allocation system integrating opportunistic computing into Apache Spark. By combining dedicated and non-dedicated resources, BigOPERA enhances elasticity and fault tolerance without requiring changes to Spark core or user applications.

The architecture leverages containerized execution and a two-tier scheduler to allocate resources dynamically based on availability. Experimental results confirm improvements in performance, efficiency, and environmental impact. BigOPERA offers a practical and sustainable extension to existing Spark deployments, paving the way for more adaptive and energy-aware cluster computing.

• Supervisors: José Carlos Cabaleiro Domínguez and Feras Mahmoud Naji Awaysheh.