C3

Title: Matching-based Scheduling of Asynchronous Data Processing Workflows on the Computing Continuum

Heidelberg, Germany | September 6-9, 2022

Authors: Narges Mehran, Zahra Najafabadi Samani, Dragi Kimovski, Radu Prodan

Abstract: Today’s distributed computing infrastructures encompass complex workflows for real-time data gathering, transferring, storage, and processing, quickly overwhelming centralized cloud centers. Recently, the computing continuum that federates the Cloud services with emerging Fog and Edge devices represents a relevant alternative for supporting the next-generation data processing workflows. However, eminent challenges in automating data processing across the computing continuum still exist, such as scheduling heterogeneous devices across the Cloud, Fog, and Edge layers. We propose a new scheduling algorithm called C3-MATCH, based on matching theory principles, involving two sets of players negotiating different utility functions: 1) workflow microservices that prefer computing devices with lower data processing and queuing times; 2) computing continuum devices that prefer microservices with corresponding resource requirements and less data transmission time. We evaluate C3-MATCH using real-world road sign inspection and sentiment analysis workflows on a federated computing continuum across four Cloud, Fog, and Edge providers. Our combined simulation and real execution results reveal that C3-MATCH achieves up to 67% lower completion time compared to three state-of-the-art methods.

Project lead/coordination: Radu Prodan
Project partners: IDC Italia, Peracton Limited, Institut Jozef Stefan, Sintef, Universiteit Twente, metaphacts GmbH, Vrije Universiteit Amsterdam, Cineca, Event Registry, Università di Bologna, Robert Bosch GmbH

Abstract: Graph-Massivizer researches and develops a high-performance, scalable, and sustainable platform for information processing and reasoning based on the massive graph representation of extreme data. It delivers a toolkit of five open-source software tools and FAIR graph datasets covering the sustainable lifecycle of processing extreme data as massive graphs. The tools focus on holistic usability (from extreme data ingestion and massive graph creation), automated intelligence (through analytics and reasoning), performance modelling, and environmental sustainability tradeoffs, supported by credible data-driven evidence across the computing continuum. The automated operation based on the emerging serverless computing paradigm supports experienced and novice stakeholders from a broad group of large and small organisations to capitalise on extreme data through massive graph programming and processing. Graph Massivizer validates its innovation on four complementary use cases considering their extreme data properties and coverage of the three sustainability pillars (economy, society, and environment): sustainable green finance, global environment protection foresight, green AI for the sustainable automotive industry, and data centre digital twin for exascale computing. Graph Massivizer promises 70% more efficient analytics than AliGraph, and 30% improved energy awareness for ETL storage operations than Amazon Redshift. Furthermore, it aims to demonstrate a possible two-fold improvement in data centre energy efficiency and over 25% lower GHG emissions for basic graph operations. Graph-Massivizer gathers an interdisciplinary group of twelve partners from eight countries, covering four academic universities, two applied research centres, one HPC centre, two SMEs and two large enterprises. It leverages world-leading roles of European researchers in graph processing and serverless computing and uses leadership-class European infrastructure in the computing continuum.

Title: FaaScinating Resilience for Serverless Function Choreographies in Federated Clouds

Authors: Sasko Ristov, Dragi Kimovski, Thomas Fahringer

Abstract: Cloud applications often benefit from deployment on serverless technology Function-as-a-Service (FaaS), which may instantly spawn numerous functions and charge users for the period when serverless functions are running. Maximum benefit is achieved when functions are orchestrated in a workflow or function choreographies (FCs). However, many provider limitations specific for FaaS, such as maximum concurrency or duration often increase the failure rate, which can severely hamper the execution of entire FCs. Current support for resilience is often limited to function retries or try-catch, which are applicable within the same cloud region only. To overcome these limitations, we introduce rAF CL, a middleware platform that maintains the reliability of complex FCs in federated clouds. In order to support resilient FC execution under rAF CL, our model creates an alternative strategy for each function based on the required availability specified by the user. Alternative strategies are not restricted to the same cloud region, but may contain alternative functions across five providers, invoked concurrently in a single alternative plan or executed subsequently in multiple alternative plans. With this approach, rAF CL offers flexibility in terms of cost-performance trade-off. We evaluated rAF CL by running three real-life applications across three cloud providers. Experimental results demonstrated that rAF CL outperforms the resilience of AWS Step Functions, increasing the success rate of the entire FC by 53.45%, while invoking only 3.94% more functions with zero wasted function invocations. rAF CL significantly improves the availability of entire FCs to almost 1 and survives even after massive failures of alternative functions

Title: Big Data Pipelines on the Computing Continuum: Tapping the Dark Data

Authors: Dumitru Roman, Radu Prodan, Nikolay Nikolov, Ahmet Soylu, Mihhail Matskin, Andrea Marrella, Dragi Kimovski, Brian Elvesæter, Anthony Simonet-Boulogne, Giannis Ledakis, Hui Song, Francesco Leotta, Evgeny Kharlamov

Abstract: Big Data pipelines are essential for leveraging Dark Data, i.e., data collected but not used and turned into value. However, tapping their potential requires going beyond existing approaches and frameworks for Big Data processing. The Computing Continuum enables new opportunities for managing Big Data pipelines concerning efficient management of heterogeneous and untrustworthy resources. This article discusses the Big Data pipelines lifecycle on the Computing Continuum, its associated challenges and outlines a future research agenda in this area.

Title: Decentralized Machine Learning for Intelligent Health Care Systems on the Computing Continuum

Authors: Dragi Kimovski, Sasko Ristov, Radu Prodan

Abstract: The introduction of electronic personal health records (EHR) enables nationwide information exchange and curation among different health care systems. However, the current EHR systems do not provide transparent means for diagnosis support, medical research, or can utilize the omnipresent data produced by the personal medical devices. Besides, the EHR systems are centrally orchestrated, which could potentially lead to a single point of failure. Therefore, in this article, we explore novel approaches for decentralizing machine learning over distributed ledgers to create intelligent EHR systems that can utilize information from personal medical devices for improved knowledge extraction. Consequently, we proposed and evaluated a conceptual EHR to enable anonymous predictive analysis across multiple medical institutions. The evaluation results indicate that the decentralized EHR can be deployed over the computing continuum with reduced machine learning time of up to 60% and consensus latency of below 8 seconds.

The H2020 project ASPIDE impressed the reviewers with the presented scientific achievements in Exascale computing and passed the final EC review with flying colors. The project officer and reviewers echoed the effort put together by the consortium partners for publicly demonstrating and sharing the ASPIDE tools while maintaining a robust scientific output at the highest level. The EC reviewers gave very positive input on the scientific tools developed within WP3, which Klagenfurt University led.

The project “Kärntner Fog” has been accepted in the BRIDGE funding call of FFG.

Abstract: Kärntner Fog aims to contribute with advanced technologies for the distributed optimized provisioning and operation of 5G applications in Austria. For this purpose, it researches and develops a unique infrastructure testbed called the Carinthian Computing Continuum (C3), consisting of heterogeneous Cloud, Fog, and 5G‐Edge devices orchestrated through novel benchmarking, monitoring, analysis, and provisioning services. The project will validate its results using modern virtual reality and smart city use cases in the 5G Playground Carinthia. The results will give companies a competitive technological advantage in exploring 5G‐compliant applications in preparation for the deployment of an Austrian‐wide 5G network by 2025.

Partners:

Alpen-Adria Universität Klagenfurt, ITEC

Fachhochschule Kärnten

siplan gmbh

The manuscript “Mobility-Aware IoT Application Placement in the Cloud — Edge Continuum” has been accepted for publication in the A* (IF: 5.823) Journal – IEEE Transactions on Services Computing (TSC).

Autors: Dragi Kimovski, Narges Mehran, Christopher Kerth, Radu Prodan

Abstract: The Edge computing extension of the Cloud services towards the network boundaries raises important placement challenges for IoT applications running in a heterogeneous environment with limited computing capacities. Unfortunately, existing works only partially address this challenge by optimizing a single or aggregate objective (e.g., response time), and not considering the edge devices’ mobility and resource constraints. To address this gap, we propose a novel mobility-aware multi-objective IoT application placement (mMAPO) method in the Cloud – Edge Continuum that optimizes completion time, energy consumption, and economic cost as conflicting objectives. mMAPO utilizes a Markov model for predictive analysis of the Edge device mobility and constrains the optimization to devices that do not frequently move through the network. We evaluate the quality of the mMAPO placements using simulation and real-world experimentation on two IoT applications. Compared to related work, mMAPO reduces the economic cost by 28% and decreases the completion time by 80% while maintaining a stable energy consumption

Conference: https://escience2021.org/

Title: Where to Encode: A Performance Analysis of x86 and Arm-based Amazon EC2 Instances

Authors: Roland Mathá, Dragi Kimovski, Anatoliy Zabrovskiy, Christian Timmerer and Radu Prodan

Abstract: Video streaming became an undivided part of the Internet. To efficiently utilise the limited network bandwidth it is essential to encode the video content. However, encoding is a computationally intensive task, involving high-performance resources provided by private infrastructures or public clouds. Public clouds, such as Amazon EC2, provide a large portfolio of services and instances optimized for specific purposes and budgets. The majority of Amazon’s instances use x86 processors, such as Intel Xeon or AMD EPYC. However, following the recent trends in computer architecture, Amazon introduced Arm based instances that promise up to 40% better cost performance ratio than comparable x86 instances for specific workloads. We evaluate in this paper the video encoding performance of x86 and Arm instances of four instance families using the latest FFmpeg version and two video codecs. We examine the impact of the encoding parameters, such as different presets and bitrates, on the time and cost for encoding. Our experiments reveal that Arm instances show high time and cost saving potential of up to 33.63% for specific bitrates and presets, especially for the x264 codec. However, the x86 instances are more general and achieve low encoding times, regardless of the codec.

Title: Handover Authentication Latency Reduction using Mobile Edge Computing and Mobility Patterns

Authors: Fatima Abdullah, Dragi Kimovski, Radu Prodan, and Kashif Munir

Abstract: With the advancement in technology and the exponential growth of mobile devices, network traffic has increased manifold in cellular networks. Due to this reason, latency reduction has become a challenging issue for mobile devices. In order to achieve seamless connectivity and minimal disruption during movement, latency reduction is crucial in the handover authentication process. Handover authentication is a process in which the legitimacy of a mobile node is checked when it crosses the boundary of an access network. This paper proposes an efficient technique that utilizes mobility patterns of the mobile node and mobile Edge computing framework to reduce handover authentication latency. The key idea of the proposed technique is to categorize mobile nodes on the basis of their mobility patterns. We perform simulations to measure the networking latency. Besides, we use queuing model to measure the processing time of an authentication query at an Edge servers. The results show that the proposed approach reduces the handover authentication latency up to 54% in comparison with the existing approach.