Distributed Caching of Scientific Workflows in Multisite Cloud - Archive ouverte HAL Access content directly
Conference Papers Year : 2020

Distributed Caching of Scientific Workflows in Multisite Cloud

(1) , (2) , (1) , (3, 1, 4) , (5) , (1)
1
2
3
4
5

Abstract

Many scientific experiments are performed using scientific workflows, which are becoming more and more data-intensive. We consider the efficient execution of such workflows in the cloud, leveraging the heterogeneous resources available at multiple cloud sites (geo-distributed data centers). Since it is common for workflow users to reuse code or data from other workflows, a promising approach for efficient workflow execution is to cache intermediate data in order to avoid re-executing entire workflows. In this paper, we propose a solution for distributed caching of scientific workflows in a multisite cloud. We implemented our solution in the OpenAlea workflow system, together with cache-aware distributed scheduling algorithms. Our experimental evaluation on a three-site cloud with a data-intensive application in plant phenotyping shows that our solution can yield major performance gains, reducing total time up to 42% with 60% of same input data for each new execution.
Fichier principal
Vignette du fichier
DEXA_2020.pdf (331.79 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-02962579 , version 1 (09-10-2020)

Identifiers

Cite

Gaëtan Heidsieck, Daniel de Oliveira, Esther Pacitti, Christophe Pradal, Francois Tardieu, et al.. Distributed Caching of Scientific Workflows in Multisite Cloud. DEXA 2020 - 31st International Conference on Database and Expert Systems Applications, Sep 2020, Bratislava, Slovakia. pp.51-65, ⟨10.1007/978-3-030-59051-2_4⟩. ⟨hal-02962579⟩
112 View
137 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More