An Analytical Tool for Georeferenced Sensor Data based on ELK Stack - INRAE - Institut national de recherche pour l’agriculture, l’alimentation et l’environnement Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

An Analytical Tool for Georeferenced Sensor Data based on ELK Stack

Résumé

In the context of the French CAP 2025 I-Site project, an environmental data lake called CEBA is built at an Auvergne regional level. Its goal is to integrate data from heterogeneous sensors, provide end users tools to query and analyse georeferenced environmental data, and open data. The sensors collect different environmental measures according to their location (air and soil temperature, water quality, etc.). The measures are used by different research laboratories to analyse the environment. The main component for data shipping and storing is the ELK stack. Data are collected from sensors through Beats and streamed by Logstash to Elasticsearch. Scientists can query the data through Kibana. In this paper, we propose a data warehouse frontend to CEBA based on the ELK stack. We as well propose an additional component to the ELK stack that operates streaming ETL which allows integrating and aggregating streaming data from different sensors and sources given the user configuration in order to provide end users more analytical capabilities on the data. We show the architecture of this system, we present the functionalities of the data lake through examples, and finally, we present an example dashboard of the data on Kibana.
Fichier principal
Vignette du fichier
paper-gistam-ngothithutrang-19.pdf (612.9 Ko) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-03313080 , version 1 (03-08-2021)

Licence

Paternité - Pas d'utilisation commerciale - Pas de modification

Identifiants

Citer

Thi Thu Trang Ngo, David Sarramia, Myoung-Ah Kang, François Pinet. An Analytical Tool for Georeferenced Sensor Data based on ELK Stack. 7th International Conference on Geographical Information Systems Theory, Applications and Management, Apr 2021, Online Streaming, France. pp.82-89, ⟨10.5220/0010439200820089⟩. ⟨hal-03313080⟩
132 Consultations
163 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More