A reinforcement-learning algorithm for sampling design in Markov random fields - INRAE - Institut national de recherche pour l’agriculture, l’alimentation et l’environnement Accéder directement au contenu
Communication Dans Un Congrès Année : 2012

A reinforcement-learning algorithm for sampling design in Markov random fields

Résumé

Optimal sampling in spatial random fields is a complex problem, which mobilizes several research fields in spatial statistics and artificial intelligence. In this paper we consider the case where observations are discrete-valued and modelled by a Markov Random Field. Then we encode the sampling problem into the Markov Decision Process (MDP) framework. After exploring existing heuristic solutions as well as classical algorithms from the field of Reinforcement Learning (RL), we design an original algorithm, LSDP (Least Square Dynamic Programming), which uses simulated trajectories to solve approximately any finite-horizon MDP problem. Based on an empirical study of the behaviour of these different approaches on binary models, we derive the following conclusions: i) a naïve heuristic, consisting in sampling sites where marginals are the most uncertain, is already an efficient sampling approach; ii) LSDP outperforms all the classical RL approaches we have tested; iii) LSDP outperforms the heuristic in cases when reconstruction errors have a high cost, or sampling actions are constrained. In addition, LSDP readily handles action costs in the optimisation problem, as well as cases when some sites of the MRF can not be observed.
Fichier principal
Vignette du fichier
ECAI2012_BPS_1 (339.8 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-02748492 , version 1 (03-06-2020)

Identifiants

Citer

Mathieu Bonneau, Nathalie Dubois Peyrard Peyrard, Régis Sabbadin. A reinforcement-learning algorithm for sampling design in Markov random fields. 20. European Conference on Artificial Intelligence, Aug 2012, Montpellier, France. pp.1056, ⟨10.3233/978-1-61499-098-7-181⟩. ⟨hal-02748492⟩
11 Consultations
10 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More