Information-Theoretic Bounds and Phase Transitions in Clustering, Sparse PCA, and Submatrix Localization - INRAE - Institut national de recherche pour l’agriculture, l’alimentation et l’environnement
Article Dans Une Revue IEEE Transactions on Information Theory Année : 2018

Information-Theoretic Bounds and Phase Transitions in Clustering, Sparse PCA, and Submatrix Localization

Résumé

We study the problem of detecting a structured, lowrank signal matrix corrupted with additive Gaussian noise. This includes clustering in a Gaussian mixture model, sparse PCA, and submatrix localization. Each of these problems is conjectured to exhibit a sharp information-theoretic threshold, below which the signal is too weak for any algorithm to detect. We derive upper and lower bounds on these thresholds by applying the first and second moment methods to the likelihood ratio between these "planted models" and null models where the signal matrix is zero. For sparse PCA and submatrix localization, we determine this threshold exactly in the limit where the number of blocks is large or the signal matrix is very sparse; for the clustering problem, our bounds differ by a factor of root 2 when the number of clusters is large. Moreover, our upper bounds show that for each of these problems there is a significant regime where reliable detection is information-theoretically possible but where known algorithms such as PCA fail completely, since the spectrum of the observed matrix is uninformative. This regime is analogous to the conjectured "hard but detectable" regime for community detection in sparse graphs.

Dates et versions

hal-02623774 , version 1 (26-05-2020)

Identifiants

Citer

Jess Banks, Cristopher Moore, Roman Vershynin, Nicolas Verzelen, Jiaming Xu. Information-Theoretic Bounds and Phase Transitions in Clustering, Sparse PCA, and Submatrix Localization. IEEE Transactions on Information Theory, 2018, 64 (7), pp.4872-4894. ⟨10.1109/TIT.2018.2810020⟩. ⟨hal-02623774⟩
12 Consultations
0 Téléchargements

Altmetric

Partager

More