Skip to Main content Skip to Navigation
New interface
Journal articles

A “big-data” algorithm for KNN-PLS

Abstract : A well known issue regarding PLS lies in the difficulty to apprehend nonlinearities. As a solution, an extension of the method, “KNN-PLS”, was developed. However, this solution is based on a neighbourhood selection algorithm whose execution time is highly dependent on the size of the database, leading to prohibitive response times. This article proposes, as an alternative, a new method designed to process large data volumes: “parSketch-PLS”. This method combines a “big-data domain” neighbour selection method, called “parSketch”, and the PLS method. Essentially, this paper presents a feasibility study, regarding the adaptation of big-data principles for spectral datasets, in non-linear contexts. The parSketch method has not been studied in the context of chemometrics and considering the specific properties of spectral data. This method is based on the approximation of sample neighbourhoods, based on spectral distances. It is then necessary to investigate the relevance of these neighbourhoods for PLS models and predictions. This article compares PLS and KNN-PLS methods with the parSketch-PLS method. In this context, PLS allows to process large volumes of data quickly but performs poorly in prediction while the KNN-PLS method returns accurate predictions, yet with much higher computational time. This paper shows that the proposed pairing offers a good operational trade-off between prediction performances and computational cost. In addition a comprehensive study of the input parameters of parSketch-PLS is conducted. The objective is to understand the influence of these parameters on the prediction performances. This article proposes a framework to interpret the neighbourhoods returned by comparing their relative sizes with the evolution of performances and the input parameters of parSketch.
Document type :
Journal articles
Complete list of metadata

Cited literature [46 references]  Display  Hide  Download
Contributor : Christelle Raynaud Connect in order to contact the contributor
Submitted on : Friday, July 24, 2020 - 3:32:17 PM
Last modification on : Tuesday, October 25, 2022 - 4:17:45 PM
Long-term archiving on: : Tuesday, December 1, 2020 - 6:47:11 AM


A “big-data” algorithm for...
Files produced by the author(s)



Maxime Metz, Matthieu Lesnoff, Florent Abdelghafour, Reza Akbarinia, Florent Masseglia, et al.. A “big-data” algorithm for KNN-PLS. Chemometrics and Intelligent Laboratory Systems, 2020, 203, pp.104076. ⟨10.1016/j.chemolab.2020.104076⟩. ⟨hal-02899789⟩



Record views


Files downloads