Byte Pair Encoding for Symbolic Music - ESEO
Pré-Publication, Document De Travail Année : 2023

Byte Pair Encoding for Symbolic Music

Résumé

The symbolic music modality is nowadays mostly represented as discrete and used with sequential models such as Transformers, for deep learning tasks. Recent research put efforts on the tokenization, i.e. the conversion of data into sequences of integers intelligible to such models. This can be achieved by many ways as music can be composed of simultaneous tracks, of simultaneous notes with several attributes. Until now, the proposed tokenizations are based on small vocabularies describing the note attributes and time events, resulting in fairly long token sequences. In this paper, we show how Byte Pair Encoding (BPE) can improve the results of deep learning models while improving its performances. We experiment on music generation and composer classification, and study the impact of BPE on how models learn the embeddings, and show that it can help to increase their isotropy, i.e., the uniformity of the variance of their positions in the space.
Fichier principal
Vignette du fichier
2301.11975.pdf (16.48 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03976252 , version 1 (06-02-2023)
hal-03976252 , version 2 (09-10-2023)

Licence

Domaine public

Identifiants

Citer

Nathan Fradet, Jean-Pierre Briot, Fabien Chhel, Amal El Fallah-Seghrouchni, Nicolas Gutowski. Byte Pair Encoding for Symbolic Music. 2023. ⟨hal-03976252v1⟩
380 Consultations
45 Téléchargements

Altmetric

Partager

More