Learning approaches rely on hyperparameters that impact the algorithm's performance and affect the knowledge extraction process from data. Recently, Nonnegative Matrix Factoriza-tion (NMF) has attracted a growing interest as a learning algorithm. This technique cap-tures the latent information embedded in large datasets while preserving feature proper-ties. NMF can be formalized as a penalized optimization task in which tuning the penalty hyperparameters is an open issue. The current literature does not provide any general framework addressing this task. This study proposes to express the penalty hyperparam-eters problem in NMF in terms of a bi-level optimization. We design a novel algorithm, named Alternating Bi-level (AltBi), which incorporates the hyperparameters tuning proce-dure into the updates of NMF factors. Results of the existence and convergence of numer-ical solutions, under appropriate assumptions, are studied, and numerical experiments are provided.

Bi-level algorithm for optimizing hyperparameters in penalized nonnegative matrix factorization

Del Buono N.;Esposito F.;Selicato L.
;
2023-01-01

Abstract

Learning approaches rely on hyperparameters that impact the algorithm's performance and affect the knowledge extraction process from data. Recently, Nonnegative Matrix Factoriza-tion (NMF) has attracted a growing interest as a learning algorithm. This technique cap-tures the latent information embedded in large datasets while preserving feature proper-ties. NMF can be formalized as a penalized optimization task in which tuning the penalty hyperparameters is an open issue. The current literature does not provide any general framework addressing this task. This study proposes to express the penalty hyperparam-eters problem in NMF in terms of a bi-level optimization. We design a novel algorithm, named Alternating Bi-level (AltBi), which incorporates the hyperparameters tuning proce-dure into the updates of NMF factors. Results of the existence and convergence of numer-ical solutions, under appropriate assumptions, are studied, and numerical experiments are provided.
File in questo prodotto:
File Dimensione Formato  
1-s2.0-S0096300323003533-main.pdf

non disponibili

Tipologia: Documento in Versione Editoriale
Licenza: Copyright dell'editore
Dimensione 2.41 MB
Formato Adobe PDF
2.41 MB Adobe PDF   Visualizza/Apri   Richiedi una copia
2203.13129v3.pdf

accesso aperto

Tipologia: Documento in Pre-print
Licenza: Creative commons
Dimensione 970.73 kB
Formato Adobe PDF
970.73 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11586/439860
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 3
  • ???jsp.display-item.citation.isi??? 2
social impact